Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
D
dify
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ai-tech
dify
Commits
18af84e1
Unverified
Commit
18af84e1
authored
Jan 04, 2024
by
crazywoola
Committed by
GitHub
Jan 04, 2024
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
fix: array oob in azure openai embeddings (#1905)
parent
025b859c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
4 deletions
+4
-4
text_embedding.py
...l_providers/azure_openai/text_embedding/text_embedding.py
+4
-4
No files found.
api/core/model_runtime/model_providers/azure_openai/text_embedding/text_embedding.py
View file @
18af84e1
...
...
@@ -54,7 +54,7 @@ class AzureOpenAITextEmbeddingModel(_CommonAzureOpenAI, TextEmbeddingModel):
_iter
=
range
(
0
,
len
(
tokens
),
max_chunks
)
for
i
in
_iter
:
embeddings
,
embedding_used_tokens
=
self
.
_embedding_invoke
(
embeddings
_batch
,
embedding_used_tokens
=
self
.
_embedding_invoke
(
model
=
model
,
client
=
client
,
texts
=
tokens
[
i
:
i
+
max_chunks
],
...
...
@@ -62,7 +62,7 @@ class AzureOpenAITextEmbeddingModel(_CommonAzureOpenAI, TextEmbeddingModel):
)
used_tokens
+=
embedding_used_tokens
batched_embeddings
+=
[
data
for
data
in
embeddings
]
batched_embeddings
+=
embeddings_batch
results
:
list
[
list
[
list
[
float
]]]
=
[[]
for
_
in
range
(
len
(
texts
))]
num_tokens_in_batch
:
list
[
list
[
int
]]
=
[[]
for
_
in
range
(
len
(
texts
))]
...
...
@@ -73,7 +73,7 @@ class AzureOpenAITextEmbeddingModel(_CommonAzureOpenAI, TextEmbeddingModel):
for
i
in
range
(
len
(
texts
)):
_result
=
results
[
i
]
if
len
(
_result
)
==
0
:
embeddings
,
embedding_used_tokens
=
self
.
_embedding_invoke
(
embeddings
_batch
,
embedding_used_tokens
=
self
.
_embedding_invoke
(
model
=
model
,
client
=
client
,
texts
=
[
""
],
...
...
@@ -81,7 +81,7 @@ class AzureOpenAITextEmbeddingModel(_CommonAzureOpenAI, TextEmbeddingModel):
)
used_tokens
+=
embedding_used_tokens
average
=
embeddings
[
0
]
average
=
embeddings
_batch
[
0
]
else
:
average
=
np
.
average
(
_result
,
axis
=
0
,
weights
=
num_tokens_in_batch
[
i
])
embeddings
[
i
]
=
(
average
/
np
.
linalg
.
norm
(
average
))
.
tolist
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment