Thanks to visit codestin.com
Credit goes to github.com

Skip to content

support for dimensions field like in OpenAI text-embedding-3, thanks #476

@ericg108

Description

@ericg108

Feature request

when initializing with SentenceTransformers, we can use the truncate_dim argument, like below:
model = SentenceTransformer("mixedbread-ai/mxbai-embed-large-v1", truncate_dim=dimensions)

and in calling OpenAI text-embedding-3, we can also pass a `` argument to get variant-length embeddings

dimensions integer Optional The number of dimensions the resulting output embeddings should have. Only supported in text-embedding-3 and later models.
see also: https://platform.openai.com/docs/api-reference/embeddings/create#embeddings-create-dimensions

Motivation

more and more embedding models are supporting Matryoshka embeddings, namely allowing users to get dimensions of varying length, like mxbai-embed-large-v1, jina-embeddings-v3 etc.
this is very useful in scenarios with limited resources. hope it could be supported. Thanks.

Your contribution

I guess it's not a big modification. I may be able to add this feature when I'm told where to modify. Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions