Matryoshka truncation

Matryoshka truncation is the deployment move: given a vector trained so prefixes are meaningful, use only length m.

OpenAI — defaults vs emit length

• text-embedding-3-small: default 1536. Supports dimensions → can return 1024, 512, …
• text-embedding-3-large: default 3072. Supports dimensions → can return 1024, 256, …
• Both are Matryoshka-capable in the API sense: the parameter shortens the sequence (conceptually prefix retention).
• text-embedding-ada-002: fixed 1536 — no Matryoshka dimensions knob.

Prefer setting dimensions when calling the Embeddings API. OpenAI notes that manually slicing a longer vector requires normalizing the result. The API may accept many integers; treat “best” cutoffs as those the model was trained/evaluated on (public MTEB tables show strong scores at shortened sizes such as 256 / 1024 for 3-large — not a substitute for measuring on your corpus).

mxbai — native 1024, truncate down

mixedbread-ai/mxbai-embed-large-v1 produces 1024-d embeddings and documents Matryoshka truncation (e.g. dimensions=512 or Sentence-Transformers truncate_dim=512). Same idea as OpenAI’s knob: one model, multiple usable lengths. Binary quantization is optional and orthogonal (Embedding quantization).

Why 1536 and 1024 show up in the same conversation

People say “OpenAI is 1536” because that is 3-small’s default (and ada-002’s only size). People say “mxbai is 1024” because that is mxbai-large’s native size. With Matryoshka truncation, OpenAI 3-small and 3-large can also emit 1024 — so 1024 is a shared operating point across vendors when you choose it, not a different species of embedding.

Index contract

Whatever m you choose, the Vector DBs collection must be created for that dimension. Mixing 1536 and 1024 in one index fails. Funnel retrieval (Adaptive / funnel retrieval) stores or computes multiple prefixes deliberately.