Technology method · wiki:deep
MRL training
Matryoshka Representation Learning trains O(log d) nested prefix sizes so each cutoff is a valid representation — the reason OpenAI dimensions and mxbai truncate work.
MRL (Kusupati et al., NeurIPS 2022 lineage / RAIVNLab) modifies representation learning so a single d-dimensional vector encodes usable prefixes at chosen sizes M (often powers of two up to d). Losses are applied at each m ∈ M; intermediate lengths often interpolate. MRL–E ties classifier weights across prefixes for efficiency.
Product APIs do not expose the training loop; they expose the consequence: Matryoshka truncation. OpenAI text-embedding-3-* and mxbai-embed-large are deployment surfaces of that idea (vendor training details are partial — do not invent the exact nesting set).
Primary research artifacts for this cell: the MRL paper + RAIVNLab/MRL GitHub (loop wave). Sentence-Transformers documents Matryoshka loss helpers for OSS fine-tunes.
See also
- Matryoshka embeddings — product framing
- Matryoshka truncation — runtime shorten
- Adaptive / funnel retrieval — multi-m search
In this taxonomy
_catalog/taxonomy/tech-mrl-training.md