Embedding: what a meaning vector actually measures

An embedding is, literally, a vector: a list of decimal numbers. None of those numbers has a reading of its own. Dimension 431 does not mean «formality» or «legal subject matter», and looking for individual meaning in it is wasted time. All that counts is relative position. Two pieces of content with similar meanings end up close together in that space and two unrelated ones end up far apart, so the distance between two vectors measures their relatedness: small distances indicate high relatedness.

Two limits follow from this, and they are worth fixing before going further. The first is that an embedding does not store the text. It is a lossy representation, made for comparing and not for reconstructing. The second causes the most confusion in reporting: an embedding measures similarity of meaning, not truth and not quality. Two sentences can sit right next to each other in vector space with one correct and the other false, because «the drug reduces the risk» and «the drug does not reduce the risk» are about exactly the same thing. Negation barely moves the vector and yet reverses the sense.

That is why a high similarity score answers the question «is this about the same thing?» and never the question «is this true?». Any dashboard presenting cosine as an editorial quality indicator is simply renaming one measurement as another.

The usual measure is cosine similarity, which compares the direction of two vectors and discards their length. OpenAI recommends it explicitly and notes that its vectors come out normalised, so that the dot product gives the same result slightly faster and Euclidean distance produces exactly the same ordering. The Gemini documentation explains it along the same lines: what counts is direction rather than magnitude, because direction reflects conceptual closeness more accurately.

Dimensionality is the number of values in the list. Current models work at known scales: 1,536 values in text-embedding-3-small and 3,072 in text-embedding-3-large, while Gemini allows between 128 and 3,072 and recommends 768, 1,536 or 3,072. In practice that figure says how much nuance fits and what it costs to store and compare it. More dimensions retain finer distinctions and take up more memory; fewer dimensions make the index cheaper and the search faster at the cost of nuance.

The drop-off is not proportional, and that is the useful part. OpenAI measured that its large model, shortened to 256 values, still beat the previous model at 1,536 on the MTEB benchmark, so shortening turns out cheaper than intuition suggests. One operational detail is frequently overlooked: some APIs distinguish whether incoming text is a query or a document from the corpus, and using the wrong type degrades retrieval without issuing any warning.

For anyone deciding a content budget, the practical consequence is a shift of leverage. When retrieval works by meaning, repeating the exact keyword variant stops being the mechanism that makes a text findable, because the system recognises synonyms and different phrasings on its own. Google says so plainly in its guidance on generative search: you do not need to write in a specific way just for AI, since its systems understand synonyms and the general meaning of what someone is looking for.

The second consequence is a limit that is almost never stated out loud. Good vector similarity puts your text among the retrieved candidates. It does not decide that it gets cited, that it gets believed, or that it is correct. Above retrieval sit other ranking and filtering systems, and fact checking is not among the capabilities of a vector.

Translated into concrete decisions: it makes sense to invest in covering a topic precisely and calling things by their name, and it makes no sense to buy a tool promising to «optimise your embeddings» for a search engine whose model nobody knows from the outside. And no similarity figure should ever serve as editorial quality control.

• Write about the topic, not about the keyword variant. Meaning-based retrieval rewards covering a subject well, and mechanical repetition no longer brings the advantage it used to.
• Name the subject, the product, the place and the date explicitly. A text full of implicit references produces a blurred vector that is hard to retrieve.
• Combine vector search with lexical search if you run your own index. In Anthropic's evaluation, adding exact matching to embeddings brought failures down from 5.7 % to 2.9 %, and it helps above all with codes, references and rare proper names.
• Choose dimensionality by budget and then verify it. Shortening usually costs less quality than expected, but that is confirmed with your own test set and not by analogy.
• Declare the task type when the API offers one, separating query from document. It is free and it prevents a silent loss of precision.
• Verify facts outside the vector. Similarity tells you what a text is about; whether it is true is told by a source, not by a distance.

• Reading cosine as a quality grade. This is the costliest confusion in the field, because it measures closeness of meaning and nothing else.
• Comparing similarity values across different models. Each model has its own scale, and 0.82 in one does not equal 0.82 in another.
• Mixing vectors from different models or versions in the same index. Distances stop meaning anything and the search returns random results that look like legitimate ones.
• Switching models without reindexing the entire corpus. The old vectors are not compatible with the new queries, even though the system keeps answering without raising an error.
• Expecting the vector to catch a negation or a contradiction. A sentence and its opposite land very close together in the space, so that filter has to live elsewhere in the system.