02.01 · Lecture · Free
Embeddings: Matching a Page That Shares No Words
Explain how text becomes a vector, why cosine similarity retrieves a relevant page containing none of the query's words, and what that model is blind to.
Curated for this lesson
How AI Answers Are Built
Stanford CS224N: NLP with Deep Learning | Winter 2021 | Lecture 1 - Intro & Word Vectors
Canonical university lecture directly on word vectors, suitable for explaining semantic similarity before modern retrieval.
Embeddings turn text into numeric vectors, placing related meanings near each other even when the wording differs. Cosine similarity compares vector direction, so a query can retrieve a page with no shared terms. The trade-off is lossy matching: exact constraints, negation, numbers, recency and domain-specific details can be missed.
What this lesson answers
- how do embeddings match pages without shared words
- what does cosine similarity mean in search
- when should I distrust embedding retrieval
Notes
An embedding is a way to turn a piece of text into a list of numbers so that similar meanings land near each other in that numerical space. The model learns these numbers from patterns in text: words and passages used in similar contexts tend to get similar vectors. Once a query and a page are both vectors, a search system can compare the direction of those vectors rather than looking only for shared words.
Cosine similarity is the common “are these pointing the same way?” comparison. If a query says “reset my password” and a page says “recover account access,” keyword search may miss it because the words differ. Embedding search can still retrieve it because both texts point toward a similar meaning in vector space. The mental model is a map of meanings, not a dictionary lookup.
The important misconception is that embeddings understand text like a person. They do not. They compress text into patterns learned from examples, so they can miss exact constraints, numbers, negation, chronology, permissions, and rare domain-specific meanings. A page can be “near” the query in general topic while still being the wrong answer in detail.
After this lesson, the student should be able to describe the retrieval pipeline: split pages into chunks, embed each chunk, embed the user query, compare vectors with cosine similarity, and return the nearest chunks. They should also know when to distrust it: when exact wording, legal conditions, code behavior, fresh facts, or subtle negatives matter.
References
Common questions
- How can a search engine match text with no shared words?
- It embeds both the query and the page into vectors trained from language patterns. Texts used in similar contexts end up pointing in similar directions. A search system can then compare those directions rather than matching terms literally, so different phrasing can still map to the same underlying intent.
- What is cosine similarity doing in semantic search?
- Cosine similarity measures whether two vectors point in a similar direction. For text retrieval, that direction acts as a rough representation of meaning. If a query vector and a page vector align closely, the page is treated as semantically relevant even if the actual words are different.
- What are embeddings bad at for retrieval?
- Embeddings are weak when the answer depends on precise details rather than broad meaning. They can blur negation, numeric limits, permissions, legal wording, code semantics, chronology and rare specialist terms. A retrieved chunk may be topically close while still failing the exact requirement in the query.
Short definition: what is Embeddings: Matching a Page That Shares No Words?