What you are looking at
Each dot is a word's vector: picture an arrow from the centre to the dot. Words with related meanings point the same way: animals up and to the right, foods down and to the right, vehicles up and to the left, programming languages down and to the left. The indigo arrow is the query. Click or drag anywhere on the plane to move it, turn and stretch it with the sliders, or pick a word to use as the query. The table ranks every word by its cosine with the query, with its dot product and distance beside it.
This is a toy. The sixteen vectors are placed by hand in two dimensions so that you can see them, and their lengths are made up. Real embedding models produce vectors with hundreds to a few thousand dimensions, and nobody places them: the model learns them in training. Every formula on this page works the same in any number of dimensions.
What an embedding is
An embedding model maps a piece of text, a word, a sentence or a whole page, to a vector of a fixed length. It is trained so that texts with related meanings get vectors that point in similar directions.
How it learns that depends on the model. Word vectors such as word2vec learned from the company each word keeps: words that turn up in similar contexts (“cat” and “dog” both appear near “pet”, “fed” and “vet”) are pushed towards similar vectors. Models that embed whole sentences and passages are usually trained on pairs that belong together, such as a question and the passage that answers it, pulling each pair together and pushing unrelated texts apart.
No single number in a vector means “animal”. Meaning is spread across all the dimensions, and what you can use is how vectors relate to each other. Pick dog as the query: its nearest neighbours are the other animals, the foods come next, and the programming languages, pointing the opposite way, come last.
Cosine, dot product and distance
Three ways to say how close two vectors are:
- Cosine similarity is the cosine of the angle between them,
a · b ÷ (|a| |b|). It is 1 when they point the same way, 0 at right angles and −1 when they point opposite ways, and it ignores length completely. - The dot product,
a · b = |a| |b| cos θ, is the cosine times both lengths. A long vector scores higher with everything it roughly points at. - Euclidean distance,
|a − b|, is the straight line between the two tips. It depends on both lengths and on the angle.
They can disagree. With the starting query, the cosine picks cat, but the dot product picks tiger, whose vector is more than twice as long. Now drag the length slider. The cosines don't move. Every dot product changes by the same factor, so their order stays put too. Distance is the one that moves: as the query grows from 0.5 to 1.5, its nearest tip goes from cat to dog to tiger.
Then press Normalise. Every vector is scaled to length 1 and slides onto the dashed circle. Now the dot product is the cosine, and distance follows from it, since |a − b|² = 2 − 2 cos θ for vectors of length 1. All three rank the neighbours the same way. That is why the cosine, which ignores length, is the usual choice for comparing meanings, and why many embedding models return vectors already normalised: search can then use the plain dot product and skip the division.
Retrieval in RAG
Retrieval-augmented generation answers questions from your own documents, and its retrieval step is this lab with more words:
- Split the documents into chunks of a few hundred tokens and embed each chunk once. Store each vector with its text.
- When a question comes in, embed it with the same model.
- Find the k chunks whose vectors are nearest the question's, by cosine, or by the dot product on normalised vectors. That is the k slider.
- Put those chunks in the prompt and ask the model to answer from them.
The right chunk often shares few words with the question. What they share is meaning, and that is what the vectors capture.
Comparing the question with every stored vector is exact, and fast enough for tens of thousands of chunks. At millions it gets slow, so vector search uses approximate nearest neighbour indexes, which give up a little accuracy for a lot of speed. HNSW (hierarchical navigable small world) links each vector to a few of its neighbours, in layers that get sparser towards the top. A search starts at the top, hops towards the query, drops a layer and repeats, so it visits a small share of the vectors and usually finds most of the true nearest ones; its settings trade speed against how many it finds. IVF (inverted file) indexes take another route: they group the vectors into clusters and search only the clusters nearest the query.
Three things often go wrong:
- Two models. Documents and questions must be embedded by the same model. Vectors from different models live in different spaces, so comparing them means nothing, even when they have the same number of dimensions. Changing the model means embedding every document again.
- Chunks that are too long. One vector has to stand for the whole chunk, so a chunk about three things points somewhere between them and is close to none of them. Drag the query into the gap straight above the centre, between the animals and the vehicles: its best cosine falls to 0.8, and its nearest words come from both groups. Chunks that are too short lose the context that makes them findable.
- Nearest is not the same as relevant. A search returns k results even when none of them answers the question. Set a minimum score, and choose it from your own data: in some models even unrelated texts score well above 0.
In an interview
Questions this lab prepares you for:
- What is an embedding, and why do texts with similar meanings end up close together?
- Cosine similarity or dot product: when do they rank results the same way? (When every vector has been normalised to length 1.)
- How does the retrieval step of a RAG system work, from the documents to the prompt?
- How would you search 100 million vectors quickly? (An approximate index such as HNSW or IVF, the trade between speed and how many true neighbours it finds, and splitting the index across machines.)
- Your RAG system keeps retrieving the wrong chunks. What do you check? (The same model on both sides, the chunk size, k, whether the answer is in the documents at all, and whether keyword search alongside the vectors would catch exact terms such as product codes.)
The code
Cosine similarity and top-k in NumPy, with six of the lab's words. Normalise every vector once, and one matrix product scores them all. It prints the same top three the lab starts with: cat, dog and tiger.
import numpy as np def normalise(vectors): lengths = np.linalg.norm(vectors, axis=-1, keepdims=True) return vectors / np.maximum(lengths, 1e-12) def top_k(query, vectors, k=3): scores = normalise(vectors) @ normalise(query) best = np.argsort(-scores)[:k] return best, scores[best] words = ["horse", "dog", "cat", "tiger", "car", "bus"]vectors = np.array([ [1.20, 0.37], [0.83, 0.46], [0.45, 0.39], [0.84, 1.12], [-0.57, 0.49], [-1.18, 0.65],]) best, scores = top_k(np.array([0.39, 0.31]), vectors)print([words[i] for i in best], scores.round(3))