Latent Space
A latent space is just the rows of a matrix, seen from two angles at once
An embedding matrix stores one high-dimensional vector per item — nothing more mysterious than a grid of numbers. This lab shows the same 12×8 matrix two ways at once: as a literal heatmap of its numbers, and as a live projection onto two directions, so nearby dots really do correspond to rows that are numerically close. Nudge one entry in the selected row and watch both views move together.Latent Lab: Projecting a High-Dimensional Matrix
Each column is an abstract coordinate, not a named trait like "size" or "color" — nothing in this dataset assigns column 3 a meaning. A cell's color is just that raw signed number at (item, coordinate): blue = positive, red = negative, intensity = magnitude. It encodes a value, not a property.
This tool takes one 12×8 matrix — 12 items, each an 8-number row — and shows it two ways at once: as a literal heatmap of its entries, and as a 2-D scatter produced by projecting every row onto two fixed directions. The sections below explain what that projection actually is, why cosine similarity drives the "nearest neighbor" readout, and where this demo's honesty limits are.
1. What a latent space actually is
An embedding matrix is nothing more exotic than rows of numbers: one high-dimensional vector per item. "Latent space" is just the name for the space those rows live in — $\mathbb{R}^8$ here, $\mathbb{R}^{768}$ or larger in a real language model. Nobody can look directly at an 8-dimensional space, let alone a 768-dimensional one, so every visualization of one is necessarily a projection down to something a human can see — 2 or 3 axes at most.
2. The data behind the demo
12 items in 3 categories — animals (cat, dog, wolf, lion), vehicles (car, bicycle, truck, airplane), fruits (apple, banana, grape, orange). Each category has one fixed 8-D prototype vector; each item's row is that prototype plus a small offset generated by a seeded pseudo-random function keyed to the item's name, so the layout is fixed and reproducible rather than random noise on every reload. This is what makes same-category rows cluster tightly and different categories separate — it is hand-built to demonstrate the mechanism, not learned from real data the way a trained embedding table would be.
3. What the projection is — and is not
The scatter plot's two axes are computed once, directly from the three category prototypes: axis 1 is the normalized difference between the animal and fruit prototypes; axis 2 is the vehicle prototype minus the average of the other two, Gram-Schmidt-orthogonalized against axis 1. Every dot's position is a genuine dot product of its 8-D row against these two directions — a real rank-2 projection, not a pre-baked (x, y) pair.
This is not PCA. Principal Component Analysis finds the two directions that maximize variance automatically, via an eigendecomposition (or SVD) of the data's covariance matrix. Here the two directions are instead read directly off the category structure that was built into the data. The visual effect — nearby points are numerically similar rows — is the same; the method for choosing the axes is simpler and specific to this dataset.
4. Reading the heatmap and the readout
The 12×8 grid below the scatter is the literal matrix: each cell is colored on a red/blue diverging scale — red for negative, blue for positive, intensity for magnitude — the standard convention for a signed heatmap. It is deliberately not teal/magenta/indigo like the category colors elsewhere in this tool, so a cell's color reads as "this number is negative or positive," never as "this row belongs to a category." The selected row is outlined so it is visible in both views simultaneously. "Nearest neighbor" is computed by cosine similarity, $\cos\theta = \dfrac{u\cdot v}{\lVert u\rVert \lVert v\rVert}$, the standard nearness measure for embeddings because it compares direction rather than raw magnitude — two rows that point the same way in latent space read as "similar" even if one happens to be a longer vector.
5. What the perturbation slider demonstrates
Picking a dimension and dragging δ adds that amount to one entry of the selected row, live. Three things move together immediately: the heatmap cell (the raw number changed), the dot on the scatter (its projection changed), and the nearest-neighbor ranking (its relationship to every other row changed). That is the concrete version of "a direction in latent space encodes a piece of information" — moving along one axis of the stored vector measurably moves what the item is closest to.
6. Honesty limit
With only 3 hand-built clusters and 8 unlabeled dimensions, nothing here justifies claiming any one dimension corresponds to a real semantic concept the way certain directions in trained word embeddings empirically turn out to (the classic king − man + woman ≈ queen result). The dimensions in this demo are deliberately left as "dim 1..8," not given invented names, because no property of this dataset earns them one. What the demo does honestly show is the mechanism — matrix rows, projection, and nearness — that makes those trained results possible to visualize in the first place.