The Generation Loop
01.04 · Concept
Logits
Read the raw logit vector from a forward pass, understand how it maps vocabulary to scores, and trace the unembedding step.
Logits are the pre-softmax vocabulary scores produced from the final hidden state by the unembedding projection. Each token id gets one score, which later processing may mask, rescale or filter before sampling. Reading them correctly means knowing where the model ends, where the sampler begins, and which token position is being scored.
What this lesson answers
- what are logits in transformer text generation
- how hidden states become vocabulary token scores
- where logits sit before softmax and sampling
Notes
Logits are the raw, unnormalized scores over the vocabulary produced by the model at each generation step: if the final hidden state is and the vocabulary embedding/unembedding matrix is , then , where is the score for token id .
Common questions
- Are logits the same as probabilities?
- No. Logits are raw scores, not normalised probabilities. A higher logit means the token is preferred relative to tokens with lower logits, but the values do not sum to anything meaningful. Softmax converts the logit vector into a probability distribution, usually after temperature, masking or other generation controls have been applied.
- Why does generation usually use only the last hidden state?
- For next-token generation, the model needs the distribution for the token after the current sequence. That distribution is derived from the hidden state at the final position. Earlier positions may have logits during a full forward pass, but they are not the ones used to choose the next generated token.
- What does the unembedding layer do?
- The unembedding layer maps the model’s final hidden vector back into vocabulary space. Conceptually, it compares that hidden representation against learned token directions and produces one score per token id. In many language models, the input embedding weights and output unembedding weights are shared or closely related.
Short definition: what is Logits?