What is the capital of Ontario?

Verified

The prompt is seven tokens under o200k_base, checked with tiktoken rather than assumed.

Toronto is a single token, id 151837. With a leading space it is 27812. The period is 13. Its bytes are 54 6f 72 6f 6e 74 6f.

Toronto is in fact the capital of Ontario.

Deliberately not measured

The probability bars carry no numeric labels. Those values were never measured, so the panel shows relative height only and says so on screen.

The embedding strip is labelled schematic.

Architecture is given as a range — 32 to 80 blocks — so no single vendor's configuration is implied.

Speculative decoding is not a stage

It is not a step between softmax and de-tokenizing. It wraps the whole forward-pass loop, which is why scene seven is drawn as a bracket around stages 02 to 04 rather than a box after them.

Position does not enter at embedding

GPT-2 added learned positional embeddings at that layer. Llama, Qwen and Mistral apply RoPE to Q and K inside each attention layer. The embedding scene leaves that slot visibly empty and the next one fills it.

The block is stacked, not singular

There is no one transformer block. There are dozens, identical, each doing the same thing to a slightly different signal — drawn with the residual spine that runs through all of them.