How LLMs Actually Work

A walkthrough of the transformer from input string to predicted token, pitched at someone who wants the mechanism rather than the metaphor. It takes in tokenisation and embedding matrices, how attention lets tokens exchange information, and why residual connections and layer normalisation are what make a deep stack trainable at all. The closing observation is the useful one: model families differ in their data, their scale and their post-training, and hardly at all in the architecture underneath.

Visit Original Article →