Learning Transformer

About

A Transformer, built by hand and shown one step at a time. Twenty-three screens follow "Attention Is All You Need" (Vaswani et al., 2017) in the paper's own order: a word becoming numbers, the three vectors, the scores, the softmax, four heads merged into one, the mask, the second attention, and the loop that writes one word at a time. What this is not: a trained model. Every weight here was set by hand, so that each one can be read in a single line. Nine numbers stand for a word instead of 512. One encoder layer and one decoder layer instead of six of each. Four heads, four words of vocabulary, one short sentence. What it is: the paper's arithmetic at a size you can check by hand. Every number on every screen is computed while you look at it — no videos, no quizzes, no account, no network. Steps 1 to 10 are free. Steps 11 to 23 are one purchase, made once and kept.