How to learn How LLMs work

From text to the next word: how a large language model reads, pays attention and writes.

2 of 8 concepts ready to learn · Markdown, for your AI

1. Text to numbers

1. Tokenization: How text is cut into the pieces a language model reads, and why those pieces are usually parts of words.
2. Embeddings: How a model turns each token into a list of numbers, so that related meanings end up close together.
3. Positional encoding (planned)

2. Attention

1. Self-attention (planned)
2. Multi-head attention (planned)

3. The transformer

1. The transformer block (planned)
2. Predicting the next token (planned)
3. Sampling and temperature (planned)