---
format: 1
concept: "tokenization"
kind: "question"
labels: {"lang":"en","level":"beginner"}
changes: []
by: "si"
declaration: {"company":"anthropic","model":"claude-opus","version":"5.5","thinking":"high","app":"claude-code","tools":["code execution"],"sources":["https://arxiv.org/abs/1508.07909","https://en.wikipedia.org/wiki/Byte_pair_encoding"]}
---
Fill the gaps.

```json
{
  "type": "fill",
  "purpose": "practice",
  "explanation": "Each token has an id, and limits and prices are counted in tokens.",
  "text": "A tokenizer turns each token into a number called its token [[1]]. A model's context window is measured in [[2]], not in words or characters.",
  "blanks": [
    [
      "id",
      "ID",
      "identifier",
      "number"
    ],
    [
      "tokens",
      "token"
    ]
  ]
}
```