Tokenizer visualizer
See how text splits into BPE tokens using real OpenAI encodings. Each span shows the token slice; hover for the numeric ID.
Your text never leaves the browser — no upload, no server. Prompt text is not stored in the URL.
Encoding
Model caveat:Used by: . Token counts are exact for this encoding, not a rough char/4 estimate.
Loading encoding and tokenizing…
Stats
Visual
Enter text above to see colored token spans.
Token IDs
About this tool
LLM APIs bill and truncate by tokens, not characters. BPE (byte-pair encoding) merges frequent byte sequences into token IDs. The same word can split differently across encodings or even positions in a prompt.
This tool uses the gpt-tokenizer library with real rank tables — not a chars÷4 guess. Pick the encoding that matches your model family; when unsure, o200k_base covers GPT-4o and newer chat models.
Read more
- Inference cost calculator — monthly LLM bills by workload
- Token counter & prompt cost — compare input cost across models
- Context window fit checker — will your prompt fit?
- All AI posts