Skip to content
Sonic
AI
Sonic
AI
Home
Discover
Library
Ask Sonic
Projects
Use with Claude or ChatGPT
Show me around
Request source or feature
OpenAI's current tokenizer has a vocabulary of about 200,000 tokens., Sonic AI
Back
“OpenAI's current tokenizer has a vocabulary of about 200,000 tokens.”
Noufar Gaspar
LLMs
Loading full analysis…