gpt-tokenizer
The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.
View on Github
Activity
- Latest release
- 1w ago
- Total releases
- 40
- Cadence
- ~5 days
- Last 12 months
- 8
Reach
- Downloads (recent)
- 1.2M
- Stars
- 856
Details
- License
- MIT
- First release
- Apr 16, 2023
Releases
| Version | Released | |
|---|---|---|
4.1.0
minor
| ||
4.0.4
patch
| ||
4.0.3
patch
| ||
4.0.2
patch
| ||
4.0.1
patch
| ||
4.0.0
major
| ||
3.4.0
minor
| ||
3.3.0
minor
| ||
3.2.0
minor
| ||
3.1.0
minor
| ||
3.0.1
patch
| ||
3.0.0
major
| ||
2.9.0
minor
| ||
2.8.1
patch
| ||
2.8.0
minor
| ||
2.7.0
minor
| ||
2.6.2
patch
| ||
2.6.1
patch
| ||
2.6.0
minor
| ||
2.5.1
patch
| ||
2.5.0
minor
| ||
2.4.1
patch
| ||
2.4.0
minor
| ||
2.3.0
minor
| ||
2.2.3
patch
| ||
2.2.2
patch
| ||
2.2.1
patch
| ||
2.2.0
minor
| ||
2.1.2
patch
| ||
2.1.1
patch
| ||
2.1.0
minor
| ||
2.0.0
major
| ||
2.0.0-beta.2
pre
| ||
2.0.0-beta.1
pre
| ||
1.0.5
patch
| ||
1.0.4
patch
| ||
1.0.3
patch
| ||
1.0.2
patch
| ||
1.0.1
patch
| ||
1.0.0
initial
|