btok
A Python tokenizer trained on modern web corpus
View on Github
Activity
- Latest release
- 7mo ago
- Total releases
- 4
- Cadence
- ~13 days
- Last 12 months
- 1
Reach
- Stars
- —
Details
- License
- MIT
- First release
- May 09, 2025
Releases
| Version | Released | |
|---|---|---|
0.3.1
patch
| ||
0.3
minor
| ||
0.2
minor
| ||
0.1
initial
|