@dkhokhlov/tokenizers
Provides an implementation of today's most used tokenizers, with a focus on performances and versatility.
- Latest release
- Nov 24, 2025
- Releases
- 1
- Known CVEs
- 0
- First release
- Nov 24, 2025
- License
- Apache-2.0
Repository
Source
- Stars
- —
- Forks
- —
- Open issues
- —
Security score
No OpenSSF Scorecard available for this repository.
Packages from this repo
Insights
Activity
- Total releases
- 1
- Last 12 months
- 1
- Cadence
- —
- Dependencies
- 0
Releases per month
last 12 monthsRelease mix
- pre 1
1
releases