language-tokenizer
Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.
View on Github
Activity
- Latest release
- 2w ago
- Total releases
- 7
- Cadence
- ~25 days
- Last 12 months
- 7
Reach
- Downloads
- 87.9k
- Stars
- 2
Details
- License
- WTFPL
- First release
- Jan 07, 2026
Releases