Sign in Sign up
cargo

splintr

A high-performance tokenizer (BPE, WordPiece, SentencePiece) built with Rust with Python bindings, focused on speed, safety, and resource optimization.

cargo View on Cargo github View on Github

Activity

Latest release
2w ago
Total releases
38
Cadence
~daily
Last 12 months
38

Reach

Downloads
35.5k
Stars
61

Details

License
MIT
First release
Nov 26, 2025
Releases
Version Released
0.22.0 minor
0.21.1 patch
0.21.0 minor
0.20.0 minor
0.19.1 patch
0.19.0 minor
0.18.0 minor
0.17.0 minor
0.16.1 minor
0.15.0 minor
0.14.4 patch
0.14.3 patch
0.14.2 patch
0.14.1 patch
0.14.0 minor
0.13.0 minor
0.12.0 minor
0.11.0 minor
0.10.0 minor
0.10.0-beta.2 pre
0.10.0-beta.1 pre
0.9.1 patch
0.9.1-beta.1 pre
0.9.0 minor
0.9.0-beta.1 pre
0.8.0 minor
0.8.0-beta.2 pre
0.8.0-beta.1 pre yanked
0.7.0-beta.4 pre
0.7.0-beta.3 pre yanked
0.7.0-beta.2 pre yanked
0.7.0-beta.1 pre yanked
0.6.0 minor
0.5.0 minor
0.4.0 minor
0.3.0 initial
0.2.0-beta.3 pre
0.1.0-beta.1 pre