Sign in Sign up
cargo

splintr

A high-performance tokenizer (BPE, WordPiece, SentencePiece) built with Rust with Python bindings, focused on speed, safety, and resource optimization.

cargo View on Cargo github View on Github

Activity

Latest release
5d ago
Total releases
35
Cadence
~daily
Last 12 months
35

Reach

Downloads
20.2k
Stars
60

Details

License
MIT
First release
Nov 26, 2025
Releases
Version Released
0.20.0 minor
0.19.1 patch
0.19.0 minor
0.18.0 minor
0.17.0 minor
0.16.1 minor
0.15.0 minor
0.14.4 patch
0.14.3 patch
0.14.2 patch
0.14.1 patch
0.14.0 minor
0.13.0 minor
0.12.0 minor
0.11.0 minor
0.10.0 minor
0.10.0-beta.2 pre
0.10.0-beta.1 pre
0.9.1 patch
0.9.1-beta.1 pre
0.9.0 minor
0.9.0-beta.1 pre
0.8.0 minor
0.8.0-beta.2 pre
0.8.0-beta.1 pre yanked
0.7.0-beta.4 pre
0.7.0-beta.3 pre yanked
0.7.0-beta.2 pre yanked
0.7.0-beta.1 pre yanked
0.6.0 minor
0.5.0 minor
0.4.0 minor
0.3.0 initial
0.2.0-beta.3 pre
0.1.0-beta.1 pre