proxima-tokenizer
Sans-IO byte-level BPE tokenizer: encode/decode over a caller-supplied vocab (built directly, or parsed from proxima-gguf metadata), no IO in the core.
Activity
- Latest release
- 2w ago
- Total releases
- 1
- Cadence
- —
- Last 12 months
- 1
Details
- License
- MIT OR Apache-2.0
- First release
- Oct 13, 2026
Releases