tiktoken_ruby
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs.
View on Github
Activity
- Latest release
- 1mo ago
- Total releases
- 19
- Cadence
- ~43 days
- Last 12 months
- 6
Reach
- Stars
- —
Details
- License
- MIT
- First release
- Mar 19, 2023
Releases
| Version | Released | |
|---|---|---|
0.0.17
patch
| ||
0.0.16
patch
| ||
0.0.15.1
patch
| ||
0.0.15
patch
| ||
0.0.14.1
patch
| ||
0.0.13
patch
| ||
0.0.12
patch
| ||
0.0.11
patch
| ||
0.0.11.1
patch
| ||
0.0.10
patch
| ||
0.0.9
patch
| ||
0.0.8
patch
| ||
0.0.7
patch
| ||
0.0.6
patch
| ||
0.0.5
patch
| ||
0.0.4
patch
| ||
0.0.3
patch
| ||
0.0.1
initial
| ||
0.0.2
patch
|