Sign in Sign up
npm

tokenizers-node

Provides an implementation of today's most used tokenizers, with a focus on performances and versatility.

Latest release
Mar 15, 2024
Releases
3
Known CVEs
0
First release
Mar 15, 2024
License
Apache-2.0
View on Npm
Repository

Source

huggingface/tokenizers
Stars
—
Forks
—
Open issues
—

Security score

No OpenSSF Scorecard available for this repository.

Packages from this repo

Insights

Activity

Total releases
3
Last 12 months
0
Cadence
~daily
Dependencies
13

Releases per month

last 12 months

Release mix

  • pre 3
3 releases
Dependencies

Depends on

0.14.2-dev0
  • npm tokenizers-node-android-arm64 0.14.2-dev0 optional
  • npm tokenizers-node-android-arm-eabi 0.14.2-dev0 optional
  • npm tokenizers-node-darwin-arm64 0.14.2-dev0 optional
  • npm tokenizers-node-darwin-x64 0.14.2-dev0 optional
  • npm tokenizers-node-freebsd-x64 0.14.2-dev0 optional
  • npm tokenizers-node-linux-arm64-gnu 0.14.2-dev0 optional
  • npm tokenizers-node-linux-arm64-musl 0.14.2-dev0 optional
  • npm tokenizers-node-linux-arm-gnueabihf 0.14.2-dev0 optional
  • npm tokenizers-node-linux-x64-gnu 0.14.2-dev0 optional
  • npm tokenizers-node-linux-x64-musl 0.14.2-dev0 optional
1–10 of 13

Used by

Nothing tracked depends on this yet.

Releases
Version Released
0.14.2-dev0 pre
0.14.1-dev0 pre
0.14.0-dev0 pre