py39-tokenizers (python/py-tokenizers) Updated: 1 month ago Add to my watchlist

Fast and Customizable Tokenizers

🤗 Tokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Version: 0.12.1 License: Apache-2 GitHub
Maintainers No Maintainer
Categories python
Homepage https://github.com/huggingface/tokenizers
Platforms darwin
Variants
  • universal (Build for multiple architectures)

Subport(s) (4)


"py39-tokenizers" depends on

lib (2)
build (7)

Ports that depend on "py39-tokenizers"



Port Health:

Loading Port Health

Installations (30 days)

3

Requested Installations (30 days)

1