py39-tokenizers

v 0.13.3 Updated: 1 year, 2 months ago

Fast and Customizable Tokenizers

Tokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

https://github.com/huggingface/tokenizers

To install py39-tokenizers, paste this in macOS terminal after installing MacPorts

sudo port install py39-tokenizers

Add to my watchlist

Installations 0
Requested Installations 0