py313-sentencepiece (python/py-sentencepiece) Updated: 3 days, 5 hours ago Add to my watchlist

Python wrapper for sentencepiece

SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training. SentencePiece implements subword units (e.g., byte-pair-encoding (BPE) [Sennrich et al.]) and unigram language model [Kudo.]) with the extension of direct training from raw sentences. SentencePiece allows us to make a purely end-to-end system that does not depend on language-specific pre/postprocessing.

Version: 0.2.1 License: Apache-2 GitHub
Reset

21 build(s) found
Builder Build Number Start Time Elapsed Time Watcher Build Status
10.6 228335 2026-03-24 6:18:14 0:00:24 78909 failed install-dependencies
14 78739 2026-03-22 12:02:38 0:00:33 29226 build successful
15 52224 2026-03-21 19:24:26 0:00:34 16719 build successful
14.arm64 73459 2026-03-21 19:15:52 0:00:17 28792 build successful
13.arm64 100078 2026-03-21 19:09:17 0:00:22 40765 build successful
12 152509 2026-03-21 16:49:26 0:01:24 55013 build successful
10.13 280878 2026-03-21 16:36:54 0:01:29 96444 build successful
10.11 315521 2026-03-21 16:30:43 0:01:02 105937 build successful
10.10 324365 2026-03-21 16:28:43 0:01:06 105638 build successful
10.15 230587 2026-03-21 16:26:44 0:01:16 76169 failed install-port
10.12 325868 2026-03-21 16:26:41 0:01:09 104794 build successful
11 204255 2026-03-21 16:26:37 0:01:44 66433 build successful
10.9 332775 2026-03-21 16:24:56 0:01:01 105531 build successful
10.6 227676 2026-03-21 16:24:03 0:00:12 78713 failed install-dependencies
10.14 253685 2026-03-21 16:23:39 0:01:30 87044 failed install-port
10.6.x86_64 265259 2026-03-21 16:19:18 0:00:08 78789 failed install-dependencies
10.8 231680 2026-03-21 16:18:56 0:00:13 76158 failed install-dependencies
10.7 246565 2026-03-21 16:17:05 0:00:07 78044 failed install-dependencies
13 105106 2026-03-21 16:15:08 0:00:40 40688 build successful
15.arm64 39842 2026-03-21 16:14:41 0:00:19 15881 build successful
26.arm64 30668 2026-03-21 16:14:28 0:00:23 5617 build successful