py311-tokenizers (python/py-tokenizers) Updated: 3 weeks, 1 day ago Add to my watchlist
Fast and Customizable TokenizersTokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility. Includes BPE, WordPiece, and Unigram tokenizer implementations.
Version: 0.22.2 License: Apache-2
GitHub
78 build(s) found
| Builder | Build Number | Start Time | Elapsed Time | Watcher | Build Status |
|---|---|---|---|---|---|
| 10.14 | 255667 | 2026-04-03 6:55:48 | 0:05:11 | 87735 | build successful |
| 10.15 | 231945 | 2026-04-02 19:27:56 | 0:05:30 | 76810 | build successful |
| 10.13 | 282449 | 2026-04-01 9:33:47 | 0:00:09 | 97035 | failed install-dependencies |
| 10.8 | 232826 | 2026-03-27 5:18:09 | 0:00:09 | 76520 | failed install-dependencies |
| 10.6.x86_64 | 266097 | 2026-03-26 17:07:12 | 0:00:12 | 79128 | failed install-dependencies |
| 10.11 | 316226 | 2026-03-25 23:50:19 | 0:00:15 | 106225 | failed install-dependencies |
| 10.10 | 325467 | 2026-03-25 23:44:06 | 0:00:08 | 105941 | failed install-dependencies |
| 10.11 | 316180 | 2026-03-25 22:08:08 | 0:00:07 | 106213 | failed install-dependencies |
| 10.7 | 247596 | 2026-03-25 17:24:03 | 0:00:13 | 78335 | failed install-dependencies |
| 10.7 | 247559 | 2026-03-25 16:37:29 | 0:00:08 | 78323 | failed install-dependencies |
| 10.6 | 228686 | 2026-03-25 6:01:15 | 0:00:29 | 78979 | failed install-dependencies |
| 10.9 | 333679 | 2026-03-25 3:35:20 | 0:00:09 | 105776 | failed install-dependencies |
| 10.9 | 333657 | 2026-03-25 3:18:51 | 0:00:08 | 105775 | failed install-dependencies |
| 10.12 | 326543 | 2026-03-25 0:34:54 | 0:00:09 | 105033 | failed install-dependencies |
| 10.6 | 228346 | 2026-03-24 6:46:25 | 0:00:37 | 78910 | failed install-dependencies |
| 10.6.x86_64 | 265467 | 2026-03-22 13:20:56 | 0:00:12 | 78864 | failed install-dependencies |
| 10.12 | 326039 | 2026-03-22 3:50:04 | 0:00:09 | 104859 | failed install-dependencies |
| 10.6 | 227876 | 2026-03-22 1:29:15 | 0:00:23 | 78762 | failed install-dependencies |
| 10.8 | 231294 | 2026-03-20 23:52:23 | 0:00:11 | 76115 | failed install-dependencies |
| 10.9 | 331989 | 2026-03-19 22:06:09 | 0:00:09 | 105303 | failed install-dependencies |
| 13.arm64 | 99617 | 2026-03-18 6:11:01 | 0:01:27 | 40569 | build successful |
| 10.12 | 325234 | 2026-03-18 2:36:38 | 0:00:18 | 104538 | failed install-dependencies |
| 10.6.x86_64 | 264476 | 2026-03-17 21:37:20 | 0:00:13 | 78515 | failed install-dependencies |
| 15 | 51304 | 2026-03-17 13:45:11 | 0:01:54 | 16515 | build successful |
| 14.arm64 | 72940 | 2026-03-16 23:32:30 | 0:01:58 | 28662 | build successful |
| 10.6 | 226714 | 2026-03-16 17:59:08 | 0:00:23 | 78431 | failed install-dependencies |
| 10.11 | 314789 | 2026-03-16 15:13:44 | 0:00:10 | 105768 | failed install-dependencies |
| 26.arm64 | 30010 | 2026-03-15 22:46:55 | 0:01:47 | 5444 | build successful |
| 14 | 77896 | 2026-03-15 18:41:17 | 0:01:55 | 29023 | build successful |
| 11 | 203452 | 2026-03-15 16:19:08 | 0:06:59 | 66168 | build successful |
| 15.arm64 | 39052 | 2026-03-15 16:13:03 | 0:01:34 | 15708 | build successful |
| 15 | 48948 | 2026-02-07 12:59:11 | 0:02:12 | 15526 | failed install-port |
| 15.arm64 | 36647 | 2026-02-07 12:53:24 | 0:01:34 | 14721 | failed install-port |
| 10.6.x86_64 | 260425 | 2026-02-04 20:38:19 | 0:00:09 | 77409 | failed install-dependencies |
| 10.6 | 223167 | 2026-02-04 14:35:10 | 0:00:23 | 77333 | failed install-dependencies |
| 26.arm64 | 27518 | 2026-02-03 15:38:55 | 0:01:40 | 4323 | failed install-port |
| 15.arm64 | 31438 | 2025-11-22 9:59:06 | 0:02:14 | 12509 | failed install-port |
| 26.arm64 | 21478 | 2025-11-22 1:58:21 | 0:01:47 | 2217 | failed install-port |
| 15 | 43253 | 2025-11-21 20:26:20 | 0:02:02 | 13317 | failed install-port |
| 10.6 | 215209 | 2025-11-21 2:15:01 | 0:00:18 | 75193 | failed install-dependencies |
| 10.6.x86_64 | 252277 | 2025-11-21 2:12:01 | 0:00:06 | 75259 | failed install-dependencies |
| 26.arm64 | 14006 | 2025-10-01 23:01:10 | 0:01:30 | 707 | failed install-port |
| 15 | 38017 | 2025-08-20 4:51:38 | 0:02:27 | 10895 | failed install-port |
| 15.arm64 | 26395 | 2025-08-20 3:25:44 | 0:01:25 | 10083 | failed install-port |
| 10.6 | 207954 | 2025-08-19 12:26:18 | 0:00:21 | 72733 | failed install-dependencies |
| 10.6.x86_64 | 244624 | 2025-08-19 4:35:19 | 0:00:06 | 72789 | failed install-dependencies |
| 15.arm64 | 13455 | 2025-03-14 10:44:01 | 0:00:59 | 5802 | failed install-port |
| 15.arm64 | 7528 | 2025-03-06 1:39:51 | 0:00:54 | 3953 | failed install-port |
| 10.6 | 191186 | 2025-01-11 3:13:17 | 0:00:20 | 67272 | failed install-dependencies |
| 15 | 25219 | 2025-01-11 3:13:17 | 0:01:56 | 5460 | failed install-port |
| 10.6.x86_64 | 226847 | 2025-01-11 3:12:53 | 0:00:15 | 67323 | failed install-dependencies |
| 15 | 17584 | 2024-10-20 11:56:53 | 0:02:10 | 2341 | failed install-port |
| 15 | 16416 | 2024-10-18 21:23:59 | 0:02:24 | 2135 | failed install-port |
| 10.6 | 177230 | 2024-08-07 13:29:26 | 0:00:20 | 61711 | failed install-dependencies |
| 10.6.x86_64 | 212848 | 2024-08-07 13:10:51 | 0:00:15 | 61756 | failed install-dependencies |
| 10.6.x86_64 | 194262 | 2024-03-25 2:50:04 | 0:00:23 | 56581 | failed install-dependencies |
| 10.6 | 158553 | 2024-03-25 2:49:22 | 0:00:28 | 56534 | failed install-dependencies |
| 14.arm64 | 8797 | 2024-01-19 14:40:35 | 0:02:01 | 4221 | build successful |
| 14 | 11403 | 2023-11-13 21:28:02 | 0:05:21 | 1701 | build successful |
| 10.6 | 134017 | 2023-08-30 23:27:51 | 0:00:20 | 48701 | failed install-dependencies |
| 10.6.x86_64 | 169369 | 2023-08-30 23:23:38 | 0:00:15 | 48740 | failed install-dependencies |
| 11.arm64 | 100436 | 2023-08-24 18:35:21 | 0:01:39 | 36112 | build successful |
| 12 | 78939 | 2023-08-19 12:42:40 | 0:05:03 | 24914 | build successful |
| 11 | 124201 | 2023-08-19 11:05:35 | 0:05:49 | 36115 | build successful |
| 10.13 | 202062 | 2023-08-19 7:17:20 | 0:10:51 | 66451 | build successful |
| 10.14 | 176755 | 2023-08-19 7:10:57 | 0:13:09 | 57210 | build successful |
| 10.8 | 145026 | 2023-08-19 6:30:10 | 0:12:27 | 45970 | build successful |
| 10.7 | 156629 | 2023-08-19 6:24:54 | 0:14:00 | 47912 | build successful |
| 10.15 | 152337 | 2023-08-19 6:11:00 | 0:09:28 | 46198 | build successful |
| 10.11 | 232611 | 2023-08-19 5:21:11 | 0:08:42 | 75606 | build successful |
| 10.6 | 131883 | 2023-08-19 5:05:02 | 0:03:15 | 48130 | failed install-port |
| 10.6.x86_64 | 167430 | 2023-08-19 4:55:03 | 0:04:21 | 48169 | failed install-port |
| 10.9 | 242707 | 2023-08-19 4:47:11 | 0:08:39 | 75183 | build successful |
| 10.10 | 236221 | 2023-08-19 4:43:03 | 0:11:56 | 75494 | build successful |
| 10.12 | 241176 | 2023-08-19 3:51:03 | 0:09:48 | 74399 | build successful |
| 13.arm64 | 32245 | 2023-08-19 3:19:33 | 0:01:59 | 10524 | build successful |
| 12.arm64 | 103424 | 2023-08-19 2:52:03 | 0:01:38 | 24918 | build successful |
| 13 | 45109 | 2023-08-19 2:32:33 | 0:05:42 | 10667 | build successful |