tokenizers (0.23.1+dfsg-1) unstable; urgency=medium

  [ Kohei Sendai ]
  * New upstream release.
  * Rebase existing patches.

  [ Mo Zhou ]
  * New upstream version 0.23.1+dfsg
  * Add myself (GSoC mentor) to uploaders.
  * Update rust dependency, add missing B-D, and patch source code to use
    rand::thread_rng instead of rng.
  * Add debian/tests/gencontrol.sh for autopkgtest.
  * Add python3-pytest-asyncio to test dependencies.
  * Upgrade debian/watch format to version 5.
  * Release to unstable.

 -- Mo Zhou <lumin@debian.org>  Thu, 09 Jul 2026 19:13:25 -0700

tokenizers (0.20.3+dfsg-1) experimental; urgency=medium

  * Initial release. (Closes: #1109641)

 -- Kohei Sendai <kouhei.sendai@gmail.com>  Fri, 09 May 2025 14:01:31 +0000