coffee/deepseek-recipe
1
0
Fork 0
mirror of https://github.com/deepseek-ai/deepseek-recipe.git synced 2026-10-28 17:17:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
deepseek-recipe/static/tokenizers/v41/README.md
icrim 57b9c84242 Initial commit 
Co-authored-by: xinchengxx <xinchengxx@deepseek.com>
Co-authored-by: XieJiSS <xiejiss@deepseek.com>
Co-authored-by: yqaty <jiayu@deepseek.com>
Co-authored-by: zhu-he <hezhu@deepseek.com>
Co-authored-by: season-guo <seasonguo@deepseek.com>
2026-09-10 18:07:01 +08:00

1.1 KiB
Raw Permalink Blame History

V4.1 tokenizer

tokenizer.json is a modified copy of the Hugging Face file deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/tokenizer.json with the image token reassigned.

Local changes

id this file upstream
129264 <｜image｜> special=true <｜deepseek_image｜> special=false

All other JSON entries match the comparison revision below.

Provenance and license

  • Comparison revision: 6821d6ad3681a4b137b066b76094fa82ebd0a380, verified on 2026-09-10.
  • Local SHA-256: 81f64d1248a68ce3663e07ab3ee48b851e5df0e32d27cb98e4c9a268151e8d99.
  • Upstream license: MIT, Copyright (c) 2023 DeepSeek.
  • Preserved license notice: ../LICENSE.

The original import revision was not recorded; the comparison revision identifies the upstream JSON used to verify the changes above.