Tokenizer files served from this folder

These files are copied unchanged in content (only re-serialized as compact JSON)
from the Hugging Face repositories listed below. They are used only to count and
display tokens in your browser.

llama3.json, llama3.config.json
  From unsloth/Llama-3.2-1B-Instruct (the Llama 3.2 tokenizer).
  Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
  Full license: licenses/LLAMA-3.2-LICENSE.txt
  Built with Llama.

qwen3.json, qwen3.config.json
  From Qwen/Qwen3-8B. Apache License 2.0 (licenses/QWEN3-LICENSE.txt).

deepseek-v3.json, deepseek-v3.config.json
  From deepseek-ai/DeepSeek-V3. DeepSeek License Agreement (licenses/DEEPSEEK-LICENSE-MODEL.txt).

mistral-tekken.json, mistral-tekken.config.json
  From mistralai/Mistral-Nemo-Instruct-2407. Apache License 2.0.

mistral-v3.json, mistral-v3.config.json
  From mistralai/Mistral-7B-Instruct-v0.3. Apache License 2.0.

The OpenAI encodings (o200k_base, cl100k_base) come from the js-tiktoken npm package (MIT).