All datasets / r-three

r-three/farsi_tokenizer_robustness

84 downloads in 30 days1 likesLast modified 29 Oct 2025View on Hugging Face

Permissive or share-alike

The declared licence (mit) is classified as permissive.

Declared licence: mit, from the tagEvidence: README.md at 5174bfe
Licence files: No licence file at the top level of the repositoryEvidence: file list at 5174bfe
Declared sources: None listed in the card metadata
Access gated on Hugging Face: No

Each cell links to what was checked: a file, a file list or a Hub record.

Tasks

multiple-choice

Citation text from the card

@inproceedings{toksuite2026,
  title={TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior},
  author={Altıntaş, Gül Sena and Ehghaghi, Malikeh and Lester, Brian and Liu, Fengyuan and Zhao, Wanru and Ciccone, Marco and Raffel, Colin},
  booktitle={Preprint.},
  year={2026},
  url={TBD}
}

All datasets marked Permissive or share-alike | JSON

The same facts for your agent

A free MCP server at mcp.datalicense.uk/mcp. Connect it from an MCP client such as Claude or Cursor.

dataset_licence({ "id": "r-three/farsi_tokenizer_robustness" })
signal: "permissive"
stated_licence: "mit"
sources: 0