All datasets / HPLT

HPLT/OpenLID-v3

284 downloads in 30 days0 likesLast modified 18 Jun 2026View on Hugging Face

Unclear

The checks could not identify a recognised licence ID in the dataset's metadata.

Declared licence: other, from the tagEvidence: README.md at f1d299c
Licence files: No licence file at the top level of the repositoryEvidence: file list at f1d299c
Declared sources: None listed in the card metadata
Access gated on Hugging Face: No

Each cell links to what was checked: a file, a file list or a Hub record.

Tasks

text-classification

Citation text from the card

@inproceedings{fedorova-etal-2026-openlid,
    title = "{O}pen{LID}-v3: Improving the Precision of Closely Related Language Identification {--} An Experience Report",
    author = "Fedorova, Mariia  and
      Arefyev, Nikolay  and
      Buljan, Maja  and
      Helcl, Jind{\v{r}}ich  and
      Oepen, Stephan  and
      R{\o}nningstad, Egil  and
      Scherrer, Yves",
    editor = {Scherrer, Yves  and
      Aepli, No{\"e}mi  and
      Blaschke, Verena  and
      Jauhiainen, Tommi  and
      Ljube{\v{s}}i{\'c}, Nikola  and
      Nakov, Preslav  and
      Tiedemann, J{\"o}rg  and
      Zampieri, Marcos},
    booktitle = "Proceedings of the 13th Workshop on {NLP} for Similar Languages, Varieties and Dialects",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.vardial-1.23/",
    doi = "10.18653/v1/2026.vardial-1.23",
    pages = "275--292",
    abstract = "Language identification (LID) is an essential step in building high-quality multilingual datasets from web data. Existing LID tools (such as OpenLID or GlotLID) often struggle to identify closely related languages and to distinguish valid natural language from noise, which contaminates language-specific subsets, especially for low-resource languages. In this work we extend the OpenLID classifier by adding more training data, merging problematic language variant clusters, and introducing a special label for mark

All datasets marked Unclear | JSON

The same facts for your agent

A free MCP server at mcp.datalicense.uk/mcp. Connect it from an MCP client such as Claude or Cursor.

dataset_licence({ "id": "HPLT/OpenLID-v3" })
signal: "unclear"
stated_licence: "other"
sources: 0