All datasets / Vicomtech
Vicomtech/just-eval-instruct-eu
The declared licence (mit) is classified as permissive.
- Declared licence
- mit, from the tagEvidence: README.md at b651b97
- Licence files
- No licence file at the top level of the repositoryEvidence: file list at b651b97
- Declared sources
- None listed in the card metadata
- Access gated on Hugging Face
- No
Citation text from the card
```bibtex
@inproceedings{ponce-etal-2026-judging,
title = {Judging Instruction Responses in a Low-Resource Language: A Case Study on Basque},
author = {Ponce, David and Gete, Harritxu and Etchegoyhen, Thierry and Zubiaga, Irune and Soroa, Aitor},
booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
month = {May},
year = {2026},
pages = {281--298},
address = {Palma, Mallorca, Spain},
publisher = {European Language Resources Association (ELRA)},
editor = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
doi = {10.63317/32e7ij6myh5i},
abstract = {Evaluating the quality of answers to a given instruction is a demanding and time-consuming task, limiting the scalability of human assessment. Large language models (LLMs) have been proposed as automatic judges to reduce this effort, but their reliability in low-resource contexts remains uncertain. Additionally, the premise that humans are reliable judges of fine-grained response quality needs to be assessed as well, if correlation with automated judges on this task is to be considered a gold standard. In this work, we investigate the performance of various LLM-as-a-judge in a low-resource scenario, namely Basque, and evaluate its correlation with human judgements. Additionally, we measure the agreement between human judgments themselves, to assess their viability as a valid reference. To perform our experi
The same facts for your agent
A free MCP server at mcp.datalicense.uk/mcp. Connect it from an MCP client such as Claude or Cursor.
dataset_licence({ "id": "Vicomtech/just-eval-instruct-eu" })
signal: "permissive"
stated_licence: "mit"
sources: 0