Poppo/KoGEM

191 downloads in 30 days0 likesLast modified 24 Aug 2025View on Hugging Face

The declared licence (cc-by-4.0) is classified as permissive.

Declared licence: cc-by-4.0, from the tagEvidence: README.md at 338e8cd
Licence files: No licence file at the top level of the repositoryEvidence: file list at 338e8cd
Declared sources: None listed in the card metadata
Access gated on Hugging Face: No

Each cell links to what was checked: a file, a file list or a Hub record.

Tasks

Citation text from the card

@inproceedings{kim-etal-2025-polishing,
    title = "Polishing Every Facet of the {GEM}: Testing Linguistic Competence of {LLM}s and Humans in {K}orean",
    author = "Kim, SungHo  and
      Kim, Nayeon  and
      Jeon, Taehee  and
      Lee, SangKeun",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.acl-long.492/",
    doi = "10.18653/v1/2025.acl-long.492",
    pages = "9955--9984",
    ISBN = "979-8-89176-251-0",
    abstract = "We introduce the $\underline{Ko}rean \underline{G}rammar \underline{E}valuation Bench\underline{M}ark (KoGEM)$, designed to assess the linguistic competence of LLMs and humans in Korean. KoGEM consists of 1.5k multiple-choice QA pairs covering five main categories and 16 subcategories. The zero-shot evaluation of 27 LLMs of various sizes and types reveals that while LLMs perform remarkably well on straightforward tasks requiring primarily definitional knowledge, they struggle with tasks that demand the integration of real-world experiential knowledge, such as phonological rules and pronunciation. Furthermore, our in-depth analysis suggests that incorporating such experiential know

All datasets marked Permissive or share-alike | JSON

The same facts for your agent

A free MCP server at mcp.datalicense.uk/mcp. Connect it from an MCP client such as Claude or Cursor.

dataset_licence({ "id": "Poppo/KoGEM" })
signal: "permissive"
stated_licence: "cc-by-4.0"
sources: 0