kanishka/comps

55 downloads in 30 days1 likesLast modified 16 Sep 2023View on Hugging Face

The declared licence (apache-2.0) is classified as permissive.

Declared licence: apache-2.0, from the tagEvidence: README.md at d182636
Licence files: No licence file at the top level of the repositoryEvidence: file list at d182636
Declared sources: None listed in the card metadata
Access gated on Hugging Face: No

Each cell links to what was checked: a file, a file list or a Hub record.

Citation text from the card

@inproceedings{misra-etal-2023-comps,
    title = "{COMPS}: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models",
    author = "Misra, Kanishka  and
      Rayz, Julia  and
      Ettinger, Allyson",
    booktitle = "Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics",
    month = may,
    year = "2023",
    address = "Dubrovnik, Croatia",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.eacl-main.213",
    doi = "10.18653/v1/2023.eacl-main.213",
    pages = "2928--2949",
    abstract = "A characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of properties (can breathe) from superordinate concepts (animal) to their subordinates (dog){---}i.e. demonstrate property inheritance. In this paper, we present COMPS, a collection of minimal pair sentences that jointly tests pre-trained language models (PLMs) on their ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. Analyses of 22 different PLMs on COMPS reveal that they can easily distinguish between concepts on the basis of a property when they are trivially different, but find it relatively difficult when concepts are related on the basis of nuanced knowledge repr

All datasets marked Permissive or share-alike | JSON

The same facts for your agent

A free MCP server at mcp.datalicense.uk/mcp. Connect it from an MCP client such as Claude or Cursor.

dataset_licence({ "id": "kanishka/comps" })
signal: "permissive"
stated_licence: "apache-2.0"
sources: 0