commoncrawl/CommonLID

Tasks

• text-classification

Citation text from the card

```text
@inproceedings{suarez-etal-2026-commonlid,
    title = "{C}ommon{LID}: Re-evaluating State-of-the-Art Language Identification Performance on Web Data",
    author = "Suarez, Pedro Ortiz  and
      Burchell, Laurie  and
      Arnett, Catherine  and
      Mosquera, Rafael  and
      Monsalve, Sara Hincapi{\'e}  and
      Vaughan, Thom  and
      Stewart, Damian  and
      Ostendorff, Malte  and
      Abdulmumin, Idris  and
      Marivate, Vukosi  and
      Muhammad, Shamsuddeen Hassan  and
      Tonja, Atnafu Lambebo  and
      Al-Khalifa, Hend  and
      Hammouda, Nadia Ghezaiel  and
      Otiende, Verrah Akinyi  and
      Wong, Tack Hwa  and
      Saydaliev, Jakhongir  and
      Nobakhtian, Melika  and
      Habibi, Muhammad Ravi Shulthan  and
      Kranti, Chalamalasetti  and
      Muchemi, Carol  and
      Nguyen, Khang  and
      Adam, Faisal Muhammad  and
      Salim, Luis Frentzen  and
      Alqifari, Reem  and
      Amol, Cynthia Jayne  and
      Imperial, Joseph Marvin  and
      Kesen, Ilker  and
      Mustafid, Ahmad  and
      Stepachev, Pavel  and
      Choshen, Leshem  and
      Anugraha, David  and
      Nayel, Hamada  and
      Yimam, Seid Muhie  and
      Alexandra Putra, Vallerie  and
      Nguyen, My Chiffon  and
      Wasi, Azmine Toushik  and
      Vadithya, Gouthami  and
      Van Der Goot, Rob  and
      C{'}horr, Lanwenn ar  and
      Dua, Karan  and
      Yates, Andrew  and
      Bangera, Mithil  and
      Bangera, Yeshil  and
      Patel, Hites

All datasets marked Unclear | JSON