publications

2026

1. EMNLP Main 2026

   PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues

   Mohammad Rifqi Farhansyah , Hanif Muhammad Zhafran , Farid Adilazuarda, and 6 more authors

   2026

   Abs Bib PDF

   Code-switching is a widespread practice among the world’s multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PingPong, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual. Our dataset consists of human-authored conversations among 2 to 4 participants covering authentic, multi-threaded structures where replies frequently reference much earlier points in the dialogue. We demonstrate that our data is significantly more natural and structurally diverse than machine-generated alternatives, offering greater variation in message length, speaker dominance, and reply distance. Based on these dialogues, we define three downstream tasks: Question Answering, Dialogue Summarization, and Topic Classification. Evaluations of several state-of-the-art language models on PingPong reveal that performance remains limited on code-switched inputs, underscoring the urgent need for more robust NLP systems capable of addressing the intricacies of real-world multilingual discourse.

   @misc{farhansyah2026pingpong,
     title = {PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues},
     author = {Farhansyah, Mohammad Rifqi and Zhafran, Hanif Muhammad and Adilazuarda, Farid and Muhammad, Shamsuddeen Hassan and Mukhtar, Maryam Ibrahim and Ousidhoum, Nedjma and Winata, Genta Indra and Purwarianti, Ayu and Aji, Alham Fikri},
     year = {2026},
     eprint = {2601.17277},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2601.17277},
   }

2. CBW @COLM 2026

   Gating Mechanisms for Continually Adapting Compressed KV Memory

   Quang Minh Nguyen , Phudish Prateepamornkul , Tri Q. Vo , and 3 more authors

   2026

   Abs Bib PDF

   Serving large language models over long contexts is expensive because cost scales with the key-value (KV) cache, and accuracy degrades when relevant evidence is buried among distracting tokens. Compressed memories address this problem by distilling a long context into a small, trainable KV cache. Cartridges learn such a cache through a synthetic question-answering objective called self-study. Real deployments, however, are not static: new corpora arrive continually, and a memory with a fixed budget must absorb each one while retaining what it already holds. We study this problem through gating mechanisms that govern what enters a compressed memory and where it is stored. We introduce two mechanisms for the dense Cartridge: a statistical TF-IDF gate that updates only the slots attended to by a new corpus but not by an earlier one, and a learned mixture-of-experts gate that routes each query to a small subset of expert sub-caches. Across two instruction-tuned models (Llama-3.2-3B and Qwen3-4B) in a continual setting, the best routed configurations improve both forgetting and acquisition relative to a dense baseline of equal capacity. However, router analysis shows little corpus-specific specialization, suggesting that routing helps mainly by limiting which parts of the cache can change during an update.

   @misc{nguyen2026gating,
     title = {Gating Mechanisms for Continually Adapting Compressed KV Memory},
     author = {Nguyen, Quang Minh and Prateepamornkul, Phudish and Vo, Tri Q. and Qorib, Muhammad Reza and Aji, Alham Fikri and Adilazuarda, Farid},
     year = {2026},
     url = {https://openreview.net/forum?id=50yXR3Uui7},
   }

3. CBW @COLM 2026

   Beyond Merging and Dropping: Adaptive Online KV Cache Compaction

   Izaaz Inhar , Faeyza Rishad Ardi , Muhammad Reza Qorib , and 2 more authors

   2026

   Abs Bib PDF

   As Large Language Models (LLMs) increasingly rely on long Chain-of-Thought reasoning, linear Key-Value (KV) cache growth creates severe memory and latency bottlenecks. Existing learnable cache compressions are typically offline and enforce rigid, uniform compression ratios. To address this, we propose an adaptive online compaction method that combines token merging with learnable token dropping. By extending the STILL compactor with Gumbel-Sigmoid gating, our approach dynamically prunes unimportant latents from a dense, merged initialization. This hybrid mechanism enables optimal, heterogeneous cache allocation across layers and heads without sacrificing reasoning capabilities. Trained sequentially on a  2M-token CoT dataset, our compactor reaches stable high compression stable 16x compression in 300 steps while keeping teacher–student divergence bounded across repeated compaction, where the uniform STILL baseline degenerates into repetition loops on the out-of-domain AIME26 benchmark.

   @misc{inhar2026beyond,
     title = {Beyond Merging and Dropping: Adaptive Online KV Cache Compaction},
     author = {Inhar, Izaaz and Ardi, Faeyza Rishad and Qorib, Muhammad Reza and Aji, Alham Fikri and Adilazuarda, Farid},
     year = {2026},
     url = {https://openreview.net/forum?id=rcR66RU5u1},
   }

4. Preprint 2026

   Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models

   Lucky Susanto , Musa Izzanardi Wijanarko , Khumaisa Nur’aini , and 3 more authors

   2026

   Abs Bib PDF

   While pixel-based language modeling aims to bypass the sub-word tokenization bottleneck by rendering text as images, recent multimodal variants such as DualGPT reintroduce text tokenizers to improve autoregressive performance. We investigate a fundamental question, does visual rendering truly decouple a model from tokenization constraints? Focusing on four Indonesian low-resource local languages that have their own non-Latin scripts (i.e., Javanese, Balinese, Sundanese, and Lampungnese), we evaluate the impact of script-tokenizer alignment within the DualGPT architecture. Our results show that, despite visual rendering, reintegrating a text tokenizer into the architecture reintroduces the same issue that pixel-based language modeling aims to resolve, which is the tokenizer misalignment problem. Despite having lower OOV and fertility rates, we show that the Llama 2 tokenizer performs significantly worse than a custom tokenizer, with improvements of up to 30.15 chrF++. Our findings serve as a warning for future multimodal variants, as text tokenizers remain a significant barrier to equitable models.

   @misc{susanto2026visual,
     title = {Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models},
     author = {Susanto, Lucky and Wijanarko, Musa Izzanardi and Nur'aini, Khumaisa and Adilazuarda, Farid and Aji, Alham Fikri and Wijaya, Derry Tanti},
     year = {2026},
     eprint = {2602.06973},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2602.06973},
   }

2025

1. Preprint 2025

   Datasheets Aren’t Enough: DataRubrics for Automated Quality Metrics and Accountability

   Genta Indra Winata , David Anugraha , Emmy Liu , and 17 more authors

   2025

   Abs Bib PDF

   High-quality datasets are fundamental to training and evaluating machine learning models, yet their creation-especially with accurate human annotations-remains a significant challenge. Many dataset paper submissions lack originality, diversity, or rigorous quality control, and these shortcomings are often overlooked during peer review. Submissions also frequently omit essential details about dataset construction and properties. While existing tools such as datasheets aim to promote transparency, they are largely descriptive and do not provide standardized, measurable methods for evaluating data quality. Similarly, metadata requirements at conferences promote accountability but are inconsistently enforced. To address these limitations, this position paper advocates for the integration of systematic, rubric-based evaluation metrics into the dataset review process-particularly as submission volumes continue to grow. We also explore scalable, cost-effective methods for synthetic data generation, including dedicated tools and LLM-as-a-judge approaches, to support more efficient evaluation. As a call to action, we introduce DataRubrics, a structured framework for assessing the quality of both human- and model-generated datasets. Leveraging recent advances in LLM-based evaluation, DataRubrics offers a reproducible, scalable, and actionable solution for dataset quality assessment, enabling both authors and reviewers to uphold higher standards in data-centric research.

   @misc{winata2025datarubrics,
     title = {Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountability},
     author = {Winata, Genta Indra and Anugraha, David and Liu, Emmy and Aji, Alham Fikri and Hung, Shou-Yi and Parashar, Aditya and Irawan, Patrick Amadeus and Zhang, Ruochen and Yong, Zheng-Xin and Cruz, Jan Christian Blaise and Muennighoff, Niklas and Kim, Seungone and Zhao, Hanyang and Kar, Sudipta and Suryoraharjo, Kezia Erina and Adilazuarda, Farid and Lee, En-Shiun Annie and Purwarianti, Ayu and Wijaya, Derry Tanti and Choudhury, Monojit},
     year = {2025},
     eprint = {2506.01789},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
     url = {https://arxiv.org/abs/2506.01789},
   }

2. Preprint 2025

   Crosslingual Reasoning through Test-Time Scaling

   Zheng-Xin Yong* , Farid Adilazuarda*, Jonibek Mansurov* , and 7 more authors

   2025

   Abs Bib PDF

   Reasoning capabilities of large language models are primarily studied for English, even when pretrained models are multilingual. In this work, we investigate to what extent English reasoning finetuning with long chain-of-thoughts (CoTs) can generalize across languages. First, we find that scaling up inference compute for English-centric reasoning language models (RLMs) improves multilingual mathematical reasoning across many languages including low-resource languages, to an extent where they outperform models twice their size. Second, we reveal that while English-centric RLM’s CoTs are naturally predominantly English, they consistently follow a quote-and-think pattern to reason about quoted non-English inputs. Third, we discover an effective strategy to control the language of long CoT reasoning, and we observe that models reason better and more efficiently in high-resource languages. Finally, we observe poor out-of-domain reasoning generalization, in particular from STEM to cultural commonsense knowledge, even for English. Overall, we demonstrate the potentials, study the mechanisms and outline the limitations of crosslingual generalization of English reasoning test-time scaling. We conclude that practitioners should let English-centric RLMs reason in high-resource languages, while further work is needed to improve reasoning in low-resource languages and out-of-domain contexts.

   @misc{yong2025crosslingual,
     title = {Crosslingual Reasoning through Test-Time Scaling},
     author = {Yong*, Zheng-Xin and Adilazuarda*, Farid and Mansurov*, Jonibek and Zhang*, Ruochen and Muennighoff, Niklas and Eickhoff, Carsten and Winata, Genta Indra and Kreutzer, Julia and Bach, Stephen H. and Aji, Alham Fikri},
     year = {2025},
     eprint = {2505.05408},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2505.05408},
   }

3. NAACL Main 2025

   WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

   Genta Indra Winata , Frederikus Hudi , Patrick Amadeus Irawan , and 48 more authors

   2025

   Abs Bib PDF

   Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.

   @misc{winata2025worldcuisines,
     title = {WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines},
     author = {Winata, Genta Indra and Hudi, Frederikus and Irawan, Patrick Amadeus and Anugraha, David and Putri, Rifki Afina and Wang, Yutong and Nohejl, Adam and Prathama, Ubaidillah Ariq and Ousidhoum, Nedjma and Amriani, Afifa and Rzayev, Anar and Das, Anirban and Pramodya, Ashmari and Adila, Aulia and Wilie, Bryan and Mawalim, Candy Olivia and Cheng, Ching Lam and Abolade, Daud and Chersoni, Emmanuele and Santus, Enrico and Ikhwantri, Fariz and Kuwanto, Garry and Zhao, Hanyang and Wibowo, Haryo Akbarianto and Lovenia, Holy and Cruz, Jan Christian Blaise and Putra, Jan Wira Gotama and Myung, Junho and Susanto, Lucky and Machin, Maria Angelica Riera and Zhukova, Marina and Anugraha, Michael and Adilazuarda, Farid and Santosa, Natasha and Limkonchotiwat, Peerat and Dabre, Raj and Audino, Rio Alexander and Cahyawijaya, Samuel and Zhang, Shi-Xiong and Salim, Stephanie Yulia and Zhou, Yi and Gui, Yinxuan and Adelani, David Ifeoluwa and Lee, En-Shiun Annie and Okada, Shogo and Purwarianti, Ayu and Aji, Alham Fikri and Watanabe, Taro and Wijaya, Derry Tanti and Oh, Alice and Ngo, Chong-Wah},
     year = {2025},
     eprint = {2410.12705},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2410.12705},
   }

4. NAACL Findings 2025

   MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding

   Zayd Muhammad Kawakibi Zuhri , Farid Adilazuarda, Ayu Purwarianti , and 1 more author

   2025

   Abs Bib PDF

   Auto-regressive inference of transformers benefit greatly from Key-Value (KV) caching, but can lead to major memory bottlenecks as model size, batch size, and sequence length grow at scale. We introduce Multi-Layer Key-Value (MLKV) sharing, a novel approach extending KV sharing across transformer layers to reduce memory usage beyond what was possible with Multi-Query Attention (MQA) and Grouped-Query Attention (GQA). Evaluations on various NLP benchmarks and inference metrics using uptrained Pythia-160M variants demonstrate that MLKV significantly reduces memory usage with minimal performance loss, reducing KV cache size down to a factor of 6x compared to MQA. These results highlight MLKV’s potential for efficient deployment of transformer models at scale.

   @misc{zuhri2025mlkv,
     title = {MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding},
     author = {Zuhri, Zayd Muhammad Kawakibi and Adilazuarda, Farid and Purwarianti, Ayu and Aji, Alham Fikri},
     year = {2025},
     eprint = {2406.09297},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
     url = {https://arxiv.org/abs/2406.09297},
   }

5. EMNLP Main 2025

   From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs

   Farid Adilazuarda, Chen Cecilia Liu , Iryna Gurevych , and 1 more author

   In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , 2025

   Abs Bib PDF

   Adapting cultural values in Large Language Models (LLMs) presents significant challenges, particularly due to biases and limited training data. Prior work primarily aligns LLMs with different cultural values using World Values Survey (WVS) data. However, it remains unclear whether this approach effectively captures cultural nuances or produces distinct cultural representations for various downstream tasks. We systematically investigate WVS-based training for cultural value adaptation and find that relying solely on survey data can homogenize cultural norms and interfere with factual knowledge. We augment WVS with encyclopedic and scenario-based cultural narratives from Wikipedia and NormAd, finding that narratives consistently improve cultural distinctiveness over survey data alone.

   @inproceedings{adilazuarda2025from,
     title = {From Surveys to Narratives: Rethinking Cultural Value Adaptation in LLMs},
     author = {Adilazuarda, Farid and Liu, Chen Cecilia and Gurevych, Iryna and Aji, Alham Fikri},
     booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
     year = {2025},
     eprint = {2505.16408},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2505.16408},
   }

6. ACL Main 2025

   NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts

   Farid Adilazuarda, Musa Izzanardi Wijanarko , Lucky Susanto , and 3 more authors

   In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics , 2025

   Abs Bib PDF

   NusaAksara is a multimodal and multilingual benchmark for Indonesian languages and indigenous scripts. The benchmark covers text and image modalities across tasks including image segmentation, OCR, transliteration, translation, and language identification, with data constructed by human experts.

   @inproceedings{adilazuarda2025nusaaksara,
     title = {NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts},
     author = {Adilazuarda, Farid and Wijanarko, Musa Izzanardi and Susanto, Lucky and Nur'aini, Khumaisa and Wijaya, Derry and Aji, Alham Fikri},
     booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics},
     year = {2025},
     eprint = {2502.18148},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     doi = {10.18653/v1/2025.acl-long.1377},
     url = {https://arxiv.org/abs/2502.18148},
   }

2024

1. NeurIPS 2024

   CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark

   David Romero , Chenyang Lyu , Haryo Akbarianto Wibowo , and 73 more authors

   2024

   Abs Bib PDF

   Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge present in both visual and textual data. However, most of the current VQA models use datasets that are primarily focused on English and a few major world languages, with images that are typically Western-centric. While recent efforts have tried to increase the number of languages covered on VQA datasets, they still lack diversity in low-resource languages. More importantly, although these datasets often extend their linguistic range via translation or some other approaches, they usually keep images the same, resulting in narrow cultural representation. To address these limitations, we construct CVQA, a new Culturally-diverse multilingual Visual Question Answering benchmark, designed to cover a rich set of languages and cultures, where we engage native speakers and cultural experts in the data collection process. As a result, CVQA includes culturally-driven images and questions from across 30 countries on four continents, covering 31 languages with 13 scripts, providing a total of 10k questions. We then benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and show that the dataset is challenging for the current state-of-the-art models. This benchmark can serve as a probing evaluation suite for assessing the cultural capability and bias of multimodal models and hopefully encourage more research efforts toward increasing cultural awareness and linguistic diversity in this field.

   @misc{romero2024cvqa,
     title = {CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark},
     author = {Romero, David and Lyu, Chenyang and Wibowo, Haryo Akbarianto and Lynn, Teresa and Hamed, Injy and Kishore, Aditya Nanda and Mandal, Aishik and Dragonetti, Alina and Abzaliev, Artem and Tonja, Atnafu Lambebo and Balcha, Bontu Fufa and Whitehouse, Chenxi and Salamea, Christian and Velasco, Dan John and Adelani, David Ifeoluwa and Meur, David Le and Villa-Cueva, Emilio and Koto, Fajri and Farooqui, Fauzan and Belcavello, Frederico and Batnasan, Ganzorig and Vallejo, Gisela and Caulfield, Grainne and Ivetta, Guido and Song, Haiyue and Ademtew, Henok Biadglign and Maina, Hern\'an and Lovenia, Holy and Azime, Israel Abebe and Cruz, Jan Christian Blaise and Gala, Jay and Geng, Jiahui and Ortiz-Barajas, Jesus-German and Baek, Jinheon and Dunstan, Jocelyn and Alemany, Laura Alonso and Nagasinghe, Kumaranage Ravindu Yasas and Benotti, Luciana and D'Haro, Luis Fernando and Viridiano, Marcelo and Estecha-Garitagoitia, Marcos and Cabrera, Maria Camila Buitrago and Rodr\'iguez-Cantelar, Mario and Jouitteau, M\'elanie and Mihaylov, Mihail and Imam, Mohamed Fazli Mohamed and Adilazuarda, Farid and Gochoo, Munkhjargal and Otgonbold, Munkh-Erdene and Etori, Naome and Niyomugisha, Olivier and Silva, Paula M\'onica and Chitale, Pranjal and Dabre, Raj and Chevi, Rendi and Zhang, Ruochen and Diandaru, Ryandito and Cahyawijaya, Samuel and G\'ongora, Santiago and Jeong, Soyeong and Purkayastha, Sukannya and Kuribayashi, Tatsuki and Clifford, Teresa and Jayakumar, Thanmay and Torrent, Tiago Timponi and Ehsan, Toqeer and Araujo, Vladimir and Kementchedjhieva, Yova and Burzo, Zara and Lim, Zheng Wei and Yong, Zheng Xin and Ignat, Oana and Nwatu, Joan and Mihalcea, Rada and Solorio, Thamar and Aji, Alham Fikri},
     year = {2024},
     eprint = {2406.05967},
     archiveprefix = {arXiv},
     primaryclass = {cs.CV},
     url = {https://arxiv.org/abs/2406.05967},
   }

2. EMNLP Findings 2024

   LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization

   Farid Adilazuarda, Samuel Cahyawijaya , Alham Fikri Aji , and 2 more authors

   2024

   Abs Bib PDF

   Pretrained language models (PLMs) have shown remarkable generalization toward multiple tasks and languages. Nonetheless, the generalization of PLMs towards unseen languages is poor, resulting in significantly worse language performance, or even generating nonsensical responses that are comparable to a random baseline. This limitation has been a longstanding problem of PLMs raising the problem of diversity and equal access to language modeling technology. In this work, we solve this limitation by introducing LinguAlchemy, a regularization technique that incorporates various aspects of languages covering typological, geographical, and phylogenetic constraining the resulting representation of PLMs to better characterize the corresponding linguistics constraints. LinguAlchemy significantly improves the accuracy performance of mBERT and XLM-R on unseen languages by  18% and  2%, respectively compared to fully finetuned models and displaying a high degree of unseen language generalization. We further introduce AlchemyScale and AlchemyTune, extension of LinguAlchemy which adjusts the linguistic regularization weights automatically, alleviating the need for hyperparameter search. LinguAlchemy enables better cross-lingual generalization to unseen languages which is vital for better inclusivity and accessibility of PLMs.

   @misc{adilazuarda2024lingualchemy,
     title = {LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization},
     author = {Adilazuarda, Farid and Cahyawijaya, Samuel and Aji, Alham Fikri and Winata, Genta Indra and Purwarianti, Ayu},
     year = {2024},
     eprint = {2401.06034},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/pdf/2401.06034v2},
   }

3. EMNLP Main 2024

   Towards Measuring and Modeling "Culture" in LLMs: A Survey

   Farid Adilazuarda, Sagnik Mukherjee , Pradhyumna Lavania , and 6 more authors

   2024

   Abs Bib PDF

   We present a survey of 55 recent papers that aim to study cultural representation and inclusion in large language models. We observe that none of the studies define "culture," which is a complex, multifaceted concept; instead, they probe the models on some specially designed datasets which represent certain aspects of "culture." We call these aspects the proxies of cultures, and organize them across three dimensions of demographic, semantic and linguistic-cultural interaction proxies. We also categorize the probing methods employed. Our analysis indicates that only certain aspects of "culture," such as values and objectives, have been studied, leaving several other interesting and important facets, especially the multitude of semantic domains (Thompson et al., 2020) and aboutness (Hershcovich et al., 2022), unexplored. Two other crucial gaps are the lack of robustness and situatedness of the current methods. Based on these observations, we provide several recommendations for a holistic and practically useful research agenda for furthering cultural inclusion in LLMs and LLM-based applications.

   @misc{adilazuarda2024measuring,
     title = {Towards Measuring and Modeling "Culture" in LLMs: A Survey},
     author = {Adilazuarda, Farid and Mukherjee, Sagnik and Lavania, Pradhyumna and Singh, Siddhant and Dwivedi, Ashutosh and Aji, Alham Fikri and O'Neill, Jacki and Modi, Ashutosh and Choudhury, Monojit},
     year = {2024},
     eprint = {2403.15412},
     archiveprefix = {arXiv},
     primaryclass = {cs.CY},
     url = {https://arxiv.org/pdf/2403.15412},
   }

4. TrustNLP @NAACL 2024

   Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text

   Farid Adilazuarda

   2024

   Abs Bib PDF

   Significant progress has been made on text generation by pre-trained language models (PLMs), yet distinguishing between human and machine-generated text poses an escalating challenge. This paper offers an in-depth evaluation of three distinct methods used to address this task: traditional shallow learning, Language Model (LM) fine-tuning, and Multilingual Model fine-tuning. These approaches are rigorously tested on a wide range of machine-generated texts, providing a benchmark of their competence in distinguishing between human-authored and machine-authored linguistic constructs. The results reveal considerable differences in performance across methods, thus emphasizing the continued need for advancement in this crucial area of NLP. This study offers valuable insights and paves the way for future research aimed at creating robust and highly discriminative models.

   @misc{adilazuarda2024turing,
     title = {Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text},
     author = {Adilazuarda, Farid},
     year = {2024},
     eprint = {2311.12373},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/pdf/2311.12373},
   }

5. EMNLP Main 2024

   SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages

   Holy Lovenia , Rahmad Mahendra , Salsabil Maulana Akbar , and 58 more authors

   2024

   PDF
6. EMNLP Main 2024

   Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting

   Sagnik Mukherjee* , Farid Adilazuarda*, Sunayana Sitaram , and 3 more authors

   2024

   PDF

2023

1. Tiny Papers @ICLR 2023

   The Obscure Limitation of Modular Multilingual Language Models

   Farid Adilazuarda, Samuel Cahyawijaya , and Ayu Purwarianti

   2023

   Abs Bib PDF

   We expose the limitation of modular multilingual language models (MLMs) in multilingual inference scenarios with unknown languages. Existing evaluations of modular MLMs exclude the involvement of language identification (LID) modules, which obscures the performance of real-case multilingual scenarios of modular MLMs. In this work, we showcase the effect of adding LID on the multilingual evaluation of modular MLMs and provide discussions for closing the performance gap of caused by the pipelined approach of LID and modular MLMs.

   @misc{adilazuarda2023obscure,
     title = {The Obscure Limitation of Modular Multilingual Language Models},
     author = {Adilazuarda, Farid and Cahyawijaya, Samuel and Purwarianti, Ayu},
     year = {2023},
     eprint = {2311.12375},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/pdf/2311.12375},
   }

2. Findings @ACL 2023

   NusaCrowd: Open Source Initiative for Indonesian NLP Resources

   Samuel Cahyawijaya , Holy Lovenia , Alham Fikri Aji , and 45 more authors

   In Findings of the Association for Computational Linguistics: ACL 2023 , Jul 2023

   Abs Bib PDF

   We present NusaCrowd, a collaborative initiative to collect and unify existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, we have brought together 137 datasets and 118 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their value is demonstrated through multiple experiments.NusaCrowd’s data collection enables the creation of the first zero-shot benchmarks for natural language understanding and generation in Indonesian and the local languages of Indonesia. Furthermore, NusaCrowd brings the creation of the first multilingual automatic speech recognition benchmark in Indonesian and the local languages of Indonesia. Our work strives to advance natural language processing (NLP) research for languages that are under-represented despite being widely spoken.

   @inproceedings{cahyawijaya-etal-2023-nusacrowd,
     title = {{N}usa{C}rowd: Open Source Initiative for {I}ndonesian {NLP} Resources},
     author = {Cahyawijaya, Samuel and Lovenia, Holy and Aji, Alham Fikri and Winata, Genta and Wilie, Bryan and Koto, Fajri and Mahendra, Rahmad and Wibisono, Christian and Romadhony, Ade and Vincentio, Karissa and Santoso, Jennifer and Moeljadi, David and Wirawan, Cahya and Hudi, Frederikus and Wicaksono, Muhammad Satrio and Parmonangan, Ivan and Alfina, Ika and Putra, Ilham Firdausi and Rahmadani, Samsul and Oenang, Yulianti and Septiandri, Ali and Jaya, James and Dhole, Kaustubh and Suryani, Arie and Putri, Rifki Afina and Su, Dan and Stevens, Keith and Nityasya, Made Nindyatama and Adilazuarda, Farid and Hadiwijaya, Ryan and Diandaru, Ryandito and Yu, Tiezheng and Ghifari, Vito and Dai, Wenliang and Xu, Yan and Damapuspita, Dyah and Wibowo, Haryo and Tho, Cuk and Karo Karo, Ichwanul and Fatyanosa, Tirana and Ji, Ziwei and Neubig, Graham and Baldwin, Timothy and Ruder, Sebastian and Fung, Pascale and Sujaini, Herry and Sakti, Sakriani and Purwarianti, Ayu},
     editor = {Rogers, Anna and Boyd-Graber, Jordan and Okazaki, Naoaki},
     booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
     month = jul,
     year = {2023},
     address = {Toronto, Canada},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2023.findings-acl.868v2.pdf},
     doi = {10.18653/v1/2023.findings-acl.868},
     pages = {13745--13818},
   }

2022

1. SUMEval @AACL 2022

   IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages

   Farid Adilazuarda, Samuel Cahyawijaya , Genta Indra Winata , and 2 more authors

   In AACL’22 Workshop on Scaling Up Multilingual Evaluation , Nov 2022

   Abs Bib PDF

   Significant progress has been made on Indonesian NLP. Nevertheless, exploration of the code-mixing phenomenon in Indonesian is limited, despite many languages being frequently mixed with Indonesian in daily conversation. In this work, we explore code-mixing in Indonesian with four embedded languages, i.e., English, Sundanese, Javanese, and Malay; and introduce IndoRobusta, a framework to evaluate and improve the code-mixing robustness. Our analysis shows that the pre-training corpus bias affects the model’s ability to better handle Indonesian-English code-mixing when compared to other local languages, despite having higher language diversity.

   @inproceedings{adilazuarda-etal-2022-indorobusta,
     title = {{I}ndo{R}obusta: Towards Robustness Against Diverse Code-Mixed {I}ndonesian Local Languages},
     author = {Adilazuarda, Farid and Cahyawijaya, Samuel and Winata, Genta Indra and Fung, Pascale and Purwarianti, Ayu},
     editor = {Ahuja, Kabir and Anastasopoulos, Antonios and Patra, Barun and Neubig, Graham and Choudhury, Monojit and Dandapat, Sandipan and Sitaram, Sunayana and Chaudhary, Vishrav},
     booktitle = {AACL'22 Workshop on Scaling Up Multilingual Evaluation},
     month = nov,
     year = {2022},
     address = {Online},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2022.sumeval-1.5/},
     pages = {25--34},
   }