Chenxi Whitehouse

I am Chenxi, a postdoc researcher at University of Cambridge working with Prof Andreas Vlachos on automated fact-checking. I pursued my PhD studies on knowledge-grounded NLP at City, University of London. Prior to that, I received my Bachelor’s degree in Information Engineering at Xi’an Jiaotong University, and my Master’s degree in Electrical Engineering at University of Erlangen-Nürnberg and University College London.

My research focuses on Natural Language Understanding and Generation, Multilinguality, with Knowledge and Reasoning. During my PhD, I interned at Google DeepMind in Amsterdam, Amazon Alexa AI in Cambridge, Huawei Noah’s Ark Lab in London, and visited MBZUAI as a research assistant. In my most recent projects I have also explored parameter efficient modelling (LoRA) and diffusion language models.

I am also part of the machine learning engineering team at Replicate, working towards open-source AI solutions.

News

Mar - 2024	Our paper Low-Rank Adaptation for Multilingual Summarisation: An Empirical Study from my internship at Google DeepMind is accepted in the findings of NAACL 2024!
Jan - 2024	Excited to share that I have joined University of Cambridge as a Postdoc researcher, working with Prof Andreas Vlachos on automated fact-checking! 🎓
Oct - 2023	Our paper LLM-powered Data Augmentation for Enhanced Cross-lingual Performance from my visit at MBZUAI is accepted in the main conference of EMNLP 2023!
Jun - 2023	Excited to share that I have joined Google DeepMind this summer as a Research Intern!
May - 2023	Our paper WebIE: Faithful and Robust Information Extraction on the Web from my internship with Amazon is accepted in the main conference of ACL 2023!

Selected Publications

NAACL

Low-Rank Adaptaion for Multilingual Summarisation: An Empirical Study

Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings, Mostafa Dehghani, Chu-Cheng Lin, and Mirella Lapata

Abs arXiv PDF

Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for conventional fine-tuning, especially in memory-intensive tasks. We investigate the potential of Parameter-Efficient Fine-Tuning, focusing on Low-Rank Adaptation (LoRA), in the domain of multilingual summarization, a task that is both challenging (due to typically long inputs), and relatively unexplored. We conduct an extensive study across different data availability scenarios, including high- and low-data settings, and cross-lingual transfer, leveraging models of different sizes. Our findings reveal that LoRA is competitive with full fine-tuning when trained with high quantities of data, and excels in low-data scenarios and cross-lingual transfer. We also study different strategies for few-shot cross-lingual transfer, finding that continued LoRA tuning outperforms full fine-tuning and the dynamic composition of language-specific LoRA modules.
EMNLP

LLM-powered Data Augmentation for Enhanced Cross-lingual Performance

Chenxi Whitehouse, Monojit Choudhury, and Alham Fikri Aji

Abs arXiv PDF code poster slides

This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited. To achieve this, we utilise several LLMs, namely Dolly-v2, StableVicuna, ChatGPT, and GPT-4, to augment three datasets: XCOPA, XWinograd, and XStoryCloze. Subsequently, we evaluate the effectiveness of fine-tuning smaller multilingual models, mBERT and XLMR, using the synthesised data. We compare the performance of training with data generated in English and target languages, as well as translated English-generated data, revealing the overall advantages of incorporating data generated by LLMs, e.g. a notable 13.4 accuracy score improvement for the best case. Furthermore, we conduct a human evaluation by asking native speakers to assess the naturalness and logical coherence of the generated examples across different languages. The results of the evaluation indicate that LLMs such as ChatGPT and GPT-4 excel at producing natural and coherent text in most languages, however, they struggle to generate meaningful text in certain languages like Tamil. We also observe that ChatGPT falls short in generating plausible alternatives compared to the original dataset, whereas examples from GPT-4 exhibit competitive logical consistency.
ACL

WebIE: Faithful and Robust Information Extraction on the Web

Chenxi Whitehouse, Clara Vania, Alham Fikri Aji, Christos Christodoulopoulos, and Andrea Pierleoni

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics

Abs arXiv PDF code poster

Extracting structured and grounded fact triples from raw text is a fundamental task in Information Extraction (IE). Existing IE datasets are typically collected from Wikipedia articles, using hyperlinks to link entities to the Wikidata knowledge base. However, models trained only on Wikipedia have limitations when applied to web domains, which often contain noisy text or text that does not have any factual information. We present WebIE, the first large-scale, entity-linked closed IE dataset consisting of 1.6M sentences automatically collected from the English Common Crawl corpus. WebIE also includes negative examples, i.e. sentences without fact triples, to better reflect the data on the web. We annotate 21K triples from WebIE through crowdsourcing and introduce mWebIE, a translation of the annotated set in four other languages: French, Spanish, Portuguese, and Hindi. We evaluate the in-domain, out-of-domain, and zero-shot cross-lingual performance of generative IE models and find models trained on WebIE show better generalisability. We also propose three training strategies that use entity linking as an auxiliary task. Our experiments show that adding Entity-Linking objectives improves the faithfulness of our generative IE models.
EACL

Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering

Chenxi Whitehouse, Tillman Weyde, and Pranava Madhyastha

Findings of the Association for Computational Linguistics: EACL 2023

Abs arXiv PDF slides

Providing explanations for visual question answering (VQA) has gained much attention in research. However, most existing systems use separate models for the answers and the explanations. We argue that training explanation models independently of the QA model makes the explanations less grounded and limits performance. To address this, we propose a novel multitask finetuning approach towards a Unified Model for more grounded and consistent generation of both Answers and Explanations (UMAE). To achieve this, we add artificial prompt tokens to training instances and finetune a multimodal encoder-decoder model on various VQA tasks. In our experiments, UMAE models surpass the prior SOTA answer accuracy on A-OKVQA by 10 15%, show competitive results on OK-VQA, achieve new SOTA explanation scores on A-OKVQA and VCR, and demonstrate promising out-of-domain performance on VQA-X.
EMNLP

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

Chenxi Whitehouse, Fenia Christopoulou, and Ignacio Iacobacci

Findings of the Association for Computational Linguistics: EMNLP 2022

Abs arXiv PDF code poster

Accurate alignment between languages is fundamental for improving cross-lingual pre-trained language models (XLMs). Motivated by the natural phenomenon of code-switching (CS) in multilingual speakers, CS has been used as an effective data augmentation method that offers language alignment at word- or phrase-level, in contrast to sentence-level via parallel instances. Existing approaches either use dictionaries or parallel sentences with word-alignment to generate CS data by randomly switching words in a sentence. However, such methods can be suboptimal as dictionaries disregard semantics, and syntax might become invalid after random word switching. In this work, we propose EntityCS, a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. We use Wikidata and the English Wikipedia to construct an entity-centric CS corpus by switching entities to their counterparts in other languages. We further propose entity-oriented masking strategies during intermediate model training on the EntityCS corpus for improving entity prediction. Evaluation of the trained models on four entity-centric downstream tasks shows consistent improvements over the baseline with a notable increase of 10% in Fact Retrieval. We release the corpus and models to assist research on code-switching and enriching XLMs with external knowledge.
AAAI-ICWSM

Evaluation of Fake News Detection with Knowledge-Enhanced Language Models

Chenxi Whitehouse, Tillman Weyde, Pranava Madhyastha, and Nikos Komninos

Proceedings of the International AAAI Conference on Web and Social Media 2022

Abs arXiv PDF code poster slides

Recent advances in fake news detection have exploited the success of large-scale pre-trained language models (PLMs). The predominant state-of-the-art approaches are based on fine-tuning PLMs on labelled fake news datasets. However, large-scale PLMs are generally not trained on structured factual data and hence may not possess priors that are grounded in factually accurate knowledge. The use of existing knowledge bases (KBs) with rich human-curated factual information has thus the potential to make fake news detection more effective and robust. In this paper, we investigate the impact of knowledge integration into PLMs for fake news detection. We study several state-of-the-art approaches for knowledge integration, mostly using Wikidata as KB, on two popular fake news datasets - LIAR, a politics-based dataset, and COVID-19, a dataset of messages posted on social media relating to the COVID-19 pandemic. Our experiments show that knowledge-enhanced models can significantly improve fake news detection on LIAR where the KB is relevant and up-to-date. The mixed results on COVID-19 highlight the reliance on stylistic features and the importance of domain specific and current KBs. The code is available at https://github.com/chenxwh/fake-news-detection.