News

Mar 13, 2024 Conference
To be presented at NAACL 2024 🇲🇽

Three papers are accepted to NAACL 2024 main conference!

  • In Mitigating Bias for Question Answering Models by Tracking Bias Influence, we present an approach to mitigate the bias of multiple-choice QA models. Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance by observing its influence on another instance. We then use the bias level detected as an optimization objective to form a multi-task learning setting in addition to the original QA task.
  • In Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models, we demonstrate that an attacker can inject backdoors by issuing very few malicious instructions among thousands of gathered data and control model behavior through data poisoning. Through such instruction attacks, the attacker can achieve over 90% attack success rate across four commonly used NLP datasets, and cause persistent backdoors that are easily transferred to 15 diverse datasets zero-shot.
  • In Instructional Fingerprinting of Large Language Models, we present a pilot study on LLM fingerprinting as a form of very lightweight instruction tuning. Model publisher specifies a confidential private key and implants it as an instruction backdoor that causes the LLM to generate specific text when the key is present. Results on 11 popularly-used LLMs showed that this approach prevents publisher overclaim, maintains robustness against fingerprint guessing and parameter-efficient training, and supports multi-stage fingerprinting akin to MIT License.
Feb 22, 2024 Conference, Talk, Poster, Demo
Presenting at AAAI 2024 🇨🇦
TimeLocationActivity
Demo Session 1, Feb 22 Thu, 19:00-21:00Exhibit Hall AB1Demo presentation: MIDDAG: Where Does Our News Go? Investigating Information Diffusion via Community-Level Information Pathways. We present an intuitive, interactive system that visualizes the information propagation paths on social media triggered by COVID-19-related news articles accompanied by comprehensive insights including user/community susceptibility level, as well as events and popular opinions raised by the crowd while propagating the information.
Poster Session 2, Feb 23 Fri, 19:00-21:00Exhibit Hall AB1Poster presentation of the paper: STAR: Improving Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models. We present a structure-to-text data generation method for complicated structure prediction tasks that first generates complicated event structures (Y) and then generates input passages (X), all with Large Language Models. We show that the data generated by STAR significantly improves the performance of low-resource event extraction and relation extraction tasks, even surpassing the effectiveness of human-curated data.
Jan 21, 2024 Preprint
New preprint on LLM ownership protection
In InstructionalFingerprint, we present a pilot study on LLM fingerprinting as a form of very lightweight instruction tuning. Model publisher specifies a confidential private key and implants it as an instruction backdoor that causes the LLM to generate specific text when the key is present. Results on 11 popularly-used LLMs showed that this approach is lightweight and does not affect the normal behavior of the model. It also prevents publisher overclaim, maintains robustness against fingerprint guessing and parameter-efficient training, and supports multi-stage fingerprinting akin to MIT License.
Oct 1, 2023 Preprint
New preprint on bias mitigation
In BMBI, we propose to mitigate bias exhibited in QA models by observing the query instance’s influence on another instance, enabling bias mitigation with extremely low resources. With our method, bias levels in multiple bias categories can be reduced without using category-specific instance-level annotation.
Aug 1, 2023 Conference, Talk
Presenting at INTERSPEECH 2023 🇮🇪
TimeLocationActivity
Aug 24 Thu, 10:00-10:20 (IST)Wicklow Hall 1Oral presentation of the conf paper: Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning. In the collaboration work with Amazon Alexa AI, we introduce a dialogue state tracking model tuning less than 1% of LM parameters and achieves better low-resource performance with prompt tuning techniques.
Jul 10, 2023 Resources, Teaching
Data Science for ALL
Check out course materials for the 2-week summer program “Data Science for ALL” that were just delivered by our joint team from UCI and UCLA!
Jul 1, 2023 Conference, Talk, Poster
Presenting at ACL 2023 🇨🇦
TimeLocationActivity
July 10, 11:00-12:30 (EDT)Metropolitan EastOral presentation of the main conf paper: DICE: Data-Efficient Clinical Event Extraction with Generative Models. We introduce a data-efficient generative model with specialized mention boundary predictions for clinical event extraction. We also propose MACCROBAT-EE, the first benchmark in the domain.
July 10, 19:00-21:00 (EDT)Metropolitan CentreFindings Spotlights of Multi-hop Evidence Retrieval for Cross-document Relation Extraction. We propose a multi-hop evidence retrieval method based on evidence path mining and ranking with dense retrievers, which shows great cross-document relation extraction performance.
July 11, 16:15-17:45 (EDT)Metropolitan CentreOral presentation of the main conf paper: Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction? We leverage additional cross-domain supervision signals from Natural Language Inference on the Relation Extraction task. We show that it achieves state-of-the-art biomedical RE performance.
July 14, 11:40-12:30 (EDT)–Poster presentation at the TrustNLP: Third Workshop on Trustworthy Natural Language Processing
May 15, 2023 Preprint
New preprints on data generation with LLMs and LLM's backdoor attack
In STAR, we propose to synthesize training data by structure-to-text generation using Large Language Models and we show that the generated data is even more effective than human-curated data instances to boost the low-resource event extraction performance.
In the new study about LLM’s backdoor attack, we demonstrate that an attacker can inject backdoors by issuing very few malicious instructions and control model behavior through data poisoning.
May 1, 2023 Conference, Talk
New INTERSPEECH paper 🇮🇪
New INTERSPEECH paper! In the collaboration work with Amazon Alexa AI, we introduce a dialogue state tracking model tuning less than 1% of LM parameters and achieves better low-resource performance with prompt tuning techniques.
Dec 15, 2022 Preprint
New preprints on biomedical relation extraction and cross-document relation extraction
We propose to formulate biomedical relation extraction as the NLI task. We design a new evidence retrieval technique for cross-document relation extraction.