Dialogue state tracking (DST) is an important step in dialogue management to keep track of users' beliefs. Existing works fine-tune all language model (LM) parameters to tackle the DST task, which requires significant data and computing resources for training and hosting. The cost grows exponentially in the real-world deployment where dozens of fine-tuned LM are used for different domains and tasks. To reduce parameter size and better utilize cross-task shared information, we propose to use soft prompt token embeddings to learn task properties. Without tuning LM parameters, our method drastically reduces the number of parameters needed to less than 0.5% of prior works while achieves better low-resource DST performance.
Introducing our #Interspeech2023 paper "Parameter-Efficient Low-Resource #Dialogue State Tracking by #PromptTuning". We show that you can outperform SOTA DST models while only updating 0.08% of model parameters given limited training data.
— Mingyu Derek MA (@mingyu_ma) August 22, 2023
More: https://t.co/bUTALYD2UK 1/3 pic.twitter.com/kJRIKGC3zE
Incorporating task metadata in the learning objective by reiteration and providing explicit segment information are especially helpful for low-resource generative #InformationExtraction. There are smaller performance gains for slots with open-ended answers by prompt tuning. 2/3
— Mingyu Derek MA (@mingyu_ma) August 22, 2023
This work is done with my amazing collaborators at @AmazonScience: Jiun-Yu Kao, Shuyang Gao, Arpit Gupta, Di Jin @kingsquare2, Tagyoung Chung and @VioletNPeng.
— Mingyu Derek MA (@mingyu_ma) August 22, 2023
We will have an oral presentation in the Thu-O5 session during 10:00-10:20 at Wicklow Hall 1 at #Interspeech2023🇮🇪. 3/3