Presenting at NeurIPS 2024 🇨🇦

Dec 9, 2024

Time	Location	Activity
Wed 11 a.m. — 2 p.m.	East Exhibit Hall A-C #4908	Poster at the main conference: GraphVis: Boosting LLMs with Visual Knowledge Graph Integration.
Sat 4:55 p.m. - 5:30 p.m.	East Meeting Room 16	Poster at the GenAI for Health Workshop: Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction.
Sat 4:30 p.m. - 5:30 p.m.	West Meeting Room 301	Poster at the Workshop on Efficient Natural Language and Speech Processing: Inferring from Logits: Exploring Best Practices for Decoding-Free Generative Candidate Selection.
Sun 12:00 p.m. - 12:05 p.m.	West Exhibition Hall A	Spotlight at the Multimodal Algorithmic Reasoning Workshop: Are Large-Language Models Graph Algorithmic Reasoners?.

Presenting Papers

GraphVis: Boosting LLMs with Visual Knowledge Graph Integration

Yihe Deng, Chenchen Ye, Zijie Huang, Mingyu Derek Ma, Yiwen Kou, Wei Wang

NeurIPS, 2024

GraphVis conserves the intricate graph structure through the visual modality to enhance the comprehension of KGs with the aid of Large Vision Language Models (LVLMs). Our approach incorporates a unique curriculum fine-tuning scheme which first instructs LVLMs to recognize basic graphical features from the images, and subsequently incorporates reasoning on QA tasks with the visual graphs.

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

Mingyu Derek Ma, Xiaoxuan Wang, Yijia Xiao, Anthony Cuturrufo, Vijay S Nori, Eran Halperin, Wei Wang

NeurIPS GenAI4Health & AAAI 2024 Spring Symposium on Clinical Foundation Models, 2024

We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes.

Inferring from Logits: Exploring Best Practices for Decoding-Free Generative Candidate Selection

Mingyu Derek Ma^*, Yanna Ding^*, Zijie Huang, Jianxi Gao, Yizhou Sun, Wei Wang

NeurIPS ENLSP, 2024

Existing works have been using decoding-free candidate selection methods to obtain candidate probability from initial output logits over vocabulary. Though these estimation methods are widely used, they are not systematically evaluated, especially on end tasks. We introduce an evaluation of a comprehensive collection of decoding-free candidate selection approaches.

Are Large-Language Models Graph Algorithmic Reasoners?

Alexander K. Taylor, Anthony Cuturrufo, Vishal Yathish, Mingyu Derek Ma, Wei Wang

arXiv, 2024

We introduce a novel benchmark designed to evaluate LLM performance on classical algorithmic reasoning tasks on explicit graphs. Our findings highlight the persistent challenges LLMs face in this domain and underscore the necessity for advanced prompting techniques and algorithmic instruction to enhance their graph reasoning abilities.

Last updated on Dec 9, 2024

Conference NeurIPS

Awarded the J.P. Morgan Chase AI PhD Fellowship 🏆 Sep 29, 2024 →