KnowLab
May 28, 2025 • Less than a minute read

Paper accepted by ACL 2025 findings – *Look & Mark - Leveraging Radiologist Eye Fixations and Bounding boxes in Multimodal Large Language Models for Chest X-ray Report Generation* – at [DOI:10.18653/v1/2025.findings-acl.909]https://aclanthology.org/2025.findings-acl.909/)

This study introduces Look & Mark (L&M), a novel approach to radiology report generation that integrates radiologist fixation cues (Look) with bounding box annotations (Mark) to guide multimodal Large Language Models (LLMs). By combining these complementary grounding strategies, L&M significantly improves the clinical relevance of generated reports, reduces hallucinations, and enhances model alignment with real-world diagnostic workflows. Importantly, L&M achieves these gains without requiring extensive fine-tuning, leveraging in-context learning to adapt both general-purpose and domain-specific models alike.

Our experiments demonstrate that L&M significantly boosts performance across both lexical and clinical evaluation metrics, with the largest gains observed in clinical metrics such as RaTEScore and RadGraph-XL. For instance, CXR-LLaVA achieved a 1.2% improvement in overall metrics (A.AVG) compared to baseline prompting, while LLaVA-Med demonstrated a remarkable 9.2% boost. General-purpose models also benefited significantly, with LLaVA-OV achieving an 87.3% clinical average (C.AVG), the highest among all tested models, even surpassing domain-specific models trained explicitly for chest X-ray report generation.

Read the full paper at [DOI:10.18653/v1/2025.findings-acl.909]https://aclanthology.org/2025.findings-acl.909/).

Kim, Yunsoo, Jinge Wu, Su-Hwan Kim, Pardeep Vasudev, Jiashu Shen, and Honghan Wu. “Look & Mark: Leveraging Radiologist Eye Fixations and Bounding boxes in Multimodal Large Language Models for Chest X-ray Report Generation.” In Findings of the Association for Computational Linguistics: ACL 2025, pages 17680–17694, Vienna, Austria. Association for Computational Linguistics.[DOI:10.18653/v1/2025.findings-acl.909]https://aclanthology.org/2025.findings-acl.909/).

Post by: KnowLab