An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision–Language Pretraining
Published in Nature Biomedical Engineering 2026, 2026
Recommended citation: Yuxiang Nie*, Sunan He*, Yequan Bie*, Yihui Wang, Zhixuan Chen, Shu Yang, Zhiyuan Cai, Linshan Wu, Hongmei Wang, Xi Wang, Ngai Shing Cheng, Luyang Luo, Mingxiang Wu, Haibo Jin, Xian Wu, Ronald Cheong Kin Chan, Yuk Ming Lau, Zhengyu Zhang, Sushan Xiao, Can Yang, Yinghua Zhao, Xiaohui Duan, Li Zhang, Li Liang, Yefeng Zheng, Pranav Rajpurkar, and Hao Chen. 2026. An explainable biomedical foundation model via large-scale concept-enhanced vision–language pretraining. Nature Biomedical Engineering. https://doi.org/10.1038/s41551-026-01764-x. (* equal contribution) https://www.nature.com/articles/s41551-026-01764-x
Abstract
Accurate medical imaging AI also needs to be interpretable to clinicians. In this work, we introduce ConceptCLIP, an explainable biomedical foundation model that integrates medical concepts directly into large-scale vision–language pretraining. We curate MedConcept-23M, a dataset containing 23 million biomedical image–text–concept triplets, and train ConceptCLIP with joint global image–text alignment and fine-grained region–concept alignment. This design enables the model to learn transferable biomedical representations while grounding its predictions in human-interpretable medical concepts.
Comprehensive evaluation across 78 datasets spanning 10 medical imaging modalities shows that ConceptCLIP achieves strong diagnostic performance while supporting a broad range of downstream tasks, including cross-modal retrieval, visual question answering, medical report generation, pathology whole-slide image analysis, and concept-based explainability. A clinician user study across three imaging modalities further shows that ConceptCLIP’s concept-based explanations can help clinicians verify model predictions and identify potential errors. These results demonstrate the potential of concept-enhanced foundation models to advance trustworthy and interpretable AI for medical imaging.
MedConcept-23M dataset available here
Recommended citation: Yuxiang Nie, Sunan He, Yequan Bie, Yihui Wang, Zhixuan Chen, Shu Yang, Zhiyuan Cai, Linshan Wu, Hongmei Wang, Xi Wang, Ngai Shing Cheng, Luyang Luo, Mingxiang Wu, Haibo Jin, Xian Wu, Ronald Cheong Kin Chan, Yuk Ming Lau, Zhengyu Zhang, Sushan Xiao, Can Yang, Yinghua Zhao, Xiaohui Duan, Li Zhang, Li Liang, Yefeng Zheng, Pranav Rajpurkar, and Hao Chen. 2026. An explainable biomedical foundation model via large-scale concept-enhanced vision–language pretraining. Nature Biomedical Engineering. https://doi.org/10.1038/s41551-026-01764-x. ( equal contribution)