Recently, a research group led by Prof. LI Hai from the Hefei Institutes of Physical Science of the Chinese Academy of Sciences has developed a new AI medical diagnostic system for zero-shot intelligent diagnosis of chest X-rays, called MultiXpert.
The research results were published in Information Processing and Management.
Chest X-rays are widely used to screen diseases like pneumonia, nodules, and pneumothorax, but reading them manually is time-consuming and depends on expert experience. Traditional AI systems can match experts in some tasks, but they rely on large annotated datasets, making them less effective for new diseases or differences between hospitals. As a result, their generalization is limited and they struggle to meet the needs of complex clinical settings.
In this study, the team proposed a multimodal, dual-stream collaborative enhancement approach and developed MultiXpert, a high-precision zero-shot diagnostic framework that requires no additional annotated data. The model processes image and text information simultaneously, integrating large language models with radiology expert knowledge to refine lesion descriptions. This deep fusion of visual and linguistic information enables the AI to "understand" chest X-rays in a way that approaches clinical reasoning, even for previously unseen diseases.
Experiments show that MultiXpert improves the average AUC (Area Under the Receiver Operating Characteristic Curve) by 7.5% across four single-label public datasets. In zero-shot settings, it outperforms mainstream vision-language models by an average of 3.9%. On multi-center private datasets from ten hospitals, it outperformed traditional supervised models, showing strong cross-center generalization and clinical adaptability.
This work offers a new approach for zero-shot chest X-ray diagnosis and marks a shift in medical AI from relying on annotations to achieving autonomous understanding, according to the team.
Schematic diagram of the structure of MultiXpert, a multimodal dual stream collaborative enhancement model (Image by WANG Tengfei)