Explainable Multimodal AI for In-Hospital Detection of Lung Cancer and Heart Failure Using Routine Data: Development, Internal Validation, and Simulated Federated Evaluation

Wait 5 sec.

Objective: To determine whether combining four routine data sources improves in-hospital detection of lung cancer and heart failure, which sources drive each prediction, whether explanations are faithful, whether models are fair, and how much benefit federated training retains. Methods: In this retrospective single-centre study, we linked MIMIC-IV admissions (73,637 patients) to chest radiographs, radiology reports, 12-lead electrocardiograms (ECGs) and laboratory tests from the first 48 hours. For each disease, excluding prior cases, we compared single-source models, gradient boosting and early, late and attention-based fusion on patient-level splits. We tested explanations for faithfulness and weight dependence, audited performance by sex, age, race and ethnicity, and insurance, and simulated five unequal hospitals to compare federated, centralised and local training. Results: Lung cancer was coded in 1.7% and heart failure in 15.6% of patients. For lung cancer, attention fusion reached an area under the ROC curve (AUROC) of 0.833 (95% CI 0.809-0.859) against 0.753 for gradient boosting on laboratory data; for heart failure, 0.885 (0.878-0.892) against 0.794 (both p < 0.001). Early and late fusion performed similarly. Laboratory data and radiology reports contributed most to lung cancer, and the ECG to heart failure. Explanations passed a deletion test; a weight-randomisation check on absolute attributions passed for ECG but not for laboratory explanations. For heart failure, discrimination fell with age. Federated training recovered 95% of the centralised gain over local training for heart failure; for lung cancer, federated averaging (FedAvg) recovered 51% and FedProx 82%. Conclusion: Fusing routine data improved in-hospital detection of both diseases over laboratory data alone, and which data matter depends on the disease. Explanation checks should be run per source and per similarity measure, and performance in older patients monitored. For the rare outcome, FedProx, which limits drift between sites, retained more of the pooled benefit than FedAvg. External validation is needed.