An explainable multimodal framework for exploring depression-associated speech representations in Alzheimer's disease

Wait 5 sec.

Depression in Alzheimer's disease (AD) is clinically challenging to characterise because affective and cognitive manifestations frequently overlap with the underlying neurodegenerative condition. This study investigates depression-related speech patterns in AD using an explainable multimodal framework that integrates linguistic and acoustic features from elicited picture-description speech. To address the absence of directly observed depression labels in the target AD cohort, we employ a source-informed cross-dataset transfer strategy. Unsupervised clustering identified two distinct speech sub-phenotypes within the AD cohort. By comparing these unsupervised representations to a depression-labelled source corpus, one cluster demonstrated mathematical proximity (measured via cosine similarity in a shared latent space) to the source depression centroid. We explicitly acknowledge that this proximity reflects geometric similarity in a shared latent space, which may encapsulate overlapping domain characteristics alongside affective variance. To validate the structural integrity and separability of these discovered clusters, a proxy supervised classification task was employed. The retained multimodal artificial neural network (ANN) integrated information from both modalities and achieved 92.73% cluster separability accuracy. This demonstrates that the representation-derived two-cluster partition was highly learnable on the independent internal hold-out test set; it does not establish clinical validity, longitudinal stability, or diagnostic validity of the identified sub-phenotypes. For interpretability, Shapley Additive exPlanations (SHAP) analysis was utilised to extract feature attributions and organise influential acoustic and linguistic features into a multimodal taxonomy. Furthermore, a Multimodal SHAP Attribution Magnitude (MSAM) score was derived as an exploratory model-based attribution score rather than a clinically validated severity measure. Ultimately, this study provides an interpretable computational baseline for characterising depression-related speech variance in AD, establishing a structural foundation for future prospective validation using independently labelled clinical cohorts.