The articles related to Multimodal deep learning in neuroimaging are represented in the table. There are collection of articles related to classification task in psychiatry and neurology based on DL methods. In each article some neuroimaging data (sMRI, fMRI, PET, DTI, EEG, MEG, MultiOmics) are fused with each other for improving model predictions. The source code of proposed method is attached if it available. TO BE UPDATED
| Article | Disease | Data/Preprocessing | Method | Fusion technique | Result | Code |
|---|---|---|---|---|---|---|
| Multimodal deep learning models for early detection of Alzheimer’s disease stage | Alzheimer’s disease classification | Data: ADNI Count: 220 patients Modality: sMRI,Genetic, Clinical Preprocessing: ART toolbox, extract 3D areas of 21 brain regions (associated with Alzheimer’s disease),Features including the brain volumes, voxel intensities, and texture based features. We extract features such as energy, entropy, and 13 Haralick texture features | 3D CNN + Sparse Denoising autoencoders + Dense network | Early,Intermediate, Late | 0.8% +- 0.03% acc (Control vs MCI) | |
| Multimodal deep learning for Alzheimer’s disease dementia assessment | Alzheimer’s disease classification | Data: ADNI Modality: sMRI, Clinical, demographics, past medical history, neuropsychological testing, functional assessments Preprocessing FMRIB Software Library v6.0 (FSL) for MRI | 3D CNN + CatBoost | Late | 0.804 ± 0.011 acc (Control vs MCI) | code |
| Combining Neuroimaging and Omics Datasets for Disease Classification Using Graph Neural Networks | Parkinson's disease (PD) classification | Data: PPMI Modality: fMRI, DTI, multi-omics (RNA Expression, Single Nucleotide Polymorphism (SNP), DNA Methylation and non-coding RNA) Preprocessing: fmriPrep, simple noise removal and Wilcoxon signed rank test for omics data | CycleGAN on structural and functional connectomes to generate missing imaging modalities, JOIN-GCLA (Joining Omics and Imaging Networks via Graph Convolutional Layers and Attention) | Early | - | - |
| Interpretable Graph Convolutional Network of multi-modality brain imaging for Alzheimer’s disease diagnosis | Alzheimer’s disease classification | Data: ADNI Count: 182 HC subjects, 476MCI subjects, and 97 AD subjects Modality: sMRI, FDG-PET,(AV45-PET) Preprocessing: SPM software | Graph Convolutional Neural Network | Early | 0.818 ±.031 (MCI vs HC vs Alzheimer) | - |
| Multi-View Imputation and Cross-Attention Network Based on Incomplete Longitudinal and Multi-Modal Data for Alzheimer’s Disease Prediction | Alzheimer’s disease classification, Alzheimer’s longitudinal classification | Data: ADNI , OASIS Count: 1387 Modality: sMRI, FDG-PET Preprocessing 90-dimensional region-of interest (ROI) features were separately extracted from the MRI and PET data for each subject. | RNN-based network with cross-attention block (MCNet) | Intermediate | 0.830 ± 0.019 ADNI-1 | code |
| Deep Learning-Based Feature Representation for AD/MCI Classification | Alzheimer’s disease classification | Data: ADNI Count: 99 MCI,52 HC Modality:MRI, PET, and CSF Preprocessing: 93 region-of-interest-based volumetric features from MRI and 93 mean intensity from PET, 3 CSF biomarkers of Aβ42, t-tau, and p-tau. | Stacked AutoEncoder +SK-SVM Stacked AutoEncoder+ multi-kernel SVM | Intermediate | MK-SVM 0.850±0.012 acc (MCI vs Control) | |
| Multi-Center and Multi-Channel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network | Alzheimer’s disease classification | Data: ADNI Count 163 NC, 44 SMC, 86 early MCI (EMCI), and 166 late MCI (LMCI) Modality: fMRI, DTI Preprocessing: GRETNA toolbox for fMRI data, PANDA toolbox for DTI data -> AAL template with 90 ROIs for each modality | Graph Convolutional Neural Network | Early | NC vs. SMC 93.23 acc, NC vs. EMCI 91.16 acc, NC vs. LMCI 94.22 acc | code |
| M-GCN: A Multimodal Graph Convolutional Network to Integrate Functional and Structural Connectomics Data to Predict Multidimensional Phenotypic Characterizations | Predict Phenotypic measures | Data: Human Connectome Project, Kennedy Krieger Institute dataset Modality: fMRI, DTI Preprocessing CompCorr and HCP pipeline for rs-fMRI, connectivity matrices with Automatic Anatomical Labeling (AAL) atlas; FSL and standard Neurodata MR Graphs package for DTI: | Graph Convolutional Neural Network | Intermediate | CFIS (Cognitive Fluid Intelligence Score) - 12.87 ± 9.65 MAE , ADOS - 2.71 ± 2.15 MAE, SRS -16.50 ± 9.44 MAE | code |