Title : AI-driven heart disease risk prediction using transformers: Insights from Framingham and Cleveland datasets
Abstract:
Background: Accurate prediction of cardiovascular risk is essential for timely prevention and intervention. While classical Machine Learning (ML) models such as XGBoost and Random Forest have shown promise, they often struggle with class imbalance and limited capacity to capture nonlinear interactions. In this study, we evaluate the utility of transformer-based deep learning models for predicting Cardiovascular Disease (CVD) risk using the Framingham Heart Study dataset (n = 4,240) and the Cleveland Heart Disease dataset (n = 303), incorporating interaction terms and explainability via SHAP (SHapley Additive exPlanations).
Methods: We compared FT-Transformer, SAINT, TabNet, XGBoost, LightGBM, Random Forest, and logistic regression-based stacking. SMOTETomek was used to address class imbalance (15% positive class in Framingham), and missing values were imputed using multivariate imputation by chained equations (MICE). Five-fold cross-validation was performed on both datasets.
The feature set included demographics (age, sex), vital signs (blood pressure, heart rate, mean arterial pressure, pulse pressure), laboratory markers (cholesterol, glucose, renal and electrolyte panels), clinical history (diabetes, hypertension, medications, smoking, alcohol, thalassemia), symptoms (chest pain, angina), and ECG findings and cardiac parameters (ST changes, max heart rate, vessel visualization). Mean arterial pressure and pulse pressure were introduced as engineered features along with existing variables in the Farmingham dataset.
Results: The FT-Transformer achieved the best performance on both datasets—Framingham: F1 score 0.82, accuracy 0.87, AUC 0.91; Cleveland: F1 score 0.89, accuracy 0.93, AUC 0.95. Transformer-based models consistently outperformed classical ML models and ensemble methods. Stacking underperformed relative to its base learners. SHAP analysis consistently identified mean arterial pressure, age, cholesterol levels, prior cardiovascular events, and smoking intensity as the most predictive features across high-performing models.
Conclusion: Transformer-based models offer substantial advantages in modelling imbalanced clinical datasets by effectively capturing complex variable interactions. FT-Transformer outperformed established ML baselines while maintaining interpretability through SHAP. These findings support the use of transformer-based architectures in CVD risk prediction workflows, with the potential to enhance clinical trust and utility through transparent, data-driven insights.
