Options
Transforming heart disease detection with BERT: Novel architectures and fine-tuning techniques
Date Issued
2024
ISBN
9798331510503
9798331510510
ISSN
2473-7569
2334-3141
Citation
Usman Javed, M., Ali Shafqat, M., Iqbal, A., Azhar, M., Aslam, S. M., & Shabbir, I. (10 Dec 2024). Transforming heart disease detection with BERT: Novel architectures and fine-tuning techniques. FIT 2024, Islamabad Pakistan.
Type
Conference Paper
Abstract
The prediction of heart disease is crucial for effective prevention and treatment. However, extracting clinical infor-mation such as CAD, smoking, hypertension, hyperlipidemia, obesity, and family history of CAD from unstructured electronic health records (EHRs) poses significant challenges to clinicians. This research introduces a novel approach that leverages an ensemble of transfer learning algorithms combined with a multi-head attention mechanism to automatically extract heart disease risk factors from EHRs. Various deep learning models, including BERT, BioBERT, BioClinical BERT, RoBERTa, and XLNet, were initially trained on medical data sets and subsequently fine-tuned on the i2b2 clinical data set. Individual models delivered strong results, with RoBERTa achieving the highest accuracy of 95. 27% and an F1 score of 94. 94%. BioBERT, BioClinical BERT, XLNet, and BERT also performed well, with precision ranging from 94. 73% to 95. 03%. However, the proposed ensemble model with multi-head attention outperformed all, achieving an accuracy of 96.35% and the F1-score of 95.76%. These findings highlight the superior ability of the ensemble model to capture complex inter-dependencies between heart disease risk factors, making it a robust tool for clinical prediction.
Subjects
Loading...
Availability at HKSYU Library
