Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy
Selection of antiseizure medications (ASMs) for epilepsy remains largely a trial-and-error approach. Under this approach, many patients have to endure sequential trials of ineffective treatments until the ``right drugs'' are prescribed. OBJECTIVE To develop and validate a deep learning mod...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Published: |
Amer Medical Assoc
2022
|
| Subjects: | |
| Online Access: | http://eprints.um.edu.my/41163/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| id |
my.um.eprints.41163 |
|---|---|
| record_format |
eprints |
| spelling |
my.um.eprints.411632023-09-11T07:54:58Z http://eprints.um.edu.my/41163/ Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy Hakeem, Haris Feng, Wei Chen, Zhibin Choong, Jiun Brodie, Martin J. Fong, Si-Lei Lim, Kheng-Seang Wu, Junhong Wang, Xuefeng Lawn, Nicholas Ni, Guanzhong Gao, Xiang Luo, Mijuan Chen, Ziyi Ge, Zongyuan Kwan, Patrick R Medicine Selection of antiseizure medications (ASMs) for epilepsy remains largely a trial-and-error approach. Under this approach, many patients have to endure sequential trials of ineffective treatments until the ``right drugs'' are prescribed. OBJECTIVE To develop and validate a deep learning model using readily available clinical information to predict treatment success with the first ASM for individual patients. DESIGN, SETTING, AND PARTICIPANTS This cohort study developed and validated a prognostic model. Patients were treated between 1982 and 2020. All patients were followed up for a minimum of 1 year or until failure of the first ASM. A total of 2404 adults with epilepsy newly treated at specialist clinics in Scotland, Malaysia, Australia, and China between 1982 and 2020 were considered for inclusion, of whom 606 (2S.2%) were excluded from the final cohort because of missing information in 1 or more variables. EXPOSURES One of 7 antiseizure medications. MAIN OUTCOMES AND MEASURES With the use of the transformer model architecture on 16 clinical factors and ASM information, this cohort study first pooled all cohorts for model training and testing. The model was trained again using the largest cohort and externally validated on the other 4 cohorts. The area under the receiver operating characteristic curve (AUROC), weighted balanced accuracy, sensitivity, and specificity of the model were all assessed for predicting treatment success based on the optimal probability cutoff. Treatment success was defined as complete seizure freedom for the first year of treatment while taking the first ASM. Performance of the transformer model was compared with other machine learning models. RESULTS The final pooled cohort included 1798 adults (54.5% female; median age, 34 years IQR, 24-50 years)). The transformer model that was trained using the pooled cohort had an AUROC of 0.65 (95% CI, 0.63-0.67) and a weighted balanced accuracy of 0.62 (95% CI, 0.60-0.64) on the test set. The model that was trained using the largest cohort only had AUROCs ranging from 0.52 to 0.60 and a weighted balanced accuracy ranging from 0.51 to 0.62 in the external validation cohorts. Number of pretreatment seizures, presence of psychiatric disorders, electroencephalography, and brain imaging findings were the most important clinical variables for predicted outcomes in both models. The transformer model that was developed using the pooled cohort outperformed 2 of the 5 other models tested in terms of AUROC. CONCLUSIONS AND RELEVANCE In this cohort study, a deep learning model showed the feasibility of personalized prediction of response to ASMs based on clinical information. With improvement of performance, such as by incorporating genetic and imaging data, this model may potentially assist clinicians in selecting the right drug at the first trial. Amer Medical Assoc 2022-10 Article PeerReviewed Hakeem, Haris and Feng, Wei and Chen, Zhibin and Choong, Jiun and Brodie, Martin J. and Fong, Si-Lei and Lim, Kheng-Seang and Wu, Junhong and Wang, Xuefeng and Lawn, Nicholas and Ni, Guanzhong and Gao, Xiang and Luo, Mijuan and Chen, Ziyi and Ge, Zongyuan and Kwan, Patrick (2022) Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy. JAMA Neurology, 79 (10). pp. 986-996. ISSN 2168-6149, DOI https://doi.org/10.1001/jamaneurol.2022.2514 <https://doi.org/10.1001/jamaneurol.2022.2514>. 10.1001/jamaneurol.2022.2514 |
| institution |
Universiti Malaya |
| building |
UM Library |
| collection |
Institutional Repository |
| continent |
Asia |
| country |
Malaysia |
| content_provider |
Universiti Malaya |
| content_source |
UM Research Repository |
| url_provider |
http://eprints.um.edu.my/ |
| topic |
R Medicine |
| spellingShingle |
R Medicine Hakeem, Haris Feng, Wei Chen, Zhibin Choong, Jiun Brodie, Martin J. Fong, Si-Lei Lim, Kheng-Seang Wu, Junhong Wang, Xuefeng Lawn, Nicholas Ni, Guanzhong Gao, Xiang Luo, Mijuan Chen, Ziyi Ge, Zongyuan Kwan, Patrick Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| description |
Selection of antiseizure medications (ASMs) for epilepsy remains largely a trial-and-error approach. Under this approach, many patients have to endure sequential trials of ineffective treatments until the ``right drugs'' are prescribed. OBJECTIVE To develop and validate a deep learning model using readily available clinical information to predict treatment success with the first ASM for individual patients. DESIGN, SETTING, AND PARTICIPANTS This cohort study developed and validated a prognostic model. Patients were treated between 1982 and 2020. All patients were followed up for a minimum of 1 year or until failure of the first ASM. A total of 2404 adults with epilepsy newly treated at specialist clinics in Scotland, Malaysia, Australia, and China between 1982 and 2020 were considered for inclusion, of whom 606 (2S.2%) were excluded from the final cohort because of missing information in 1 or more variables. EXPOSURES One of 7 antiseizure medications. MAIN OUTCOMES AND MEASURES With the use of the transformer model architecture on 16 clinical factors and ASM information, this cohort study first pooled all cohorts for model training and testing. The model was trained again using the largest cohort and externally validated on the other 4 cohorts. The area under the receiver operating characteristic curve (AUROC), weighted balanced accuracy, sensitivity, and specificity of the model were all assessed for predicting treatment success based on the optimal probability cutoff. Treatment success was defined as complete seizure freedom for the first year of treatment while taking the first ASM. Performance of the transformer model was compared with other machine learning models. RESULTS The final pooled cohort included 1798 adults (54.5% female; median age, 34 years IQR, 24-50 years)). The transformer model that was trained using the pooled cohort had an AUROC of 0.65 (95% CI, 0.63-0.67) and a weighted balanced accuracy of 0.62 (95% CI, 0.60-0.64) on the test set. The model that was trained using the largest cohort only had AUROCs ranging from 0.52 to 0.60 and a weighted balanced accuracy ranging from 0.51 to 0.62 in the external validation cohorts. Number of pretreatment seizures, presence of psychiatric disorders, electroencephalography, and brain imaging findings were the most important clinical variables for predicted outcomes in both models. The transformer model that was developed using the pooled cohort outperformed 2 of the 5 other models tested in terms of AUROC. CONCLUSIONS AND RELEVANCE In this cohort study, a deep learning model showed the feasibility of personalized prediction of response to ASMs based on clinical information. With improvement of performance, such as by incorporating genetic and imaging data, this model may potentially assist clinicians in selecting the right drug at the first trial. |
| format |
Article |
| author |
Hakeem, Haris Feng, Wei Chen, Zhibin Choong, Jiun Brodie, Martin J. Fong, Si-Lei Lim, Kheng-Seang Wu, Junhong Wang, Xuefeng Lawn, Nicholas Ni, Guanzhong Gao, Xiang Luo, Mijuan Chen, Ziyi Ge, Zongyuan Kwan, Patrick |
| author_facet |
Hakeem, Haris Feng, Wei Chen, Zhibin Choong, Jiun Brodie, Martin J. Fong, Si-Lei Lim, Kheng-Seang Wu, Junhong Wang, Xuefeng Lawn, Nicholas Ni, Guanzhong Gao, Xiang Luo, Mijuan Chen, Ziyi Ge, Zongyuan Kwan, Patrick |
| author_sort |
Hakeem, Haris |
| title |
Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| title_short |
Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| title_full |
Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| title_fullStr |
Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| title_full_unstemmed |
Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| title_sort |
development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy |
| publisher |
Amer Medical Assoc |
| publishDate |
2022 |
| url |
http://eprints.um.edu.my/41163/ |
| _version_ |
1778161634653503488 |
| score |
13.211869 |