A convolutional neural network developed by a Tempus Labs / Geisinger Health collaboration received breakthrough medical device status for silent AF detection. Like many other ECGs that stare at AIs, all it needs is a patient’s baseline ECG!
The model
This model was developed through deep learning – the algorithm was fed the raw data from patients’ ECGs, without any labeling whatsoever. All the AI had to work with was raw electrical data coming from patients’ hearts. To accomplish such deep learning, a veritable wealth of data is required. And in this aspect, Geisinger delivered – the Pennsylvania based healthcare provider gave researchers a staggering 2.8 million ECGs to work with!
Researchers threw away any and all abnormal ECGs, leaving 1.6m pristine, apparently normal ECGs for the AI to train on. Through Geisinger’s EHR records, researchers knew which patients ended up getting diagnosed with AF in the year after the ECG was taken. In the end, the AI model was capable of detecting these patients with “1-year incident AF” with 69% sensitivity and 81% specificity.
Is this the ultimate AI that stares at ECGs?
The sensitivity and specificity figures don’t sound that impressive, but consider this; even a seasoned cardiologist could tell you nothing of a patients’ odds of having AF by looking at a perfectly normal ECG. The fact that this model does have some useful insights to give is almost magical, as arrhythmia expert Professor Michael A. Rosenberg states in their editorial.
In our opinion, this neural network is likely the best we’ve seen yet. A massive, “clean” dataset coupled with meticulous methodology means that we’re unlikely to see a better iteration of an “AI that stares at ECGs” soon. Which begs the question – is this the limit of ECG-based AI detection? It is not at all unlikely that there just isn’t enough data in a simple, 5-second surface ECG to always paint the complete picture about a patients’ atrial electromechanical status. Maybe future algorithms trained on smaller samples and with arguably sloppier development processes will just overestimate their accuracy because of overfitting. Maybe machine learning approaches that are still in their infancy, coupled with globally-sourced data amounting in the tens of millions will achieve unbelievable levels of accuracy. Only time – and considerable effort on the part of scientists – will tell!
AIs finally get to see action
Regardless of future fine-tuning of AF detecting models, the fact that the Tempus algorithm got FDA clearance for actual use in the field is a huge badge of merit. There are definitely ways to take advantage of the algorithms available to day, no matter how nascent they may seem. Drawing parallels from the world of self-driving cars, AI algorithms don’t need to achieve perfection to enter clinical use. They just need to perform better than current tests and approaches. And, to be frank, our current options for classifying patients based on their AF risk are very, very limited.