An artificial intelligence (AI) algorithm developed by Cincinnati Children’s researchers that identify patients who may benefit from epilepsy surgery is ready to test in adults and at other hospitals. The algorithm runs on a software program embedded into a hospital’s electronic health record (EHR). Using natural language processing techniques, it analyzes previous provider notes for each patient with an upcoming appointment in the epilepsy clinic. The algorithm considers the words, tone, and themes in the notes, and uses a scoring system to identify patients who meet the criteria for surgery evaluation.  When patients are identified, the algorithm sends reminders to providers. Physicians and advanced practice registered nurses who receive these notices are three times more likely to refer patients for a surgery consult, says Judith Dexheimer, Ph.D., a principal study investigator.

The algorithm was translated into clinical care at Cincinnati Children’s in 2018. Since then, Dexheimer; Hansel Greiner, MD, co-director of the hospital’s Epilepsy Surgery Program; and Ben Wissel, MD-PhD candidate at the University of Cincinnati (UC), have completed multiple studies to verify the algorithm’s methods, check the artificial intelligence for racial bias and guide the machine learning tools to dig deeper into the natural language processing to improve the technology. All of this allows providers to refer patients for surgery earlier in their epilepsy journey.

“This means the provider is aware that they are candidates for surgery earlier in the disease course,” Greiner says.

Reducing Time to Surgery

Now that the machine learning algorithm has traction at Cincinnati Children’s, the team is preparing to deploy it at UC Health in Cincinnati, where it will send reminders to physicians who work with adult patients. Epilepsy is one of the leading neurological disorders in the United States, affecting more than 479,000 children. Approximately 30% of epileptic patients have poor seizure control despite using available medications and are potential candidates for neurosurgical intervention. Currently, there are significant delays in making a referral for neurosurgical intervention, even though guidelines exist. In some cases, good candidates are never referred. The machine learning algorithm is a collaboration project with the John Pestian Lab at Cincinnati Children’s and the hospital’s divisions of biostatistics, emergency medicine, informatics, neurology and neurosurgery.The goal is to reduce the time to initial surgery evaluation for patients with intractable epilepsy. The national average in pediatrics is 10 years.

Follow-Up Conversations with Physicians Yield Key Insights

As machine learning pings physicians for certain patients, the research team also follows up on cases where a surgical consult is not made. They talk with providers to learn information the AI can’t capture.

This can include situations where the patient and family are not ready for surgery or want to try another medication before taking a step toward surgery, Dexheimer says. Greiner predicts that this type of follow-up will become part of the standard for clinical care as advanced machine learning is integrated into the medical toolbox.

“There is this responsibility that comes back to the care team to vet the recommendation and then send a response back to the research team and the IT [information technology] team to make the algorithm better,” Greiner says.
Greiner, Dexheimer, and the rest of their team also brainstorm solutions for why the algorithm might fire incorrectly or what items the machine learning tool might miss in the EHR.

Future Applications for Machine Learning

Soon, Greiner and Dexheimer plan to deploy their epilepsy machine-learning algorithm at other health centers. The goal is to answer the question, “Can we use it at a community hospital and identify patients who should be referred to a specialty hospital?” Greiner says. 

Lessons learned while developing an algorithm that spots epilepsy patients who may achieve better outcomes with surgery may also be useful in managing other medical conditions, Greiner says.

“We have a framework that others can use and follow,” he says. “It should mean less work to replicate this with an EHR for other disease states.” 

Receiver operating characteristic (ROC) and precision-recall curves for the pediatric (A, B) and adult (C, D) datasets. The gray dotted line represents the performance of a random classifier. “Demographics + Time + AEDs” represents the baseline logistic regression model that included anti-epileptic drug prescriptions, and “Demographics + Time” represents the baseline model without anti-epileptic drugs.