Broad Range of Acute Neurological Illnesses Identified by AI Platform

Eric Oermann, MD, Instructor in the Department of Neurosurgery at the Icahn School of Medicine at Mount Sinai

Eric Oermann, MD

An artificial intelligence platform was shown to identify disease in CT scans in 1.2 seconds, faster than human diagnosis, according to a recently published study conducted at the Icahn School of Medicine at Mount Sinai.

The platform was designed to identify a broad range of acute neurological illnesses like stroke, hemorrhage and hydrocephalus. This is the first study to utilize artificial intelligence for detecting a wide range of acute neurologic events, demonstrating a direct clinical application.

“This artificial intelligence program represents the first use of volumetric weakly supervised classification, as well as one of the first randomized controlled trials demonstrating a benefit from a deep learning technique applied to a real clinical problem,” Eric Oermann, MD, senior study author, instructor, department of neurosurgery, Icahn School of Medicine at Mount Sinai, told NeurologyLive. “I think this technique and others in development are all incredibly exciting opportunities for clinicians. Deep learning based technologies ought to lead to increased clinical efficiency and, hopefully, better outcomes.”

Researchers demonstrated a 3D convolutional neural network (3D-CNN) architecture that performed weakly supervised classification, which built on the researcher’s expertise in natural language processing and Mount Sinai’s clinical datasets, to screen head CT images for acute neurologic events.

Features were automatically learned from a clinical radiology dataset which comprised of 37,236 head CT scans and were annotated with a crowdsourcing platform and a natural language processing (NLP) pipeline. The 3D-CNN was trained to identify whether an image contained acute neurological illnesses or non-critical findings for the purpose of triaging studies so that images could be ordered in a work queue based on the probability of a critical finding as opposed to the order of acquisition.

The effectiveness of the approach to triage radiology workflow accelerating time to diagnosis was demonstrated through a randomized, double-blind prospective trial of image interpretation in a simulated clinical environment. Patients included in the test cohort had an average age of 59.7 years and 51% were female. Patient symptoms were mostly non-specific, with the top 3 being headache (27%), altered mental status (17%), and ataxia (9%). The studies were evenly distributed amongst settings, with 36% coming from the emergency department, 33% coming from an inpatient unit, and 31% coming from an outpatient setting. Researchers identified an association between a clinical presentation suggestive of neurological disease and positive findings on imaging (47% vs. 24%, P = .016).

The platform triaged head CT scans based on severity and both the model and radiologists were tested for how quickly findings could be recognized and notification could be provided. Performance of the model was assessed using standard metrics of classification performance, which included the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy, as well as runtime.

Based on the results for predicting critical findings, researchers concluded that a triage system could be constructed to function at a human level of sensitivity and theoretically notify clinicians in 50% of critical cases with a 25% false alarm rate.

The data revealed that the average time for the algorithm to preprocess an image, run its inference method, and if necessary, signal an alarm, was 150 times faster than for clinicians to read an image (1.2s vs. 177s; P = 2e-17), which means that deep learning based technology could meaningfully triage studies both as an alarm mechanism and as a triage/prioritization system.

When prioritizing reports, researchers identified a significant difference in queue position between the urgent and routine cases with more urgent studies appearing earlier in the queue in the prioritized list (P = .01). With a processing and interpretation time of roughly 1 second, a triage system can alert clinicians to a critical finding that may otherwise remain untouched for minutes or hours.

Since the results are based on studies and reports from a single hospital system, external validation in a formal multicenter clinical trial is required.

“On the technical front my research really raises the question as to how far we can push weakly supervised learning techniques,” Oermann added. “My suspicion is that with the right clinical use cases this will be the way forward since it leads to rapid deployment and, hopefully, an immediate impact on patient care. In the long run, however, strongly supervised approaches will be the way we start to achieve human or superhuman performance.”

In the future, the research will entail a shift toward optimizing label study generation, specifying pathological regions on medical images to create strongly supervised learning approaches, and the involvement of the medical community to ensure novel techniques for increasing data efficiency are logistically integrated into current medical practice. The system re-engineering will be accomplished within the next 2 years.

REFERENCE

Titano J, Badgeley M, Schefflein J, et al. Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nature Medicine. 2018.

doi

: 10.1038/s41591-018-0147-y.