Google AI researchers have shown how artificial intelligence can make ophthalmologists more effective. Ophthalmologists are doctors who treat eye diseases, i.e., eye doctors. The researchers showed that together, physicians and AI can improve eye care.
AI, which stands for Artificial Intelligence, continues to evolve rapidly. It is speeding up disease diagnosis. In fact, AI can sometimes diagnose more accurately than doctors. Healthcare professionals and researchers today suggest that technology might soon replace tasks that doctors currently perform.
However, according to the conclusions of a Google AI research group study, algorithms and physicians are most effective when they work together. The researchers, from Google Research and other centers of excellence, wrote about their work in the journal Ophthalmology (citation below).
The authors said that theirs was one of the first studies to examine AI and physicians working together. The researchers specifically focused on diagnostic accuracy.
Google algorithm diagnosed as well as ophthalmologists
Google AI had shown in a previous study that its algorithm diagnosed diabetic retinopathy roughly as well as ophthalmologists. Diabetic retinopathy is a common diabetic eye disease. This latest study expanded on Google AI’s work.
In this latest study, the research team wanted to determine what else the new computer-assisted system could do. Could it do more than, for example, simply diagnosing disease?
They wanted to develop a new computer-assisted system that could explain the AI’s (algorithm’s) diagnosis. The new system they developed improved not only the diagnostic accuracy of ophthalmologists, but also the algorithm’s accuracy.
According to a press release by the American Academy of Ophthalmology:
“More than 29 million Americans have diabetes, and are at risk for diabetic retinopathy, a potentially blinding eye disease. People typically don’t notice changes in their vision in the disease’s early stages.”
“But as it progresses, diabetic retinopathy usually causes vision loss that in many cases cannot be reversed. That’s why it’s so important that people with diabetes have yearly screenings.”
“Unfortunately, the accuracy of screenings can vary significantly. One study found a 49 percent error rate among internists, diabetologists, and medical residents.”
Avoiding pitfalls when AI ‘explains’ its predictions
Recent advances in artificial intelligence will probably help improve access to diabetic retinopathy screening. Accuracy will probably also improve. However, how AI will work in the offices of ophthalmologists or other clinical settings is less clear.
According to previous studies on computer-assisted diagnosis, some screeners over-relied on the machine. This led to repeating the errors that the machine made. It also led to under-relying on it and discarding accurate predictions.
Google AI researchers wrote that if the computer could ‘explain’ its predictions, some of these pitfalls could be avoided.
Helping ophthalmologists read algorithm’s predictions
To test their theory, the authors developed two types of assistance to help doctors read the algorithm’s predictions:
- Grades: the strength of evidence for the algorithm’s prediction was graded from one to five.
- Grades + Heatmap: they enhanced the grading system with a heatmap. It measured the contribution of each pixel in the image to the prediction of the algorithm.
The researchers asked ophthalmologists – one retina specialists in training, four retina specialists, four general ophthalmologists, and one trained outside the US – to read each image once under 1-of-3 conditions: grades only, grades + heatmap, or unassisted.
In a press release, the American Academy of Ophthalmology wrote:
“Both types of assistance improved physicians’ diagnostic accuracy. It also improved their confidence in the diagnosis. But the degree of improvement depended on the physician’s level of expertise.”
“Without assistance, general ophthalmologists are significantly less accurate than the algorithm, while retina specialists are not significantly more accurate than the algorithm. With assistance, general ophthalmologists match but do not exceed the model’s accuracy, while retina specialists start to exceed the model’s performance.”
AI can assist ophthalmologists diagnose more accurately
Lead researcher, Rory Sayres, PhD., said:
“What we found is that AI can do more than simply automate eye screening, it can assist physicians in more accurately diagnosing diabetic retinopathy. AI and physicians working together can be more accurate than either alone.”
AI is another tool, like technologies that preceded it, that will make the judgment, knowledge, and skill of physicians even more central to quality care, Dr. Sayres said.
Dr. Sayres added:
“There’s an analogy in driving. There are self-driving vehicles, and there are tools to help drivers, like Android Auto. The first is automation, the second is augmentation.”
“The findings of our study indicate that there may be space for augmentation in classifying medical images like retinal fundus images. When the combination of clinician and assistant outperforms either alone, this provides an argument for up-leveling clinicians with intelligent tools.”
Citation
“Using a Deep Learning Algorithm and Integrated Gradients Explanation to Assist Grading for Diabetic Retinopathy,” Rory Sayres, Ankur Taly, Ehsan Rahimy, Katy Blumer, David Coz, Naama Hammel, Jonathan Krause, Arunachalam Narayanaswamy, Zahra Rastegar, Derek Wu, Shawn Xu, Scott Barb, Anthony Joseph, Michael Shumski, Jesse Smith, Arjun B. Sood, Greg S. Corrado, Lily Peng, and Dale R. Webster. Ophthalmology. DOI: https://doi.org/10.1016/j.ophtha.2018.11.016.
Video – What is Artificial Intelligence?
This video explains what artificial intelligence is.