featured-image-standard-abstract-lines-dots-nodes-connecting

Detecting blind spots in AI – when they learn the wrong things

Published: 17:31, January 25, 2019

Researchers have developed a new model that identifies AI blind spots. Sometimes, what autonomous systems learn from training examples do not match events that are actually occurring in the real world. In other words, AI systems sometimes miss things and make mistakes. Researchers have found a way of detecting those mistakes.

The researchers, from MIT and Microsoft, say that we could use this model to improve safety in AI systems. The model could help, for example, autonomous robots and driverless vehicles function better.

Driverless vehicle AI systems have blind spots

Autonomous vehicles have AI systems that engineers have trained in virtual simulations. The simulations prepare the AI for almost every event that might occur on the road.

However, the systems have blind spots, i.e., they make unexpected errors in the real world. In other words, some events that should alter cars’ behaviors do not alter them.

Let’s imagine, for example, that a driverless car has been trained in a limited simulator. It does not have the sensors that differentiate between different scenarios, such as ambulances with flashing lights on the road and white cars.

If the vehicle is cruising down the road and an ambulance turns on its sirens, the car might not know that it has to slow down and pull over. It doesn’t know because it just sees a big white car.

Detecting training blind spots with a human monitor

In two papers, the researchers described a model that uncovers these training ‘blind spots’ using human input.

As occurs with traditional approaches, the AI system underwent simulation training. However, a human then closely monitored the system’s actions in the real world. The human provided feedback when the system made a mistake or missed something. They also provided feedback when the system was about to make a mistake.

Researchers subsequently combined the human feedback data with the training data. They used machine-learning techniques to produce a model that pinpointed potential ‘blind spots.’ In other words, situations where the AI system would most likely need information about how to act appropriately or correctly.

The researchers used video games to validate their method. During the games, they simulated a human correcting the learned path of an on-screen character.

Their next step will be to incorporate their model with traditional training and testing approaches with human feedback. Specifically, for AI systems in robots and driverless vehicles.

Helping systems better know what they don’t know

First author, Ramya Ramakrishnan, a graduate student in the Computer Science and Artificial Intelligence Laboratory at MIT, said:

“The model helps autonomous systems better know what they don’t know. Many times, when these systems are deployed, their trained simulations don’t match the real-world setting [and] they could make mistakes, such as getting into accidents.”

“The idea is to use humans to bridge that gap between simulation and the real world, in a safe way, so we can reduce some of those errors.”

AI stands for artificial intelligence. It consists of software technology that makes devices such as computers and robots think like us. The technology also makes them behave like us. AI has the ability to learn along the way, just like we do. We call this machine learning.


Christian Nordqvist Avatar

Other News

Shopping games can frustrate customers when rewards and rules are hard to understand

Nov 22, 2026

Text-message business training helped Kenyan entrepreneurs briefly, but gains faded

Nov 22, 2026

Food traceability labels raised stated interest, but ad test found no clear difference

Nov 22, 2026

Letting shoppers keep an item for a partial refund lifted later spending in a field experiment

Nov 22, 2026

Skincare claims that look scientific changed shopping choices in an online experiment

Nov 21, 2026

Crowded settings can make unfamiliar products less appealing in consumer experiments

Nov 21, 2026

Words that describe texture can affect online clothing choices, study finds

Nov 21, 2026

Impact measurement support helped Indian social enterprises grow in a field experiment

Nov 21, 2026

Lyft experiment found a premium ride option shifted some customers to cheaper fares

Nov 21, 2026

Online retailer improved pricing by testing how shoppers respond to rivals

Nov 21, 2026

Environmental reminders reduced return shipments in a fashion retailer experiment

Nov 21, 2026

When a discount is hard to claim, it can reduce sales of the product

Nov 20, 2026

A small step to claim a discount increased redemptions in a coffee campaign

Nov 20, 2026

Ad platforms can target likely buyers without creating the most extra sales

Nov 20, 2026

Sales contests worked better when employees competed in comparable groups

Nov 20, 2026

Social discounts encouraged existing grocery customers to buy more in an experiment

Nov 20, 2026

Free shipping and returns brought more shoppers back in a retail experiment

Nov 20, 2026

Learning about rivals’ robot use changed Italian firms’ investment plans

Nov 20, 2026

Automated lead nurturing does not improve every sales pipeline

Nov 20, 2026

Seeing competitors’ prices changed how nail salons set their own

Nov 20, 2026