MIT builds robot system that lets machines erase and rewrite their own bad memoriesMIT researchers have developed a new memory system that helps robots identify and correct inaccurate descriptions of objects stored in their memory. Called UQ-DAAAM, the system evaluates multiple descriptions of the same object from different viewpoints to determine how confident a robot should be in what it has identified. It then selects additional viewpoints to reduce uncertainty and combines the observations into a more accurate description. The corrected information is stored in the robot’s memory, potentially improving its ability to navigate environments and perform complex tasks.Robots fix memoriesMIT researchers have developed a new artificial intelligence system that allows robots to identify, evaluate, and correct unreliable descriptions of objects stored in their memory. The technology, called UQ-DAAAM (Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment), combines computer vision, language models, and uncertainty quantification to improve how robots recognize and remember objects in their surroundings.The system builds on an earlier robot memory framework called DAAAM, which uses a vision-language model (VLM) to generate detailed descriptions of objects captured by a robot’s onboard cameras. These descriptions are stored in a three-dimensional map-based representation of the environment, allowing the robot to retrieve information about objects when answering questions or performing tasks.However, the original system has limitations when objects are partially visible, distant, or captured in blurry or poorly lit images. Under these conditions, the vision-language model can generate incorrect descriptions. For example, it might mistake a gray chimney for a squirrel on a rooftop. Without a mechanism to correct these errors, the inaccurate information could remain in the robot’s memory and affect subsequent decisions.UQ-DAAAM addresses this problem by introducing a method to measure uncertainty in the descriptions generated by the vision-language model. As a robot moves through an environment, its cameras continuously capture images, often recording the same objects from multiple viewpoints. The new system uses these additional observations to determine whether an existing description is reliable.According to researchers, it first evaluates the model’s confidence in the words it generates when describing a potentially misidentified object. It then measures consistency by comparing that description with captions generated from images of the same object captured from different angles.The system combines these two measurements into a single uncertainty score representing how confident it is in its understanding of an object. If the score exceeds a predefined threshold, UQ-DAAAM identifies the description as potentially unreliable and initiates a refinement process.Improves object recognitionInstead of analyzing every captured image, the system uses a rule-based method to select additional viewpoints that could help reduce uncertainty. For example, if a robot mistakes a chimney for another object, it may select an image showing smoke rising from it to identify the structure more accurately.The researchers mathematically demonstrated that this selection method can identify useful images without repeatedly running the vision-language model. This reduces computational requirements, allowing the system to operate on a moving robot with limited processing resources.Once the additional images have been selected, a large language model (LLM) combines their descriptions into a revised caption. The system gives greater weight to newer descriptions and those containing less ambiguity to produce a more reliable representation of the object. The corrected description is then stored in the robot’s memory for future reference.In testing, the refinement process resolved ambiguities for approximately 92 percent of the objects in the robot’s environment, requiring only two additional captions per object. It also improved object-description accuracy by 5 percent compared with existing state-of-the-art methods.The researchers further found that UQ-DAAAM improved the accuracy of answers to questions about objects, their locations, and when the robot last encountered them.The team now plans to extend the technology to different robotic platforms, including mobile robotic manipulators. They are also exploring whether the system can identify areas of high uncertainty and direct robots to revisit those locations to collect better observations.