Vision system used to study development of memories for artificial intelligence
A new form of AI memory could allow robots and autonomous vehicles to imagine future events, think ahead, and react more quickly.
A team of computer scientists from the University of Maryland have employed a new form of computer memory that could lead to advances in autonomous robot and self-driving vehicle technology, and possibly in the development of artificial intelligence as a whole.
Humans learn naturally how to tie information about their position in the world with the information they gather from the world and learn to act based on that information. For instance, if a ball is thrown at a person enough times they learn to gauge where they are in relation to the ball and raise their hands to catch the ball. This process is called "active perception" and allows humans to anticipate future actions based on what they sense.
A human being's sensory and locomotive systems are unified, which means our memories of an event contain this information combined. In a machine like a robot or drone, on the other hand, cameras and locomotion are separate systems with separate data streams. If that data can be combined, a robot or drone can create its own "memories," and can learn more efficiently to mimic active perception.
In their study, titled "Learning Sensorimotor Control with Neuromorphic Sensors: Toward Hyperdimensional Active Perception," the researchers from the University of Maryland describe a method for combining perception and action - what a machine sees and what a machine does - into a single data record.
The researchers used a DAVIS 240b DVS (dynamic vision sensor) from iniLabs, that only responds to changes in a scene, similarly to how neurons in the human eye only fire when they sense a change in light, and a Qualcomm Flight Pro board, attached to a quadcopter drone.
Using a form of data representation called a hyperdimensional binary vector (HBV), information from the drone's camera and information on the drone's velocity were stored in the same data record. A convolutional neural network (CNN) was then tasked with remembering the action the drone needed to take, with only a visual record from the DVS as reference. The CNN was able to accomplish the task in all experiments with 100% accuracy by referencing the "memories" generated by combining the camera and velocity data.
The principle behind this experiment, allowing a machine vision system to reference event and reaction data more quickly than if the two data streams were separate, could allow robots or autonomous vehicles to predict future action taken when specific visual data is captured, i.e. to anticipate action based on sensory input. Or, to put it another way, to imagine future events and think ahead.