Turaga’s lab to improve the reliability of computers in identifying objects in images
The director of the School of Arts, Media and Engineering at Arizona State University has won a $1 million federal grant to improve the way computers discern images for the US Department of Defense.
Pavane Turagawho is a teacher at the same time School of Arts, Media and Engineering and the School of Electrical, Computer and Energy Engineeringreceived an 18-month AI-Explorations grant from the Defense Advanced Research Projects Agency earlier this month.
Turaga’s ASU laboratory, Geometric Media Labworked on the fusion of different methods, including mathematical, physical and data-driven, to better process and understand imagery data.
“This is a large defense agency that is traditionally known for investing in high-risk, high-reward projects,” said Turaga, who uses the pronouns them/them.
“This specific call was a response to a persistent problem in the world of machine learning and artificial intelligence.”
Computers use neural networks to process images. Networks use deep learning techniques to obtain data to identify objects in images.
“The last decade of work on neural networks has shown them to be very fragile and unreliable,” Turaga said.
“We don’t know why they do what they do. There’s a picture of a cat and they’ll say it’s a dog, and there’s usually no satisfactory explanation for that.
The images that neural networks process are affected by many factors, such as lighting and camera angle.
In the DARPA project, Turaga’s team will integrate insights from physics into neural networks to come up with more reliable and robust machine learning models.
They will use Riemannian geometry, originally invented by the early 20th century mathematical community to study the curvature of surfaces. Through careful reworking and merging with deep networks, we will be able to assess how an object’s shape and geometry can make things easier to identify.
ASU’s Geometric Media Lab is well positioned to be the bridge between the world of neural networks and the world of physics, said Turaga, whose doctoral dissertation focused on advancing a similar technique of use geometric methods to code physics models.
A neural network can be trained, by repetition, to recognize images. Gratings do this by focusing on regions of the image.
“Networks’ focus is often on color and shading – it’s very diffuse and not precise,” Turaga said. “It’s about looking at a lot of things and making a decision that may or may not be optimal.
“We propose, let’s look at more detailed geometry. What is the shape? And encode that definition in a way that the neural networks can train.”
The network will process pixels in the same way but with better reliability.
“When we say, ‘Tell me what you’re paying attention to while making this decision,’ that more clearly focuses on the characteristics of objects that make it what it is,” they said.
“We are able to get the neural network to pay attention to the relevant part of the object, such as shape and texture. If you don’t, you get a neural network that pays attention to a lot of distracting things. »
Turaga’s lab is able to show how their technique improves network processing through “attention map” images that look like heat maps.
“Our approach shows that we can train networks to be attentive by highlighting shapes more precisely.
“We’re trying to make neural networks more reliable by creating these visualizations that help us trust what they’re doing.”
System reliability is critical as DARPA is interested in defense applications, such as drone imagery.
Using standard benchmark data tests, Turaga’s method shows a four percentage point increase in accuracy, from 68% to 72%
“What’s interesting is not that we’ve pushed (accuracy), but to say that it’s increased, because we’re paying attention to the underlying property of the object.
“We are actually encoding the knowledge of the shape and geometry of the object.”
Turaga said they couldn’t have done this kind of work without a joint appointment between the schools.
“This role gives me the time and space to think about these kinds of speculative questions, which wouldn’t be easy if I was just in traditional engineering.
“It allows me to step back and say, ‘Let’s think more deeply about how media technologies encode knowledge.'”
The Geometric Media Lab works with the Global Security Initiative to ASU on the grant project, a portion of which is subcontracted to Johns Hopkins and Florida State universities. The project will last 18 months and will include teams of students and professors. The co-principal investigators are Associate Professor Visar Berisha and Assistant Professor Gautam Dasarathy, both in ASU’s School of Electrical, Computer and Energy Engineering, and also include student Rajhans Singh and postdoc Ankitha Shukla in as key contributors.
Top chart by Alejandro Cabrera/ASU News