It’s not unusual to see through the eyes of a robot or cyborg, at least in science fiction movies. The screen might show a pixelated world with a high-tech overlay, identifying people by body type, and motorcycles by make and model. If a person’s clothes and ride are suitable, the Terminator can even tell if the owner is likely to put up a fight.

That type of artificial intelligence may not be far in the future, albeit in a more defensive position and with less emphasis on Harley-Davidsons. With $13 million in funding from the Office of Naval Research, Saab has partnered with Purdue’s Institute for Control, Optimization and Networks (ICON) to develop AI models that can automatically spot threats in a complex, dynamic naval battlespace. Kraken, a student-built drone identical to the one at the top of this page, will model an aggressor aircraft for this threat identification system.

Those models will provide human-readable information to people who can decide whether to engage with a potential threat. The TSUNOMI program (Threat and Situational Understanding with Networked-Online Machine Intelligence) hopes to deal with an increase in intrusions into sensitive areas and serious physical attacks on critical infrastructure, both in military and civilian domains.

“There is an urgent need to create technological solutions that allow networks of sensors equipped with sophisticated AI to quickly detect and identify potential threats,” says Arvind Raman, the John A. Edwardson Dean of the College of Engineering.

Though these technologies are first intended for naval use, they will be adaptable: A Saab release says this system could be paired with radar systems at military and commercial airports, complementing the company’s growing portfolio in the U.S.

Shaoshuai Mou is the Elmer Bruhn Associate Professor of Aeronautics and Astronautics, effective July 1, 2023. This is one of multiple new term-named professorships established this year through a generous gift from an AAE alumnus.

From many sensors, one sense

Shaoshuai Mou, co-director of ICON and co-lead of the TSUNOMI project, says one of their challenges will be determining how to combine information from many sensors to identify what is in that space.

“We will be doing data fusion with information from different sensors, to determine what is there,” Mou says. “If we want to determine if something is a table, we might look for four corners, measure its height and compare it to what we know. It’s the same with a target: An object’s shape, flight speed, the noise it makes and other variables can help to determine if it’s a threat or not.”

Christopher Brinton, the Elmore Assistant Professor of Electrical and Computer Engineering, is co-principal investigator on TSUNOMI.

Developing these data fusion models can also provide guidelines on how many sensors, and of what type, might be needed to effectively survey a particular area. But identifying a drone is only part of the problem: Pinning down intent requires a nuanced look at how it behaves.

“Every autonomous vehicle has a mission, what we call an object function.” Mou says, “We can use inverse optimal control and optimal control methods to figure out the object function of its autonomous system. With that, we can predict its future trajectory. If it’s headed to a sensitive area or not, for example, can tell us a lot about what it intends to do.”

AI plays a key role

In this context, the machine-learning (ML) AI algorithms have to satisfy three key criteria, says Christopher Brinton, the Elmore Assistant Professor of Electrical and Computer Engineering and co-principal investigator on TSUNOMI. First, the AI must make accurate decisions in identifying and categorizing threats. The other two, Brinton says, are less obvious: It must be light on resources, and be able to share the context for its decisions.

“Resource utilization is a crucial objective, as many naval systems are pushing for reductions in size, weight and power of their hardware, to provide agility in contested environments. Unfortunately, accuracy is often inversely related to resource utilization, as it takes more computing and communication power to drive inference quality higher,” Brinton says.

Shreyas Sundaram, the Marie Gordon Professor in the Elmore Family School of Electrical and Computer Engineering, is co-director of ICON and co-lead of the TSUNOMI project.

“Contextual awareness is also critical, as ultimately, a human operator will have to evaluate the decisions that were made through AI/ML, and potentially provide refinements over a longer timescale.”

A ‘natural fit’ for Boilermakers

With the project’s expansive scope, it was a "natural fit" Saab would select ICON as its partner, says Shreyas Sundaram, co-director of ICON and co-lead of the TSUNOMI project. ICON has in-house expertise in many of the areas TSUNOMI touches: Its 84 faculty members span the College of Engineering, College of Science, College of Agriculture and Purdue Polytechnic Institute.

“ICON’s rapid growth, expertise and significant level of activity make it a top-tier center for autonomous systems research and education. The breadth and depth of expertise within ICON also allows us to quickly pull together key experts to form effective teams to tackle new projects,” said Sundaram, who is also the Marie Gordon Associate Professor in the Elmore Family School of Electrical and Computer Engineering.

Researchers will use the motion-capture equipment and other tools at the Purdue UAS Research and Test (PURT) Facility as its main research space. Close proximity to Saab’s growing presence in West Lafayette, including its advanced manufacturing facility and engineering resources, is another perk of the partnership.

Purdue is expected to receive $4.3 million of the total grant as a subcontractor through Saab. That funding will primarily be used to support graduate students, some faculty time and to purchase sensors. Erik Smith, president and CEO of Saab, said it’s a big win for everyone involved. “This program represents a real step toward robust and trusted artificial intelligence. TSUNOMI will help stakeholders make decisions quicker and with more confidence.”

The Kraken Rises

The Guapo UAV, pictured above with the Purdue Aerial Robotics Team (PART), was built for the international sUAS Competition in 2023. PART placed 13th overall and 3rd among U.S. schools. The Kraken, which is being used for data collection on the TSUNOMI project, was built using the same composite molds and looks identical to Guapo.

However, everything from composite materials to the internal structure is different, allowing Kraken a lower MTOW and a better flight envelope. PART system leads Aidan Bilger and Mateo Llerena headed the Kraken build. “We will be acting in an aggressor role initially, replicating adversary UAS threats,” says Bilger. “This aircraft is excellent as an aggressor because of its flexibility. Our ability to operate autonomously allows us to replicate our flights, which is critical to another important factor: consistency.”

The PART team included more than 30 students across engineering disciplines. The subsystem leads on the team were: Matthieu Opdyke, Eric O'Keefe, Corey Auerbach, Aidan Bilger, Mateo Llerena, Diego Montes, Evan Kamm, Cameron Johnson, Kevin Cai and Leonard Jung. Charles D’Onofrio was the team's graduate student mentor.