How AI and Sensor Fusion Are Transforming Autonomous Targeting
Key Highlights
- Defense and industrial applications share process-flow challenges, from gathering useful data to making decisions based on it.
- Advances in mobile computing and dual-use technology have reduced the cost and expanded the capabilities of defense vision systems.
- Testing across varied conditions remains essential because sensor performance can falter with low contrast, occlusion, weather, or changing illumination.
Rapid technological advances are happening across all sectors, including defense. And as in so many other areas, applications that just a few years ago seemed out of reach are now becoming the norm.
One important area of the defense industry, autonomous targeting systems, has evolved dramatically over the past two decades, driven by advances in sensors, computing power, machine vision, and AI. What once required specialized hardware, limited datasets, and highly constrained operating environments can now leverage sensor fusion, real-time image analysis, and aided target recognition in the field.
Vision Systems Design wanted to learn more, so we reached out to Bryan Bockmon, CEO of AimLock (Littleton, CO, USA), a company that specializes in autonomous targeting and engagement systems, about the evolution of autonomous targeting technologies.
Editor's note: The following Q&A may have been edited for style and/or clarity.
Vision Systems Design (VSD): When you first began working on autonomous weapon systems, what were the biggest limitations in imaging/vision and target recognition and what are some of the technological advances that have been most instrumental in getting us to where we are today?
Bryan Bockmon (BB): Incredibly, this was about the same time as the revolution in mobile phone technology. During that time, we were transitioning from extremely limited mobile power, battery technology, and small processor technology to something that's completely transformed the world now where everyone can have smartphones that last all day long and can process anything we need to do throughout the day. So defense technologies tracked along that same process. We used to have highly specialized systems that cost us astronomical amounts of money to engineer and test. And now by borrowing some of that dual use technology, we get dramatic cost savings.
VSD: Modern systems rely on sensor fusion from visible, thermal, and other sensing modalities. How has the fusion of these data streams changed target detection, tracking, and classification performance compared to earlier generations of systems?
BB: It's funny. Every person blends their senses together, every single day, to sense their own environment. You got your vision, your hearing, your touch, your smell. And we don't often think about how much we take advantage of all those senses. But if you remove your access to any one of them, or especially two of them, you quickly realize how limited your perception capabilities become. So, really, the mass adoption of machine learning has been the transforming technology that allows any of these machines that we build to be able to pull that information together and make sense of it as one coherent thought process.
VSD: Many machine vision engineers struggle with edge cases and environmental variability. What lessons did autonomous targeting developers learn about handling cluttered scenes, occlusions, adverse weather, and changing illumination conditions that can be applied to other applications, such as industrial inspection?
BB: If you back up 20 years, the algorithm developers weren't intimately familiar with every little mechanism that led to target recognition. They usually wrote those algorithms specifically for the most common cases. Then, through this continual testing process, they became familiar with where the failure points were going to happen. Now, with machine learning, there's a lot of stuff that's opaque. Like we have no idea sometimes how some of those algorithms are blending together to get the detections. So, occasionally, you'll get an edge case that's a complete surprise. And that always results in significant testing, rapid pivots, and blending other technologies together to get the job done in time.
VSD: Moving from a successful demonstration to a fielded system is often where vision projects fail. What were some of the major challenges in validating image analysis and target recognition performance under real-world operating conditions?
BB: Let's say we're in a conventional military environment. We're working with thermal imaging cameras—that's really common. They work well in a lot of circumstances, especially in high contrast thermal conditions at night where you have the emission of all of the heat signatures and then you've got really cold backgrounds. We've got great contrast, which leads to excellent performance.
But then there are those transitional moments during the day where all of the backgrounds are the same. Temperature is everything that you're hoping to image, and you just don't have any contrast at all. And for engineers who work in the sensing world that's a great example to learn from because no matter how much money you put into one sensor modality, there may always be a circumstance where it's weak. And that's where it makes sense to add other layers.
VSD: AimLock emphasizes aided target recognition rather than simply automating the entire decision chain. How does that philosophy influence the design, testing, and performance requirements of the underlying computer vision system?
BB: We don't get to operate purely in a technical space most days. Most days we have to cross over into the political world and into the financial world. And so as policy is developed throughout the years, especially here in the U.S., our policy makers and our military decision makers have adopted terminology that allows them to distinguish between things that may be prohibited and things that are specifically desired.
In our case, autonomous weapons treaties are politically charged topics. And so whenever we want to automate a portion of the system without diving into the complexities of the politics of autonomous weapons, we'll talk about, well, we're going to aid the target recognition process. And that became a political term that was adopted by the military so that the engineers could continue on with the things that they had to do, while letting the policy makers sort out the other stuff.
There are plenty of systems that are autonomous weapons. We usually use them in defensive situations. They have automated target detection, automated target recognition, and automated fire control of the weapons.
VSD: Looking back over your career, which perception technology capabilities that once seemed impossible or impractical are now considered standard, and what breakthroughs do you think will define the next decade of autonomous perception systems?
BB: There are two things that seem to be dominating this pace of change that we're under. One is the rapid ability for AI companies to collect data that can be used to train, any of the neural nets that we use for vision or for radar or for any other perception technologies. That data collection used to cost millions of dollars for any company that wanted to do just a simple little problem. And now it has become the entire game of many of these information systems companies. Second, now that we have the ability to rapidly configure robotic systems, we're getting more general-purpose robotics and fewer specialized robotics. Now, there will always be a place for specialized robotics because they're good at one job. They are extremely cost-effective and can be tuned perfectly for that job. But now we have robotic systems that can do seven or eight or twenty jobs. And being able to train them affordably to take care of those jobs just completely transforms our industrial space for military equipment.
VSD: Many industrial machine vision applications and autonomous targeting systems share similar foundational challenges. What crossovers have you seen between defense applications and commercial robotics or automated inspection systems?
BB: It's really a process flow problem in both cases. In industrial inspection, you want to know what your target parameters are. What target objects do you need to inspect? Where are they? How long are they going to be there? Do you need to manipulate the lighting conditions to make sure you can gather the data you want to gather?
And then after you've done all of that information gathering, you've got a decision-making matrix. Do I let this assembly line continue on? Do I reject certain pieces and move them into a side bin for further processing? The same thing happens on the military side of the house. We've got a small number of weapons effects—guns, rockets, missiles, microwave— whatever it may be. And our autonomous systems have to decide which of these effects need to be put into play to handle this situation that's come down the conveyor belt. It's great when you can use engineering and math language to get the job done.
About the Author
Jim TatumJim Tatum
Senior Editor
VSD Senior Editor Jim Tatum has more than 25 years experience in print and digital journalism, covering business/industry/economic development issues, regional and local government/regulatory issues, and more. In 2019, he transitioned from newspapers to business media full time, joining VSD in 2023.
Related
Industrial AI
