In this episode of Visions: A Machine Vision and Automation Solutions Podcast, VSD Head of Content Sharon Spielman talks with Arthur Erickson, CEO and co-founder of Hylio, about modern drone vision systems. They discuss  advances in camera cost and performance, edge computing, sensor fusion (radar, stereo vision, LiDAR, thermal), payload and gimbal design, vibration damping, and real-time AI processing to enable applications across industry sectors as well as Hylio’s full-stack approach to hardware and software integration for robust, field-ready drone solutions.

Visions: A Machine Vision and Automation Solutions Podcast, is the podcast for engineers, designers, integrators, and end users who want to keep an informed eye on the imaging and machine vision industry. Every Tuesday we will explore the latest in imaging trends, developments and solutions. Here you will find interesting, useful insights and observations from expert interviews, solo episodes, even the occasional panel discussion, all of which aim to expand your knowledge on imaging and machine vision.

Related: Smart Surveilance: The Impact of AI-Enabled Drones on Public Safety and Critical Infrastructure

Transcript

Well, hello and welcome to Visions, a Machine Vision and Automation Solutions Podcast. I'm your host, Jim Tatum, senior editor of Vision Systems Design and Visions is an Endeavor Business Media production from your friends at Vision Systems Design. Here you'll find the latest on everything from end user machine vision solutions to trends, developments, and perspectives on all things machine vision and imaging. Whether you've been working in the industry for a while or you're just starting to take a closer look at it, this podcast is designed to grow your knowledge and bring greater focus to your understanding of the imaging and machine vision industry. And now on to our show. 

Well, hi everybody, and welcome to Visions. Today we're going to take a deeper dive into how advances in cameras, sensor fusion, edge computing, and AI are making drone vision systems more capable across a wide range of applications. In fact, recently, VSD's head of content, Sharon Spielman, had a terrific conversation about some of these very issues with Arthur Erickson, who's CEO and co-founder of Hylio, a Texas-based company that specializes in drone technologies and applications. Let's give a listen to what they had to say.

Welcome. Today, we're looking at drone vision systems, an area where advances in machine vision, sensor integration and real time image processing are changing how drones serve the critical roles in a range of industries and applications, including emergency response and security. For our Vision Systems Design audience, the challenges go beyond just adding cameras. Success hinges on optimizing sensor performance, flight stability, and manufacturing precision to deliver real time, actionable visual data in challenging environments. As automation and AI driven machine vision advance, these systems are becoming more robust, adaptable, and capable of learning on the fly. Joining us today is Arthur Erickson, CEO and co-founder at Hylio, to share his insights on the technical evolution of drone vision from sensor technology and payload design to AI driven processing, and what it means for advancing situational awareness in the field. Thank you for joining us today.

Thank you for having me.

You bet. First, why don't you tell me a little bit about yourself, your background in the drone market?

Sure. Um, well, it's nothing too exciting. I won't harp on too much about myself. I, uh, went and studied aerospace engineering at UT Austin. So University of Texas in Austin, and that's actually where I started the company with three other, uh, fellow UT alum. So it's a longhorn company, classic longhorn startup dorm room type of story. But yeah, it was through my curriculum as an aerospace engineer undergrad, only that I first got into drones. And basically for our capstone project, for example, that entire year, it was design a drone from scratch. In this case, it was a fixed wing platform. So plane style design. We were working with the Austin Fire Department to do a simulated mission in which we locate a target and then drop an aid package, right, simulated aid package on that target. And that was how we were graded. So it was kind of a combination of those technical skills they picked up in school, and then just entrepreneurial spirit. And of course, my great team put that all together. And then we started Hylio. It'll be twelve years in January. So all the way back in twenty fifteen again while I was still in school. So that's where we come from. Since then, primarily we have focused on the agriculture industry. So meaning that we design and build and sell drones that are more or less replacing traditional crop dusting methods. Uh, that's airplanes and helicopters. But I should also mention that's applications done by tractor or self-propelled ground rig as well. So we're offering a more precise, environmentally friendly, better for your wallet, better for the neighbors wallet solution. So that's typically what we do. But to swing back to this conversation a bit, as a company has grown in size, we've become more successful, had more engineers on staff, had more mature products that we could bring to bear. We've branched out to other industries, such as emergency response. So a bit what we're talking about here with wildfire detection and suppression and other things along that vein as well. So great about us.

Yeah. So twelve years. So I imagine that you've seen quite a bit of change. So from a machine vision integration perspective, what key advancements have you seen in drone vision systems recently?

Yeah, I think probably one of the main, it's not maybe super exciting, but one of the main things is just unit cost, right? So bringing the unit cost of relatively high resolution cameras down and making them lightweight as the drone industry explodes, making them cheaper. It's just obviously going to be a forcing function for more use out in the field. So I think personally a lot of that has come from the EV market, right? So basically all these cars now obviously auto is a massive industry as they've integrated more digital vision systems. We've, as a drone industry reap those benefits. Camera technology as well, right? So every cell phone has a camera. So yeah, I think just price point is a big one there. So just packing more punch in a smaller and cheaper package. And then on the AI side or on the compute side, edge, compute as a concept and instantiated in more efficient, again, cheaper but more efficient edge processing power is a key, of course, to actually having these practically come into play in a scenario that's very time sensitive, such as emergency response, right? So you can't be beaming stuff back to the cloud, waiting twenty four hours to have some server do it, and then acting on the information. It's going to be done right there in the field. Right? So let's talk about the types of cameras and sensors that are incorporated to optimize the image capture across those harsh operational environments that you're talking about. Yeah. So it really depends. Um, if we're talking about something like identifying a fire in a wildlife or wildfire scenario, you don't actually need that much vision to you don't need that much resolution, I should say. It doesn't have to be like a high fidelity feed that you're reading because a fire is pretty obvious, right? So especially if you have a thermal sensitive sensor, a fire is pretty dang different than its surroundings. So in a case like that, it's fairly black and white. You don't need a super high powered thermal sensor, or you probably don't even need thermal. In general, you could get away with RGB. So that's pretty easy. But in a scenario, like if you go to our traditional industry, which is again, precision agriculture, you again, depending on your context, might need some pretty high resolution camera work to be able to identify a somewhat different looking green thing versus another green thing, right? So the point you want to say versus the point you want to kill like a weed. So typically, I mean, to ground it, most of our customers, like on our sensing platform we have, which we call the photon. A lot of the customers are going with a 4K twenty megapixel RGB sensor with optical zoom up to five to seven x, and this is typically good enough for them to be pretty speedy with some crop scans where you can fly at three to four hundred feet above the crop and identify literal, you know, individual weeds or individual leaves. Even on plants, you could see stuff like fungal outbreaks, um, little white spots or black spots, for example, on the leaves of corn or something to, to that nature. So it really comes down to, again, your use case, like what you need your ground sampling distance to be, but people are operating anywhere from half a centimeter per pixel to two or three centimeters per pixel based, again on ultimate use case, right? So how, how do the drone vision systems, um, deal with the varying lighting conditions and the weather interference and the motion blurring to maintain the image fidelity for the downstream processing. How do you select how do you specify that for your drones? Again, this is like designing your system. Not to go on a tangent here, but like our photon, which I just mentioned, is designed intentionally to be very easy for you to swap payloads out, right? So you can get really any third party sensor like a Sony IOX, you know, it's basically like their DSLR, but shrunk a little bit. And then you can stick it on like a drone you could use like multispectral or hyperspectral cameras, thermals, you could put leaders on there, which I know is not a camera, but or a vision system. But I mean, you could put anything on there, right? Because it's going to change. So like our typical camera that we would sell to an agricultural customer would be somewhat of a one size fits all. Right. So like decent at least like HD, so 1080, you know, video streaming capabilities. But if they're not using it for precise wheat ID just like a navigational tool, that's all they need. Um, HDR right? So it can actually perform really well in low light settings. But it's not true night vision, right. So we do have some customers that actually strictly operate at night or need to operate at night for some application, whether that be AG or something else. And in which case, you know, they would, we'd have a straight up thermal based system or infrared near infrared perhaps, and we'd have an illuminator coupled with the sensor. So you're illuminating it and then receiving that data back. So not sorry to give you a bad answer, but just like it really, we do all sorts of stuff based on the based on the customer, so we can integrate by design. It's a very extensible system. So if they need night vision, we can do that. If they need thermal, we can do that. They need high resolution RGB. We could do that.

Okay. So in terms of the processing architecture, what onboard compute AI capabilities enable real time interpretation of visual data is that that, that photon that you're talking about, uh, for obstacle detection and terrain mapping And then that also kind of makes me think about, um, I know you said you cover ag, you cover emergency services and I believe surveillance. Do you cover, um, aerospace and defense? I know you were starting out with agriculture, but are you branching out into all of the industries or can we talk about that a little bit?

Yeah. So we are, um, basically doing all of the above. Uh, we view our drones as a platform technology. And so not to belabor it, but, but what I was just getting at, we make it very easy for any third party provider or company to put sensors or modules onto it. So AG is one thing. But yeah, when you talk to the and I keep bringing up the fire people, but like, it's just something that's top of mind because we're doing that right now. But fire suppression people or the police department, right? They're going to have, um, like in the case of fire, you might need to drop a flaming tennis ball, essentially. And this is to do a controlled burn, or you might need to spray out a special compound that like, literally, like directly suppresses the fire. The police might have even something like a loudspeaker on the drone, which is just a different thing to do. Crowd control, right? To speak, to crowd, disperse, disperse, perhaps like a tear gas canister. Right. That's something we haven't done yet, but we're in conversations about. So things of that nature. I mean, there's all sorts of different things you can stick on these things. Bringing it back though. It's just yeah, can you build a base that's like eighty percent there? And just like the French fries that go with the hamburger are like relatively easy. Um, and maybe like, you know, a few weeks worth of integration work for our team and then the customer's team to actually implement. So, um, that's kind of what we see generally for a lot of these, uh, enterprise level customer interactions right now is like a little bit mostly like the stock unit we sell. And then like a little bit of customization on top.

Okay. So why don't you talk a little bit about that? So the, the payload design, the weight distribution and the vibration damping, um, how do all those impact the optical sensor performance and flight stability in the drones themselves. I mean, because, you know, a speaker as opposed to having to carry fire suppression are very different.

Yeah, no, that's a good point. Uh, and yeah, I should elaborate more on your last question about kind of how we do some of the processing, but it's a very fair point that vibrations are the enemy of clear visual data. You have to solve that first mechanically, of course. So we've designed a brushless motor actuated three axis gimbals. And that's typically what our cameras are mounted on our drone platform. So that gives you control of where the camera's pointing while you're flying. Uh, but it is actively, uh, stabilizing the camera feed, uh, the entire time that to get a little bit, even nitty gritty about it, like you've got this gimbal, but it itself is mounted on vibration dampening structures, right? A rubber type structures that are placed in such a way through analysis we've done and testing to really minimize that vibration even further. Even beyond that, you still have vibration. So then you apply like a digital stabilization layer, which there's a fair amount of open source tools out there that we play around with and stuff that we've developed in-house as well. So that's like edge detection. And you know, I won't get into it, but the basic things that camera companies are applying to stabilize moving frames, we're not doing anything like we're not a camera company at the end of the day, right? So we're not doing anything, a bleeding edge or anything in that regard. Um, but really just trying to, uh, cross our T's and dot our eyes and make sure that you mechanically eliminate the vibrations as much as possible, do as little digital stabilization as you can, because ultimately, obviously that's going to lead to artifacts and stuff like that that you don't want if you do too much digital correcting. So we try to do it the right way mechanically. What we employ object recognition and whatnot is a variety of things. Uh, again, a lot of it is leveraging open source stuff out there like YOLO, um, machine learning detection libraries. So it's like, uh, basic stuff like, can you identify a human versus a cow versus a car? There's a lot of stuff out there that's available to everybody already for that. But really, it comes down to using the right tools or algorithms for the right problem and designing the software, the interface in such a way where the customer or the operator could, could switch between the stuff when needed. Right? So yeah, you might use a different model when you're doing detection of a car or a person, you know, maybe in a law enforcement sense versus the model you would use, of course, for identifying what we call like volunteer cotton in a cornfield or vice versa, or johnsongrass in a cornfield. So it's more of like an interface thing. These tools all exist under the hood, of course, but can you seamlessly switch between them and give, like the customer that power without overcomplicating it to use the right model at the right time?

So how, how do you go about, um, when you're sitting down with your customers? Are you selecting off the shelf? Are you using customized components? How does that work? And I'm speaking specifically for the vision systems on the drone. Um, I, I don't know how, uh, I guess I'm kind of looking when you're doing your drones, when you're making them, I would imagine you have your own components that you're putting together for the drone itself, but then you're going to a third party for the vision systems. Is that correct?

That's usually right, at least for now. Uh, we are actually developing like from the board level up like our own camera. But even then, I mean, of course you're still buying the sensor from Sony or whomever. So like usually, like right now there's a company we utilize called Econ systems. They're actually based in India, but they make great products. A lot of them are designed for drones to our point earlier, they're cheap, they're lightweight, good stabilization, etc.. So we use their cameras, but we build a gimbal around it. And then we have an on board compute. Typically we're using like Jetson or Nanos right now. So little computer. Nvidia powered of course. And this is doing a lot of this post-processing we're talking about in real time. So segmentation or just a obstacle detection or object detection target detection. And then spitting out whatever is needed into the flight controls, which are then going to turn that into like an actionable maneuver.

And you're providing the hardware and the software to your customers.

Yes. That's correct. So we are full stack in that sense, which not a lot of people do what we do, I should say, and there's a lot of companies out there that outsource the software or vice versa, outsource the hardware drones. It's an orchestra, right? It's all these things working in perfect harmony. And we've just found, because we've done a mix of both, right? When we were a younger company just for kids out of UT Austin, there was a lot of system integrating. We had to do where we were. We were going out and buying hardware that we didn't design, trying to make it work. But ultimately, there's a lot of limitations there, especially at the edges. But eventually for the entire system, like you're looking at some pretty serious inefficiencies or sometimes downright inability to do a desired outcome because you ultimately didn't control that supply chain, that you know, the entire design of the system. So again, with the orchestra that is a drone, we think it's imperative that you do both the hardware and the software for ultimate outcomes.

So speaking of the orchestra, then the listener of the orchestra, which would be the people who buy the drones. Are there any vetting processes? Okay. So can you talk a little bit about sensor fusion and how it is evolving in these systems to improve the situational intelligence accuracy and the robustness?

Yeah, that's pretty important when the drone is flying. I mean, this is especially important for obstacle detection and avoidance. These drones, like in the agriculture context, are often flying quite fast forty miles per hour. Um, they're flying low. So yes, they're flying over depopulated cornfields. So it's not really a risk to humans, but the drone is at risk as it's flying low, fast, and there's trees and there's guide wires and whatnot. So you need a pretty robust vision system. So it's got to be able to actually see everything. And it's quite a noisy environment in terms of obstacles that these systems are detecting, right. Because you have all this vegetation, again, you have all these trees, power lines, whatever you need to identify everything, even as small as a half inch power line. Um, you need to identify everything, but then you need the computer on board to say, is that actually something I need to worry about? Right. And throw away the stuff? Uh, that is just noise, like actual noise. Also ignore the stuff that is a real obstacle but isn't in your path, for example. So you're doing all this like calculation on board in real time. What's the heading of the drone? What's the speed? Where are these obstacles in, you know, proportion to my heading or in relation to my heading, I should say. And so it takes a lot and like to bring it back to your point about sensor fusion, like there's no single sensor that does everything perfectly. Right. And so, for example, I'll crown it again. We use radars a lot for obstacle detection because they're a pretty good overall fit because in the drone world you're spraying products. So there's a lot of moisture in the air like mist and fog. And there's sunlight reflecting off this moisture. There's dust that you're kicking up. So all these things, radar can, can relatively easily penetrate through and give you still a pretty good picture of what's on the other side, but versus like a camera, which might be obscured by dust or liquid, or a lidar system, which would just bounce off the first thing it sees. Um, so you have to play this compromise game with what's the best fit for most of the situations. Or what you can do is what we're starting to do now, which is like, it's going to increase the cost a bit, but add more, more different sensors and fuse them together. So then you are actually truly covering all the bases, right? So I think the big push for us right now is, okay, we have radars, which I just mentioned. We're trying to incorporate binocular vision. Um, not as you know, replacement to the radars in conjunction, again along the theme of using it, because what the binocular vision can do better than a radar is make, uh, let's say more practical planned decisions or help you make more practical plan decisions with how to go around an obstacle. So proactively avoiding like a tree line or something versus a radar, which just sees it, and then you're reacting to it and kind of losing a lot of energy there. Right? And, you know, seeing the tree line stopping and then doing a bit of a Roomba approach and getting around it right versus where you could see it like a human would with binocular vision, say, okay, I can kind of see the way around it and then predict the next thing that way. Yeah. So yeah, that's where it gives you more, more benefit. But um, yeah, sorry, that was a kind of a meandering answer.

No, that was good. Maybe you could talk about some of the system integration challenges that you face when combining the cutting edge vision hardware with the drone, flight control and communication modules.

Yeah, I think this is like a mostly a compute challenge, as in you only have so much compute. So a lot of these sensors are more than enough in terms of data. Like, you know, you could be pulling up gigabytes basically off these things per flight, if not more. Um, so yeah, you're getting plenty of information. The challenge is for the engineering team is yeah, to what we were discussing earlier. What do you throw away? Like, what do you use that limited compute for that drone. Every gram counts, right? So you can't just have a giant server rack of processing power on this thing. You have to have, um, you know, tiny little jets in, which is good, but it's, it's small. So just deciding, doing that triage, deciding like, what's the right information to actually store and compute is the challenge there. Yeah. It's like a budgetary problem almost.

So do you have, does the software that you provide the customers, does it have, say, selection boxes of, of what is needed for each particular application so that they cannot be wasting, um, data collection because it's not going to do anything with that data anyway. So how does that selection happen?

You know, we don't really have a lot of that yet, but that is in the works. Uh, you would have like a drop down menu in our software, um, to your point, and it would be, okay, here's the model that's best for identifying humans. Here's the one for use for plants, etcetera, etcetera. Um, of course, we'd want to also expose camera settings to people. So like, you know, what is your data rate? You know, are you limiting the resolution or whatever? If it's a software defined camera system. So just trying to give the customer more customization without overwhelming them, right? Because most of these operators aren't like these camera vision nerds, right? So they need to be explained to things in, you know, outcome type of type of language like C, human, C, plan C, whatever. That's what I was just going to say. I, that's honestly what I was just going to say. If you ask somebody what data rate do you need? Do you also tell them if you're looking for a plant, you'll need this. If you're looking for a human, you'll need this. If you're looking for a fire, you'll need this. Exactly. I mean, we're going to hide that behind like these buttons that are that are doing that crunching on the back end. I mean, there's going to be like a pro selection menu where it's like, okay, like you can tweak anything you want. We don't guarantee functionality. Right. So that's for the power users. But generally for most users, it's going to be again driven by function. And so we'll do all the setting tweaking under the hood. Gotcha. Okay. So looking ahead, how do you see advances in AI image processing algorithms and new sensor modalities influencing the next generation of drone vision systems for OEMs and integrators and machine vision industries? I'm just thinking of your twelve years that you have in the in this industry, maybe looking at crystal ball for the next twelve. But, um, yeah, so I guess I'll leave with a bit of a hot take. Like I think, uh. Uh, AI is, of course, being overused. I think that's obvious to everybody for a lot of things that it doesn't need to be used for. That's not going to change in the drone world either. So you've already seen this over the years, but even more so now you're going to see companies trying to apply AI and relatively expensive in terms of compute models and whatnot to do very mundane tasks that could be solved with simple linear regression data analysis principles, right. So not AI at all, but just basically input output, you know, formulaic stuff. So I think you're going to see a lot of overuse of AI, which is going to be really computationally expensive and inefficient. And therefore the ultimate product that these various companies hypothetically are selling, it's not going to work that well because again, it's a budget, it's a triage problem. If you're not, if you're blowing all of your compute on like a relatively dumb use of AI, you're not going to have anything left over and you're not going to be a very effective product. So you're going to see this big boom of these companies promising the world with AI models on their drones, and they're not going to do that much. Then there's going to be some sort of contraction or consolidation or whatever you want to call it. How they use AI are going to rise to the top, and hopefully we're the latter. But like, you know, we're humble. And of course we're learning stuff every day, so we'll try to be better. Just introspect on that.

Terrific. Well, I can't thank you enough for spending a little bit of time with us today. Arthur. Thanks again. Uh, until next time. This is Sharon Spielman with Vision Systems Design. Stay curious.

Well, that's a wrap for this episode of visions produced by Endeavor Business Media, a division of endeavor B2B. Thanks very much for tuning in. If you enjoyed today's show, be sure to subscribe to the podcast and share this episode with a colleague who would find it helpful. Until our next episode, you can find us at vision dash systems dot com or on LinkedIn, Facebook, or for more insights, updates, and breaking news to keep you in the know. Thanks for tuning in. Until next time, stay focused on your visions.