AEye’s Apollo LiDAR Heads to the Moon

AEye CEO Matt Fisch explains how Apollo LiDAR was adapted to withstand launch shock, vacuum, radiation, and extreme temperatures aboard Lunar Outpost’s Pegasus Rover.

Key Highlights

  • AEye’s Apollo software-defined LiDAR was selected for Lunar Outpost’s Pegasus Rover.
  • The system must endure launch shock, vacuum, radiation, and extreme lunar temperature swings.
  • MEMS-based design and radiation-hardened electronics help adapt the LiDAR for space.
  • Reprogrammable scan patterns support autonomous navigation across the lunar surface.
Listen on Apple buttonListen on Spotify buttonListen on iHeartRadio buttonListen on Podbean button

In this episode of Visions: A Machine Vision and Automation Solutions Podcast, host Jim Tatum interviews Matt Fisch, CEO of AEye, Inc., about their Apollo software-defined LiDAR being selected for Lunar Outpost's Pegasus Rover. They discuss technical challenges of adapting automotive-grade LiDAR for the lunar environment—surviving launch shock, vacuum, radiation, and massive thermal swings—and how MEMS-based design, radiation-hardened electronics, and reprogrammable scan patterns made the system suitable for autonomous lunar navigation.

Visions: A Machine Vision and Automation Solutions Podcast, is the podcast for engineers, designers, integrators, and end users who want to keep an informed eye on the imaging and machine vision industry. Every Tuesday we will explore the latest in imaging trends, developments and solutions. Here you will find interesting, useful insights and observations from expert interviews, solo episodes, even the occasional panel discussion, all of which aim to expand your knowledge on imaging and machine vision.

Related: AEye's LiDAR System Chosen for Moon Rovers

Transcript

Hi everybody and welcome to visions. Today we're joined by Matt Fish, CEO of AI Inc., a leader in software defined high performance lidar solutions. You know, it's not often we venture off planet, but this story was just too good to pass up. AI's Apollo LiDAR has been selected for lunar outpost Pegasus rover, bringing advanced perception technology to NASA's return to the moon. Pretty exciting stuff. Welcome, Matt, Thanks for joining us.

Thanks, Jim. Exciting indeed. We're really excited to say that would be an understatement, actually. Glad to be here.

First and foremost, how does one get involved in a project like this? I mean, I know LiDAR is made for navigating the earth. This is kind of new. Can you give us some background on it?

Yeah, absolutely. Uh, as we get into the discussion, we'll see that our products has some unique attributes tied to it that made it particularly interesting. aside from some of the, regulatory discussions that are going on right now that essentially require a US based LiDAR provider, in this case, supply chain question. We'll get into the details about some very unique characteristics of our Apollo LiDAR that caught the attention of lunar outpost and made it most suitable for this mission. Okay. Well, getting back to sort of my first thought, Apollo in the LiDAR was essentially designed for use here on earth. Um, so what are some major technical challenges in adapting Apollo for the lunar environment? Yeah. So look, I mean, if you take a look at the NASA Artemis project and read through the announcement, you'll see that the intention here is to actually autonomously navigate a vehicle on the lunar surface. So here's the problem. Statement leaders. Technology that has been used in space before, just, for example, for docking maneuvers. I think there's been some press coverage about that in the past. And that's essentially what we call a pure ranging function where the laser is focused on looking at a docking locking point and measures distance essentially. Uh, and, look, it's hard to build electronics, especially laser based electronics that function in space. But the challenge faced by NASA and Lunar Outpost was that essentially, how do you bring autonomous driving grade lidar? And that is 3D lidar, essentially, that can look and characterize objects and perform complex operations. How do you bring such a capability and make that space hardened? Because, what we heard and based on our own industry research showed that really what was available today off the shelf was docking lidar or one dimensional ranging lidar. And so we started thinking, wow, we really built this thing to withstand the trials and tribulations of the automotive industry at the time. it came across to us like the most rigorous, uh, set of shipping and usage parameters out there, right? You've got cars that run everywhere on the planet ranging from, you know, minus forty degrees to fifty degrees Celsius. and, it was a good starting point, right? And given that the industry, the space industry in this, case could maybe take many, many years to develop a space grade lidar, using space grade expertise from the ground up. The approach that we offered here is to say, look, we've got a really robust automotive solution. Could we harden it for space? so to speak. So that was sort of the overall thought process here. I know this is an audiocast, but I'm going to take my background off for a second because I want to show you something that really caught the attention, uh, of our customer and our partner in this case. And let me go off. Uh, this is the heart of our scanning system. This is a MEMs device. It's a micro electrical mechanical system. And I tilt it just right. You can see like a dot in the center. And that dot size is about one millimeter. And that is the only part that moves in the system, right? Today you'll see a LidAR. So you can just driving down the street two days ago and I saw Waymo go by. You have these big coffee cans on the Waymo and you see things spinning inside. Those are actually motors and mechanical mirrors. Those guys don't survive. this is not a typo. A thousand G shock testing which is required. First, the first and foremost mission, for a LiDAR going into space, especially one that's going to land on the moon, is can it survive the launch and the landing? And that kind of shock testing is even far more rigorous than what we see in the automotive industry. But this guy here, since we have de minimis, uh, moving parts in it, in fact, the only part is, you know, one millimeter size, it has almost no inertial mass. Um, that really caught the attention first and foremost of the customer. It's that whole ruggedness reliability piece. And then the next question is, does it work in a vacuum? and, lo and behold, because of the inertial mass of this device and the very light mechanical structures of the MEMs, we tested in a vacuum and it worked quite well. The third piece that's really sort of front and center for the reliability is immunity to radiation. Okay. So earth has this thing called the magnetosphere that surrounds the planet and protects us in all living beings and all living things from dangerous radiation in space. The moon does not have that kind of protection. And, this wreaks Havoc. on electronic devices. Sure. And so the mission for us, just purely from a device perspective is how do you, harden the electronics in the system. So it's impervious and immune to radiation. Last but not least, there's massive thermal swings on the moon. It's actually the most extreme environment in the solar system, believe it or not. Just the simple action of going from shadow to sun. Can swing the temperature five hundred degrees Fahrenheit. I knew it was big shocks. Minus four hundred in the shadow of a crater and plus two fifty if you're in the sun. And by the way, if you're standing behind a rock and then you step into the sun, or vice versa, those thermal changes are almost immediate. We're getting some help from Lunar Outpost on that part, because they know how to build rugged enclosures, but incredibly robust and difficult. And I think next we'll dive into some of the functional aspects for autonomy. But that was the bar or the ticket from entry is that, look, you guys, you have an automotive product, can you take it and harden it? to run in a vacuum, to survive, launch and to survive solar and other kinds of very damaging radiation. And that was the testing we endured. And, with some changes, modifications on the electronics piece to be, you know, more like space grade electronics or are fault tolerant. We're able to pass. And that really opened the door. Of course, there's a whole autonomous piece, which I'm sure we'll talk about next.

Okay. Well, then getting into, more the electronics, from a sensing standpoint, how different is lidar navigation from autonomous driving here? And what changes do you have to make to make it work?

Yeah, absolutely. Look, the data that we have to provide, in essence from the LiDAR system, uh, in technical terms, we call it a point cloud. It's basically a fancy name for a 3D image that comes out by and large, that's the same. Okay, so just the mechanics of sensing what's out there and delivering a data set that's digestible by the autonomous navigation system, I would say eighty to ninety percent of that is the same. There are a few things that are different, however. Um, first of all, uh, there's no map of highways on the moon, right? That's probably one of the key differences, right? And so the autonomous system has to do more reasoning and thinking, if you will. and I would say just as an example, one of the key challenges is the terrain and a camera can't see in 3D. It can't necessarily sense how far away a rock is or how rough the surface is, or what's the slope of the surface. Um, look, when I look at how autonomous systems are implemented today in terms of robo taxis and things like that, they're only using a small piece of the LiDAR data. For years, those systems have relied on cameras and in some sense, radars. And then now they're sprinkling a little bit of that lidar data on top of it. which, you know, if we take it back to the moon now, I think this is also a glimpse of the future of what we'll see on terrestrial driving as well. But there's so much more information you need about the terrain, the slope. And this one is actually, uh, super exciting. Um, when you shoot a laser, by the way, uh, my wife and I watch the forensic files and a lot of times they do analysis on stuff and they do, you know, spectroscopy or whatnot. When you shoot a laser out into the world and it bounces off of something embedded in that laser waveform is information about the material. Is it ice? Is it something else? and this is also important when you're navigating in an area where you may be encountering, you know, new states of matter and things like that. So, um, thanks to some of the advent of artificial intelligence, we're able to do real time processing, the laser at a level that's never been done before. And we can actually, for example, detect whether or not there's ice ahead on the surface. So that's new. this is going to be incredibly valuable. if you think about robo taxis today, for example, in terrestrial, you see them running in sunny states with great weather. What happens when it's snowing and there's black ice on the road? You see a lot of this stuff carrying over. There's one part about this that makes this feasible. Uh, in the shorter term, when it comes to the overall perception system, is that you don't have kids springing out with soccer balls right in front of you on the moon. Not today. There's not so much unpredictable behavior that it takes years to train sensors, to look for all that we're doing mainly navigation, looking at the terrain and what the terrain is composed of in some sense computationally and autonomous. Driving wise, that is a subset of the problem that's, you know, being solved today, for example, compared to a Waymo that's running on the streets of San Francisco. So there are some things that make this feasible. The timeline of this project is incredibly aggressive. NASA is looking to launch in 2028. so that's a lifesaver, in this project. And we don't have to deal with all the unpredictabilities of humans and all the things running around on the surface of the moon. but nonetheless, we have to do a much deeper job of providing rich data about the terrain far beyond what's being done, in terrestrial navigation today. Well, just looking ahead, how will that extrapolate into terrestrial applications? where are we now? And what could you see in five or ten years with that? Yeah. so there are two things. Let me talk about sensor and then let's talk about the interface too. I talked a lot about the sensor piece just now, about being able to classify materials. We can actually take stuff out of a scene that's irrelevant. like trees, for example, that are planted on the side of the road. This reduces the data set that the brain in the car has to deal with. And it reduces cost, right? Making autonomy more available. instead of like a high end, inexpensive type of feature. So I think this is really going to, you know, contribute directionally to cost reduction. Um, bad weather driving. I mentioned it, if you check some of the history of the announcements we've done, we have a collaboration with the University of Toronto Robotics School on a project called Winter. And it's a little bit of a play on words there, double entendre. but they work. That work with them has helped us to unlock some of these capabilities where we can identify hazardous road conditions. For example, black ice would be a potential. and also do things with particulates in the air, rain or snow, LiDAR because it sees differently than humans is able to cut through particles in many cases. And we've been working with U of T on that for some time. So lidar, we believe, will usher in, uh, expansion of autonomy and robo taxis and these things outside of what we call the Sunbelt states. Um, secondly, because lidar views the world, uh, the way machines view the world, you and I, uh, if you look at a lidar picture, you'd say, oh, that doesn't look so good. Well, that's because it only contains the data that a machine needs to see to make decisions. It's not a beautiful picture per se. Uh, additional, use of the data that's coming out of LiDAR. This is what we're going to see is going to reduce the cost. Again, this is going to reduce complexity in the compute systems in the vehicles that we see. It'll accelerate the deployment times of new features and make it more accessible. I see that as being one of the most profound impacts.

Yeah. Okay. just kind of as an aside, I know lidar has some, drawbacks, it is challenged by things such as dust clouds and rain and fog. Do you have anything like that on the lunar surface you're dealing with or you're just dealing with, you know, rocks and obstacles, that sort of thing?

Yeah. Dust can be an issue. LiDAR actually sees better than a human does in dust and fog. We also have some techniques, again, because we're not trying to solve for kids chasing a ball into the street or car jumping in front of us suddenly at one hundred kilometres per hour. Um, we have things that we can do. For example, there's a technique in the light industry we call multiple echoes, which allows as particulate moves in the air, it allows us to take multiple shots of a scene and um, through the use of artificial intelligence, essentially see through that particulate. As it works today, we can see better than a human. As we're working with our partners in the perception space, we can make it even better, because the environment is simpler. the real time processing and reaction time driving on the moon opens up new doors for us. And eventually we'll take those techniques, back to earth in the future as computing continues to advance, I think, you know, you've got companies out there like Nvidia that are bringing AI capabilities to the edge, which will be important in bringing those techniques from the moon to the earth. And then you also see new companies coming out there that are bringing more reasoning, type of intelligence, that will operate more on localization as opposed to mapping and precision mapping. as, those type of software begins to get deployed on the edge LiDAR is going to be even more valuable because again, that data coming to that intelligence, is optimized for machines. so there's just a number of things that are going to help. It's going to drive costs down. Bad weather driving. We're learning, in essence, bad weather driving on the moon because we can employ new techniques there. As edge computing begins to advance back on earth, we'll be able to bring those techniques back to earth and improve bad weather driving, for example.

Okay. my understanding is you guys are, software defined LiDAR or specialized in that. Um, what's the advantage of that over any other approach?

Thank you. In the end, once you pass the bar on ruggedness. I believe this is what won the deal because think about it. If you send something into space in the moon, you're never getting it back. And it's never. Maybe someday in the future we'll send an astronaut to replace something, but it's not serviceable, essentially, ever once you launch it. Okay. The one of the things that that we think is amazing about that MEMs chip that I showed you is that we can control shot by shot, that laser. And it allows us to create an intelligence that can be reprogrammed completely. Today's LiDAR has a spinner that kind of looks like a lighthouse, and it's very fixed way of scanning that's driven by mechanical constraints. We can create all kinds of we call it scan pattern. Just imagine it going into zoom in mode, zoom in, zoom out mode. And, you know, rethinking about how we look at a scene. Look, this is the first time this is being done. And, um, we don't know. We're taking a really good guess at the problems we're going to encounter, how we're going to need to see things like frame rates, resolution and whatnot. That might not be right. And this was just really a key point in, you know, finding the perfect fit for this mission is that we can just reprogram it. There's no hardware change that's required. We can drastically alter the nature of how the LiDAR works through an over-the-air. Oh, maybe we call it a OTS, uh, over the space, command back to the lidar unit that you can reprogram how it works remotely. I think this sealed the deal at the end of the day, especially when you can just never touch the device or maintain it again once it goes out into space.

Yeah. That does make a lot of sense, those are things a lot of people just don't think about. You know, you're shooting it up there and it's not coming back. it sounds simple, but I just don't think a lot of people think about that. you have been very thorough Um, can you think of any, uh, other lessons that might, you might learn out of this whole thing with, with mapping and LiDAR in the the moon, I guess for lack of a better way of putting it.

Yeah, a couple of things, if you don't mind. I just want to talk a little bit about. Absolutely. We'll keep it somewhat technical, but for us as a business, the fact that we're going to have to pass all these reviews and audits and we were selected for this program says a lot about the robustness of our product. And by the way, pick an industry mining, autonomous trucking passenger vehicles. Look, we have all this new LiDAR technology because of advances in tech. It's more affordable and we can start putting it into consumer markets. But the question is, is okay. Is it really rugged enough. It's new. is it going to stand up to ten years of driving around the streets of Manhattan, for example, or in the Gobi desert? Um, this is a great proof point for us. and by the way, that software defined ability, is also another great proof point because vehicle manufacturers. They don't want to be bringing things in for service. if they find something that they need to change and they, that is where this OTA term comes from is that they just want to be able to update the sensors while they're out there in so many parts that in the garage at night. So these are really strong proof points for us.

 

About the Author

Jim Tatum

Jim Tatum

Senior Editor

VSD Senior Editor Jim Tatum has more than 25 years experience in print and digital journalism, covering business/industry/economic development issues, regional and local government/regulatory issues, and more. In 2019, he transitioned from newspapers to business media full time, joining VSD in 2023.

Sign up for our eNewsletters
Get the latest news and updates

Voice Your Opinion!

To join the conversation, and become an exclusive member of Vision Systems Design, create an account today!