Deploying Vision-Guided Robotics in Real-World Factories

Sharon Spielman interviews Eureka Robotics’ Francisco Suarez about physical AI, vision-guided bin picking, and precision handling—from supermodels trained on millions of parts to edge-based controllers.

Key Highlights

  • Eureka Robotics’ supermodel is trained on millions of parts to support challenging bin-picking and precision-handling tasks.
  • Edge-based AI controllers and flexible platform design help integrate vision and robotics into real-world applications.
  • The episode examines emerging trends in physical AI and factory automation.
Listen on Apple buttonListen on Spotify buttonListen on iHeartRadio buttonListen on Podbean button

In this episode of Visions, VSD's head of content Sharon Spielman interviews Francisco Suarez from Eureka Robotics about physical AI and vision-guided robotics for bin picking and precision handling. They discuss Eureka's supermodel trained on millions of parts, edge-based AI controllers, open platform flexibility, sensor integration, real-world performance metrics, best practices for deployment, safety and data security, and emerging trends. Listeners get practical advice for integrators and engineers on reducing deployment time, improving accuracy and cycle times, and addressing challenging parts and factory conditions.

Visions: A Machine Vision and Automation Solutions Podcast, is the podcast for engineers, designers, integrators, and end users who want to keep an informed eye on the imaging and machine vision industry. Every Tuesday we will explore the latest in imaging trends, developments and solutions. Here you will find interesting, useful insights and observations from expert interviews, solo episodes, even the occasional panel discussion, all of which aim to expand your knowledge on imaging and machine vision.

Transcript

Physical AI is really making waves in industrial robotics, opening up new ways for machines to see, understand and actually interact with the real world around them. Today, we're chatting with Francisco Suarez from Eureka Robotics to get his take on how this technology is changing what's possible on the manufacturing floor. Thank you for joining us, Francisco.

Thank you for having me.

So first, why don't we get a quick introduction to Eureka Robotics as well? As I had read a press release about the partnership that you guys have with Shelley Automation. So if you could talk about that as well, that would be a good starting spot.

Yeah. So Eureka Robotics spun out of Nanyang Technological University, Singapore in 2018 and we're currently specializing in physical AI vision guided robotics for high mixed beam picking and precision handling across automotive, aerospace and electronics. Regarding the partnership with Shelley, it's a strategic collaboration where we are trying to span physical AI deployments across North America factories, and Shelley provides a regional systems integration and automation expertise and we Eureka provide a. Are agnostic AI controllers and the vision software to reduce deployment timelines from maybe months to days.

Great. All right. So first, I'd like to dive into the core technology that Eureka Robotics has. Can you explain how Eureka's AI vision system leverages a supermodel trained on millions of parts, and what sets that machine learning approach apart from traditional vision algorithms.

So the traditional approach relies on rigid rule based geometric matching. And this, this approach usually breaks under challenging conditions such as shifts in the lighting. When you have reflections on the parts or skews with small variations. So a lot of this requires an intense amount of manual programming slash tuning for every single part. So the idea with the supermodel is to have a foundational neural network that we pre-trained on millions of industrial parts and, well, also millions of synthetic images. And then this kind of enables the zero shot picking for our model items. So these models will inherently generalize across surfaces and occlusions random orientations. So that's that's the idea.

Okay. Great. can you talk a bit about how you designed the software interface to make advanced AI accessible to engineers and integrators? Those are our readers, at different skill levels.

We have been following a very iterative approach. So. we started with tooling that was developed for our own use. So it was kind of like very low level. and then we started adding visual interface interfaces just to make our lives easier. but once we started collaborating with partners, with customers. Then we realize, okay, there is a difference of skills, as you mentioned, right? So we have introduced a tools that allows any level of, with a person with any level of skill to directly work on our system. So we have things such as wizard that allows the operator to directly import the CAD file. And then there is a step by step guide that takes you through things such as calibration, a peak point definition, a working spaces and and these kind of things. So this has been the approach very iterative, taking customers and partners feedback very seriously and doing it quick.

Okay. All right. So now I want to move over to the software lifecycle and architecture. Um, how are software updates and AI model training managed on site without disrupting the production?

So, we run things on the controller. So it's what nowadays we refer to edge computing. And what we do is that everything is containerized uses containers. So during schedule shifts, for maintenance, what we'll do is update these containers and. The tuning, the configuration changes the upgrades. We run them ourselves. Let's say offline. So it's not running on the production line, uh, with synthetic data, with real captured data, if we have access to it. And then at the end of the day, when we deploy to production is just a matter of a couple of. Yeah, probably it's going to take one hour just to make the upgrade. And so that's how we go about not disruptive on site updates.

Okay. So what advantages does the open platform running models locally with in-house retraining offer integrators in terms of flexibility and control?

So this open model has to I mean you could you could think of like a two fold area. So two areas where you have benefits. First, um, we support robots from different brands. And then it's arm agnostic, right? So that's one of the key benefits. And then in terms of the key of running things locally is that the customer has a control over the data and we have consistent latency. Usually it's going to be low because we are going to tune it for that. So just your cycle times are going to be consistent. You have zero dependency on the cloud. And yeah your your IP. It is going to be protected because you control everything. Everything is there. So you are not sharing part geometries or any proprietary information.

All right. I kind of want to move over to, sensor integration and hardware. So can you explain the role and the importance of the 3D camera in achieving reliable and accurate bin picking, and how critical is sensor quality?

Here, the camera is needed when you need to. So the camera is going to give you point clouds. So this 3D information where you are able to compute things such as surface normals. And these are essential to estimate the orientation of your parts and such that you can have precise calculation and also such that you are able to to do collision free motions to, for example, go into the bin. So that's the role of the 3D. And then in terms of sensor quality, things like a reflection suppression resolution, these are critical because they will affect the quality of the point clouds. point clouds usually get affected by things such, uh, shiny, oily or black metallic parts, which usually you will see them kind of like these phantom Surfaces like you have like these ripples in the point cloud. So yeah, definitely sensor quality also plays an important role to avoid these kind of failures.

All right. I want to move over to, real world deployment insights and lessons proven, uh, so proven performance in manufacturing, um, with more than thirty million pix completed in production at clients like Toyota and Pratt and Whitney, what performance metrics or lessons stand out that demonstrate system maturity and reliability?

I'll say that for this kind of customers, you need to prove your picking accuracy. So reaching this kind of ninety nine point five percent accuracy is of utmost importance. another thing that we have found that it is really important for customers is the cycle times. So sometimes they they are trying to reach the cycle times around two three seconds. So your vision system has to be really fast. Otherwise you're going to be the bottleneck in terms of key lessons. Um, maturity is definitely defined by how you handle edge cases. I think sometimes when we are developing, when we are testing, we like to go through the happy path, but as early as possible, you need to start addressing edge cases, damage bins, overlapping parts, clutter bins, oily, shiny, all of these things that I was mentioning. Because at the end of the day, you should be addressing these edge cases and this shouldn't trigger line stops.

I want to talk a little bit about best practices. Um, and this might be a place for you to talk about your guide that you offer on your website. So what practical advice or best practices would you offer to machine vision engineers looking to optimize bin picking projects for speed accuracy and reduce deployment time.

Yeah. So first, um, mechanical design is very important. so tooling clearance, gripper design reachability, this usually costs more picking failures than any vision miss detection. So at the end of the day optimizing the whole system to the end effector. has to be done even before you start tuning your algorithms. And as I mentioned before, you should start testing your worst parts as early as possible because you want to be as close as possible to the production set up production conditions. So you need to start benchmarking using dirty scratch or reflective parts rather than using the brand new ones that they don't have any effects. And camera placement is another factor that sometimes engineers take too lightly. So we want to be mindful of minimizing shadows. Maybe the beam is occluding your parts that you see the whole being so correct. Camera placement is very important. And yeah, as you mentioned, we have that in our field guide for engineers and many more details that can be found there.

Okay. I'll be sure to, post the link for that. when we put this out there. all right, so let's talk about overcoming common vision system challenges. What are the typical challenges around vision system accuracy or robustness that you've overcome and how should professionals prepare for these?

So yeah, I mentioned some, right? Yeah. Reflections and yeah, metal surfaces tend to be challenging. dark surfaces, dark rubber or plastic because the way they absorb light. and something that we have learned the hard way, maybe is the shifting factory daylight. Sometimes the factories, they just run like during the day. they don't have any controlled lighting. And then your vision system needs to be able to deal with that. So the way that we address these challenges is that we leverage on a randomized synthetic training. So domain specific but randomized so that the AI models basically are invariant to lighting shifts. And then even like for the case of dark parts, we oftentimes will use things such as structured light to improve and to help the 3D sensing. So those are the ways that we try to overcome those challenges.

All right. Let's move on to safety and security. So, there's some embedded processing concerns as physical AI symptoms become more embedded on the factory floor. How does Eureka address safety and security considerations inherent in AI vision processing and data management?

Yeah. So I think there are two areas in terms of safety. There is the classic one in terms of functional safety. So how you make sure that your system is not going to collide or crash. So, what we work very hard on is to get to the point where the AI calculates a candidate grasp or picks as deterministic as possible. And the motion planner, also plays an important role Because that also drives determinism in these systems. And yeah, the traditional approach as well of using hardware interlocks to make sure that functionally everything is safe, both for the system and for human operators. At the end of the day, you want to prevent collisions with. You don't want to collide with any terms in terms of data security and on premise. Edge computing is going to ensure that no images or CAD files leave your your factory network because You control that you don't need to connect the controller to the network. Okay.

So, you know, we've made leaps and bounds in the technology, over the years. And as you look ahead, what emerging technologies or trends in machine vision and AI do you see shaping the future of physical AI for manufacturing automation?

I think first, a real acceleration. It's starting to show the power that you get out of this approach. So what you are able to train virtually entire cells and stress test in this digital twin world using physics engines. And then you can do this overnight. So you get to do millions of tests. So this is something that it's maybe not that new, but I think it's picking up recently these days. Okay. And the benefit is that it's going to slash on site commissioning because You will have done. A good amount of testing in this world. The other thing that I see that is picking up is something that we have been talking about the industry, but we're starting to see actual systems doing it is the multimodal action models. So basically moving from this passive part detection to an approach where you do end to end. So kind of like see an act policies and that will adjust dynamically even while the robot is moving. So I think that's probably a bit more kind of like in the future.

Perfect. Yeah. Is there anything else that you would want to let the readers and the audience that's listening to this at VSD know about Eureka Robotics, specifically about machine vision and any vision systems that you use?

I think we cover most of it. I'll invite them to check the the field guide, and we put a good amount of effort writing, writing that one. And then yeah, I think it's going to be a great resource for them.

Contributors:

About the Author

Jim Tatum

Jim Tatum

Senior Editor

VSD Senior Editor Jim Tatum has more than 25 years experience in print and digital journalism, covering business/industry/economic development issues, regional and local government/regulatory issues, and more. In 2019, he transitioned from newspapers to business media full time, joining VSD in 2023.

Sign up for our eNewsletters
Get the latest news and updates

Voice Your Opinion!

To join the conversation, and become an exclusive member of Vision Systems Design, create an account today!