To reduce food waste, Lufthansa Industry Solutions is developing a machine vision system to detect and analyze untouched and partially eaten food leftover on inflight catering trays.
The IT consulting firm, a wholly owned subsidiary of Lufthansa Group (Cologne, Germany; www.lufthansagroup.com), is currently in the planning stages for production readiness. Lufthansa Industry Solutions has completed a minimum viable product phase—a process to develop an early version of the vision solution with enough features to satisfy early adopters, according to Tim-Patrick Sass, Business Manager of Artificial Intelligence and Data Analytics at Lufthansa Industry Solutions (Norderstedt, Germany; www.lufthansa-industry-solutions.com).
Production rollout probably will not occur before 2024, according to the website Simple Flying, which also said testing of the system would likely occur throughout 2023.
The idea behind the waste-reduction machine-vision solution is to gather data on what food is leftover on which flights with a goal of tailoring food offerings to each specific group of passengers, leading to a reduction in the amount of food thrown out after each flight.
Lufthansa Industry Solutions is working on the project in partnership with other departments and subsidiaries of the Lufthansa Group, including Lufthansa; Swiss International Air Lines; Eurowings Discover, an airline focused on leisure travel; and Lufthansa Group CleanTech Hub.
Designing a Machine Vision Process
An important step in improving an operational process such as in-flight catering is to gather and analyze objective information.
That’s where machine vision comes into play. The edge-based vision solution likely will be positioned near a conveyor where food trays are manually placed after they are removed from a flight. The solution will take pictures of the trays once they are on the conveyor and before they are cleaned and sanitized for reuse.
Based on the images, a deep learning algorithm will detect each uneaten component of food in the picture and match it with a component in a database. It also will determine the weight of the food left on the tray.
Information on the uneaten components will then be stored in a database and associated with a flight and booking class. An airline would then be able to further analyze the data to “gain deeper insights,” explains Sass.
The solution will not rely on ambient light; however, the specific illumination setup will vary, depending on the specific conditions at each location where the trays are unloaded and processed, he explains.
While a final decision has not been made, Sass says it's likely that the AI processing necessary to detect the food components and weights will be done on the edge device, while further analysis and report generation will be completed in the cloud.
The team also has not made final selections on specific components for a production device. However, they will likely include single-board computers from Raspberry Pi Foundation or Nvidia as well as a connected camera, Sass says.
By analyzing specific KPIs, Lufthansa Group would be able to design better catering plans and serve passengers by providing food they’ll eat.
Going Green
Equally important, the revamped catering offerings will also help the airlines involved in the project improve their environmental stewardship.
Through Lufthansa Group CleanTech Hub, Lufthansa Group has set a goal of reducing net CO2 emissions by 50% by 2030, compared with 2019, and to become carbon-neutral by 2050. The CleanTech Hub also focuses on non-Co2 emissions, waste, and noise.
The airline industry as a whole generates a lot of cabin waste. The International Air Transport Association (IATA), a trade group, (Montreal, Canada; www.iata.org) estimates that 20-50% of cabin waste is comprised of untouched food and beverages.
