Zeabuz (Trondheim, Norway), a spin-off company from the Norwegian University of Science and Technology’s (Trondheim, Norway) research center for autonomous marine operations and systems (AMOS), has developed and deployed an autonomous passenger ferry vessel.
The all-electric powered vessel, christened MF Estelle, is operated by the Swedish ferry company Torghatten (Brønnøysund, Norway) under the brand name Zeam, an acronym for Zero Emission Autonomous Mobility.
Instead of a captain and crew, the vessel is equipped with a large array of sensors, as well as AIS (automatic identification system), an automatic tracking system that is integrated with GPS and supplements marine radar. AIS is used by maritime vessels to track marine traffic.
Zeabuz CEO Erik Dyrkoren says autonomous maritime vessels have been in development for some time, however, in recent years, there has been an increased urgency to bring the idea to fruition. For one, fewer people choose to work on board vessels as captains or crew members. By integrating sophisticated technology such as machine vision and AI, the need for live crew members to operate a vessel is reduced.
Another important aspect is the trend to reduce road traffic and the corresponding carbon footprint while providing a viable, sustainable option for transportation of people and goods. One way to do this, he says, is to utilize and revitalize waterways throughout urban areas. The MF Estelle, which can carry up to 24 passengers, is currently deployed and operating in Stockholm, Sweden, where it launched operations in 2023.
To give a personal example, Dyrkoren says he recently traveled by road from one ferry dock on one side of the water to the other dock on the other side. The trip took him 20 minutes. By contrast, the MF Estelle runs on a predefined course, about 700 m across the Riddarfjärden Bay, to reach either destination, in approximately 6 minutes.
The Boat That Runs Itself
The ferry is equipped with onboard computer, AI-assisted software, and an array of sensors. Dyrkoren says that the MF Estelle is equipped with two Ouster (San Francisco, CA, USA) OS1 LiDAR sensors, one mounted at the front of the vessel and one at the rear; and 11 Simrad (Lynnwood, WA, USA) Halo25 radar sensors mounted on the roof. Four LUCID Vision Labs (Richmond, BC, CAN) Triton TRI054S-CC cameras equipped with 5.4 MPixel, 2880 x 1860 resolution Sony IMX 490 sensors, enabling wide-angle image capture in daylight or nighttime lighting conditions, are mounted to cover each side of the vessel. “So that gives us the 360° perception that is required. The more different types of sensors we have, the better perception we get.”
Designed for Advanced Driver Assistance Systems (ADAS) and autonomous driving, the cameras, rated IP67, are designed to operate in harsh conditions.
The sensor systems, connected to the onboard computer by cables, gather data/images and stream them to the computer, which Dyrkoren says is an off-the-shelf gaming or industrial PC.
“The architecture of our system allows us to connect to nearly any type of sensor,” Dyrkoren explains. “At the first stage, we process the influx of data from each sensor type. The data from all these processed streams are then fed into the heart of our system, the sensor fusion, in which detected objects and associated parameters from all sensors are analyzed and combined into one situational awareness (SA) dataflow.”
The combination of several sensors increases accuracy of the SA, while also providing redundancy when operating in challenging environmental conditions—rain, snow, fog, for example—that may limit the capabilities of some sensors, he adds.
Related: Researchers Develop 3D Model System for underwater Mapping
AI Does the "Thinking"
AI-assisted software, written and developed by Zeabuz, combines and analyzes the data to precisely identify and track objects around the vessel and take appropriate action to keep the Estelle on course while maintaining its safety. For example, if the system “sees” another vessel traveling on a potential collision course with the ferry, the software might respond by slowing or even stopping the Estelle and remaining in place until the other boat passes, then resume its course.
“When operating without a captain on board, our system not only needs to navigate the vessel safely to its destination, it also has to know that the technical condition of the vessel is satisfactory,” Dyrkoren says. “Hence, we have a separate system only dedicated to what we call supervisory control, which continuously monitors the health of the systems onboard—these are all essential parts of a captain’s responsibilities, which also needs to be automated.”
High Level Autonomy
Dyrkoren notes that when it comes to discussing or describing autonomous vehicle operations, there are several different levels of autonomy, none of which ever completely removes human supervision out of the equation. Degrees of autonomy, as generally defined defined by the International Maritime Organization (London, England), run from fully manual operation by a human operator (degree 1/no autonomy) to fully automated operation with remote limited human monitoring/supervision (degree 4/high level autonomy).
The MF Estelle is designed to operate at Level 4, Dyrkoren says. Currently, one crewmember is on board the boat while it operates. That individual does not perform the traditional roles and duties of a captain or seaman, but instead monitors and maintains onboard equipment, and can intervene in circumstances for which human intervention becomes necessary.
Remote employees based in an operations center on shore stays in communication with the vessel via computer and 5G wireless network, but they monitor, rather than control, the operation of the vessel, Dyrkoren says. They have protocols they follow for unusual or emergency situations, should they arise, and they can intervene if needed. If the ferry encounters an unsolvable situation, the Zeabuz system is designed to transition to or set itself to one of several programmed “minimum risk conditions,” in any conceivable circumstance. MRCs can be any of a number of actions, from standing still on Dynamic Positioning—a computer-controlled system used to automatically maintain a vessel’s heading and positioning without the use of mooring lines or anchors—to moving a certain distance, proceeding to the nearest safe spot onshore, or dropping anchor.
“This ability is essential for ensuring safe operation without a captain on board,” Dyrkoren says. “This also allows sufficient time for the remote operations center to assess the situation, get the proper overview, and then execute the necessary response, such as remotely control the vessel out of the situation.”
The goal is to transition completely to remote supervision by the end of 2024.
Related: Depth Cameras Can Fill LiDAR's Autonomous Vehicle Blind Spots: Here's How
Sailing into the Future
If all goes as planned, Zeabuz and Zeam will deploy at least two more ferries in Stockholm in 2024.
“Estelle is a sustainable and green pioneer, offering a solution to traffic congestion and inspiring alternative modes of transportation,” Torghatten CEO Stein Andre Herigstad-Olsen stated in a press release in June 2023. “With its electric, self-driving capabilities and advanced digital security, Estelle sets the stage for future advancements in the industry.”
Dyrkoren also says Zeabuz is working with other companies to develop similar operations in other cities in countries across Europe, the Middle East, and even in the U.S.
Regulatory issues can be challenging, he notes. For example, regulations and protocols addressing autonomous maritime traffic are still something of a work in progress. Nonetheless, some countries are showing great interest in the idea of utilizing waterways in this way. Not only are many cities built along major waterways, making ferry service a good fit for mass transportation, but the goals of replacing diesel powered maritime vessels with green electric vessels to further reduce carbon footprint, as well as significantly reducing the numbers of vehicles operating on highly congested roadways, are becoming much higher official priorities.
“We are basically reliant on having local authorities be open minded towards testing these things out, because the standard order-based mobility regulations don’t allow for unmanned operations yet,” he says. “But some countries are really moving quickly now to allow this—France, for instance—because they see all these inland waterways that are hardly in use, while the roads are full, and they recognize they need to do something about that.”
