Ferry Operators Count on NIR Cameras, AI, for Accurate Passenger Tallies

Challenges Counting Fast Moving Cars and People

In the past, passenger counts were conducted when passengers purchased tickets; they would buy them onsite and crewmembers would individually count passengers as they boarded the ferry. However, when the ticket purchasing process became automated and the boarding process free flow—that is, people and vehicles boarding the ferry without having to move through lines and turnstiles— it became necessary for human employees to manually count passengers in every vehicle being driven onto a ferry.

However, people in vehicles move quickly, making the counting process difficult and error prone. Indeed, Norwegian maritime authorities, suspecting that the counts were chronically inaccurate, investigated, verified that the counts were, in fact, chronically off and gave the ultimatum for the ferry operators to fix the problem or face corrective action, says Andre Ødegårdstuen, product manager at Adigo Mechatronics.

Then the COVID epidemic hit, and the emphasis on automation solutions to minimize human contact and interaction became paramount. In fact, four of the largest ferry companies in Norway agreed to sponsor the development of an automated passenger counting system, Ødegårdstuen says.

How the AI Vision Passenger Counting System Works

The cameras capture images from each vehicle and transmit them, via ethernet, to the computer, which is loaded with AI software developed and written in-house by Adigo. The software analyzes and processes the images, provides an accurate passenger count from each vehicle, then immediately sends that information to the ferry’s passenger logging system via 4G wireless network.

“There is no preprocessing on the camera, as the AI works best with the raw data,” Ødegårdstuen explains. “The data transmission from the camera is via GigE. There are multiple tasks being handled by different machine learning models, such as detecting the presence of a vehicle in the lane, determining the vehicle type, detecting passengers and an aggregate model that collects info from other models to give the final output. We do not use any conventional computer vision, as machine learning approaches are in practice always more robust given a certain scale.” 

The cameras also read license plates and classify the type of vehicle passing through the inspection station. The system does this as part of the ticketing and boarding process, and OCR technology is utilized for this purpose, Ødegårdstuen says.

The cameras, Ødegårdstuen says, operate at approximately a 45 fps frame rate and half-millisecond exposure time.

Ødegårdstuen also notes that the image data the system collects is sensitive information, and Europe has strict regulations about how this kind of information is used and stored. Therefore, all data processing is done locally, on-site, with no external transmission or storage of that data outside the counting system process.

“We have to restrict where we store that information, so we try to minimize the spread of those images,” he says. “We process them locally and then we delete them. In some rare cases, when we see that the system is uncertain, we can take some of the images and annotate them in order to improve the machine learning algorithms.”

Overcoming Challenges for the AI Vision System

One challenge was placement of the system. Seawater and salt air can cause issues with hardware and infrastructure, therfore, the system had to be located onshore, away from the dock and the ferry, Ødegårdstuen says.

The prototype system utilized five cameras mounted on poles on either side of the boarding lane(s). This, however, is cost prohibitive for smaller ferry companies, says Ødegårdstuen, because hardware, especially cameras, is expensive not only to purchase but to set up and operate.

“To use it, you have to have (ethernet) cables crossing the road, so you either have to dig or you have to erect some sort of bridge above and that costs the ferry companies a lot of money. So to reduce the total cost of installing the passenger counter system, we’ve tried to optimize it so it will operate effectively with fewer cameras installed on only one side of the road.”

Another challenge was the initial reluctance of crews to embrace the system, Ødegårdstuen says.

“In the beginning, we got some resistance from the crews because they were worried that we were automating their jobs away,” he says. “This is a legitimate concern—they were right to worry about that—but now that we have the system in place, they see that there are other things they can do that are more rewarding, and less of a hassle, than to stay outside in the Norwegian winter and count people.”

What is the Future of the AI Vision Counting System?

The system, while still a work in progress, has proven its usefulness. Ødegårdstuen says. Indeed, the Adigo team has found that it is not only significantly faster than manual counting but has achieved a 98% accuracy rate in identifying and counting passengers.

Future plans call for several nPax systems to be installed on several new ferry routes in Norway as early as the end of this year. Negotiations with ferry companies in other European countries is also underway, Ødegårdstuen says.