A Fish Tale

To maximize profits, fish farmers are using a machine-vision system and smart software to sort good fish eggs from bad

Andrew Wilson, Editor

According to figures published by Statistics Norway, the total sales value of reared salmon and trout in Norway was $2.4 billion in 2007, with the quantity of fish sold estimated at more than 814,000 tons, an increase of 17% from the previous year. As can be seen from these figures, fish farming in Norway alone is no small business, with the companies involved looking for ways to maximize the profitability of their products. To do so, many breeders are looking to save costs by using automated systems to ensure their reared salmon and trout are the largest and healthiest while minimizing the cost of breeding, rearing, and feeding the fish.

Click here to enlarge image

One of the simplest methods of producing fish eggs is that of tank spawning. In this method, male and female fish are placed in a spawning tank and, after the females ovulate, the males stimulate the female to release the eggs and fertilize the spawn. Fertilized eggs are then carried with drain water from the spawning tank to a screened collector where they are packaged and sent to fish hatcheries.

Before these eggs are allowed to properly develop, however, they must be checked to see if the embryo within is alive or dead, too small, or whether the egg is infertile or contains developing twins. If the eggs are dead, infertile, or too small these eggs must be rejected. In what may seem a draconian measure, those eggs that contain twins must also be rejected since fish developed from such embryos may be smaller and frailer than those that contain a single developing salmon or trout.

Trout and salmon

“In Norway,” says Thor Vollset, president of Tordivel, the company behind Scorpion Vision Software (http://www.scorpionvision.com), “a small number of hatcheries supply large batches of salmon and trout eggs to fish farmers across the country. Before shipping the eggs they must be properly sorted to ensure that over 98% will finally develop into large healthy fish. To accomplish this, eggs must be sorted at the rate of up to 125,000 eggs per hour or nominal inspection rate of 40 eggs/second—a task once relegated to manual inspectors that sampled batches and used statistical analysis to estimate the percentage of good and bad eggs.”

To automate this process, the company Maskon, a developer of advanced automation systems, called upon Tordivel to develop a machine-vision solution to sort both salmon and trout fish eggs. Now, after the manufacturer has built 12 machines that are being used in five different locations by breeders in Iceland and Norway, the machine can sort eggs at the rate of 125,000/hour (see Fig. 1). To accomplish this task, the designers coupled a number of custom mechanical feed mechanisms with custom lighting systems, off-the-shelf cameras, software, and PC-based mechanical sorting systems.

FIGURE 1. To automate the process of sorting fish eggs, Maskon Automation Industries called upon Tordivel to develop a machine to sort both salmon and trout fish eggs at the rate of 125,000/h.Click here to enlarge image

“After a batch of 10 liters of fish eggs is poured into the machine’s holding tank,” says Vollset, “incoming water levels are adjusted so that the eggs float to a specific level. With the eggs at this level, a specially designed wheel containing 56 holes lifts the eggs in a counterclockwise motion in front of an illumination and camera system. To properly inspect the eggs, an image of the egg must be taken from both sides since the developing embryo will be located at a random position within the egg.”

Illuminating eggs

Because the carrier wheel is stepped at 1.75 rotations/s, 40 fish eggs can be inspected every second. The egg must be backlit from both sides and two images captured every 1/40 s. As each egg is presented to the lighting system, a master Omron PLC strobes a custom white LED illuminator (see Fig. 2). At the same time, the PLC triggers an A601f FireWire camera from Basler, also attached to the PC.

FIGURE 2. To backlight the egg from both sides, two custom-built white LED illuminators are triggered and the backlit images captured by two FireWire cameras.Click here to enlarge image

The captured image is transferred to the PC using a PCIX-based FireWire interface card and the uBCore 1394b driver from Unibrain. At the same time, an identical lighting and camera system, mounted diametrically opposite the first is triggered in a similar manner to capture an image of the opposite side of the egg. Two images of the egg are transferred to the host dual-core, 2.6-GHz Pentium CPU.

After two images are captured by the vision system, they are processed in two independent threads using Scorpion Vision 6.0. Eight selected image analyses are required to determine whether an egg is good or bad. If, for example, the embryo within the egg cannot be determined infertile or dead, too small, or whether twins are present, then the egg is rejected.

The machine uses a PCI 1730 I/O card from Advantech to tell the PLC which eggs to eject. If a single fertilized embryo is alive and present, then the egg is classified as good. Should the system be unable to classify the egg, then the egg is kept in the wheel and re-inspected up to three times before being classified as uncertain.

It is first necessary to isolate the position of the hole that contains each egg. Since this location is known, a circular region of interest (ROI) is used to isolate this region within each image. After the region is isolated, the location of the egg must be determined, which is accomplished by extending trace lines from the predetermined ROI toward the center of the image. The edge transitions of the trace lines can then be used to allow a radial arc finder to generate another circle of minimum fit that will outline the egg. If this circle is too small in diameter, then the egg will be too small and will be rejected.

Dead and damaged

“Measuring the intensity of pixels within this circular region determines whether eggs are dead or damaged. Because there will be no blood flow in embryos that are dead, this intensity level will be dark and the egg must be rejected (see Fig. 3a). Alternatively, if the intensity level is too light, then the embryo will not be forming properly or may be unfertilized and also must be rejected,” says Vollset (see Fig. 3b).

FIGURE 3. Measuring the intensity of pixels within a circular region determines whether eggs are dead or damaged. No blood flow in dead embryos means the intensity level will be dark and the egg must be rejected (top left). If the intensity level is too light, then the embryo is not forming properly or may be unfertilized and must be rejected (top right). Should four dark areas assumed to be eyes be present, the egg likely contains twins and must be rejected (bottom left). If two dark areas are determined, the egg is assumed to be good (bottom right).Click here to enlarge image

To determine whether a specific egg is good, the growing embryo is assumed to have two eyes. These will appear as dark areas within the image and are located dynamically by computing a percentile of the histogram of the intensity pixels within the image. Should four such regions be present, then the egg is likely to contain twins and must be rejected (see Fig. 3c). If two such areas are determined, the egg is assumed to be good (see Fig. 3d).

“One of the factors that can spoil the interpretation of these results is the presence of edge noise within the image,” says Vollset. To overcome this, Scorpion Vision incorporates a fuzzy logic or feature classification algorithm that uses parameters such as image intensity, contour arc, and the distance from the center of dark pixels to determine whether image noise is present. If so, then the image is deemed unreadable and will be rejected. The feature classification is a standard part of Scorpion’s Blob tool; it divides blobs into classes based on shape, size, position, and intensity.

After the images of these eggs are examined, they must be sorted by the Maskon system. They are separated as good, bad, or uncertain or not classified. To sort these eggs, software within the PC uses the processed image data to trigger an Omron PLC that in turn triggers pneumatic actuators positioned in front and around the wheel. As the examined fish eggs then traverse the wheel, these actuators blow the eggs into the three different holding tanks.

“In the development of the end-user interface for the system,” says Vollset, “it was important to allow each manufacturer to customize the user interface as required.” By writing this interface in a combination of Python and .NET, reports for custom functions and screens can allow the operator to visualize reports for each batch of eggs, the date the eggs were delivered and sorted, the number of eggs to sort, and whether salmon or trout eggs were inspected. “The final interface was implemented on a touch-screen since the system must operate in a wet environment,” he says.

features, advantages, benefits

After fish eggs are sorted, the good ones are transferred to breeding pens where they are allowed to develop into fish. Breeders are looking to maximize process efficiency with machine vision. “Once the fish have hatched,” says Thor Vollset of Tordivel, “they are fed by distributing food pellets throughout the pens. In this process, breeders must ensure that the fish are not overfed, since this will result in wasted food.”

Feed Control has developed an automatic fish-monitoring system in conjunction with Tordivel. In operation, the system uses Gigabit Ethernet cameras from Prosilica, wireless LANs from Moxa, and Scorpion Vision Software to analyze the number of food pellets in the water. This information is then used by the breeders to automatically control the amount of food delivered during any specific period.

After the fish reach more than 200 mm in length, they must be vaccinated before they are shipped to packing plants for distribution. At present, this process is performed manually, but according to Vollset, Maskon and Tordivel are currently working on a machine-vision system to automate vaccination.