By Steve Tomanovich
Between July and August each year, the US National Oceanic and Atmospheric Administration (NOAA) Fisheries Service conducts surveys to determine the abundance and size distribution of deep-sea scallops in areas between Cape Hatteras, NC, and Georges Bank (a large elevated area of the sea floor that separates the Gulf of Maine from the Atlantic Ocean, situated between Cape Cod, MA, and Cape Sable Island, Nova Scotia, Canada). To do this, sample height measurements from 125,000 scallops are taken from approximately 500 randomly selected locations using an 8-ft scallop dredge towed at 3.5 knots for 15 min. To increase the accuracy and speed of these measurements over current methods, William Kramer, an IT specialist at the NOAA Woods Hole Laboratory on Cape Cod, obtained a Pioneer Funding grant from the Chesapeake Bay Trust to develop a prototype machine-vision system.
System specifications required measuring scallops from 25 to 200 mm at speeds of 1800 scallops per hour (2 s/scallop). The design of the conveyor-based system had to incorporate a washing system to remove much of the debris from the catch before the scallops could be measured. The stand-alone system was also required to log readings to the PC-based vision system and the ship’s onboard computer system, which is used to log additional data about the catch.
Depending on their age and environment, scallops may be covered in barnacles or other organic material, making them appear dark in reflected light. Scallops are typically much lighter on the bottom than the top, except for “albino” scallops that are white on both sides. Because the system was required to handle scallops loaded with either side facing up, a backlight system provided a high-contrast silhouette of each scallop regardless of its shell color and texture.
Seeing scallops
Designed and developed by Compass Technical Consulting, the prototype vision system consists of a standard Series 2200 conveyor from Dorner Manufacturing modified to include a stainless-steel idler roller, an electroluminescent backlight, and a translucent belt (see Fig. 1). An aluminum shroud was fabricated over the conveyor to house the camera and laptop PC. To capture images of the scallops as they traverse the conveyor, an IEEE 1394-compatible Lightwise 1.3 monochrome camera from Imaging Solutions Group with an 8-mm-focal-length lens was positioned 35 cm over the conveyor. Running at 30 frames/s at 1280 × 1024, the camera provides 1/4-mm/pixel resolution and is compatible with LabView software from National Instruments.
Minimizing lens distortion was critical to ensure consistent readings regardless of a scallop’s position on the belt. With a 30-cm field of view (FOV) and a working distance of approximately 35 cm, the 8-mm lens provided the required focal length but resulted in distortion that caused scallop measurements to vary depending on their position. Although correcting the entire image using point-by-point remapping resulted in accurate measurements, it was too time-consuming. Rather, a mathematical correction was implemented that made no correction for scallops in the center of the belt and maximum correction for those located at the edges. This correction algorithm reduced the effect of lens distortion on scallop measurements to an insignificant level.
Soft triggers
One challenge was the range of scallop sizes and geometries that needed to be inspected. Sea scallops, Icelandic scallops, and calico scallops can range from a few millimeters to 200 mm in height. Scallop thickness is also variable, ranging from a few millimeters to more than 50 mm. All three species are loaded in any orientation and located anywhere across the belt. Because of the size, orientation, and thickness variations of the scallops and the presence of debris on the belt, typical camera-triggering techniques such as light or contact switches could not be used. With these external triggering methods, performance could also decrease because of the exposure of the devices to salt air and water.
To solve this problem, a “soft triggering” method was developed using regions of interest (ROIs) in the camera’s FOV. As images are captured continuously at 30 frames/s, each image is inspected for the presence of a scallop and the image processed if a scallop is detected. To be effective, each scallop must only trigger the system once, and the algorithm must execute fast enough to avoid a bottleneck in the overall processing time.
A series of overlapping ROIs was created that would act as separate soft triggers. A simple and fast calculation of values taken at each soft trigger determines if a scallop is in the camera’s FOV. If the value of any trigger drops below a specified threshold, the image is saved for further processing (see Fig. 2). If no trigger value is below the threshold, the image is discarded and the next image is checked. To avoid double-counting, a flag is set for each ROI when it is triggered, and all subsequent images are ignored until the average value in that ROI is above the threshold and the flag is reset. When a scallop successfully triggers the system, the image is passed to the main processing algorithm that is designed to filter out unwanted debris, select the correct scallop to be processed, and calculate the scallop height in millimeters from a measurement of its area (see Fig. 3).
Geometric measurement of shell height is problematic because locating the correct flat edge of a scallop is difficult. Height is measured from the bottom of the shell “hinge” to the furthest point perpendicular from the hinge. To design an algorithm that would replicate this measurement method exactly, the flat hinged edge needed to be detected in a repeatable and robust manner. Since shells may be cracked or broken, flat-edge detection was abandoned in favor of a more robust method that calculated the final height from the scallop shell area. Shell area measurements do not rely on the presence of specific physical characteristics in the shell geometry.
Waddel disks
Shell measurement is a simple task using LabView’s Vision Development Module, which has functions for detecting, filtering, and analyzing particles. LabView’s analysis library calculates the Waddel Disk Diameter (WDD) of the scallop: that is, the diameter of a circle having the same area. Effectively, the shell height was derived by calculating the diameter of a circle having an area equal to the scallop area.
This calculation was tested on a sample set of scallops to see how it correlated to manual measurements. Data showed that smaller scallops were generally underestimated while larger scallops were overestimated. One explanation is the tendency of scallops to grow wider as they become larger. To adjust the final measurement, a correction equation was implemented as a function of the equivalent diameter (WDD)
Scallop height = a (WDD)2 + b (WDD) + c
The roots a, b, and c were derived empirically and implemented into the height calculation. Samples were rerun through the system and plotted against the known heights (see Fig. 4). Accuracy at the prototype stage was determined to be acceptable, although further testing under more rigorous conditions will be necessary. A custom user interface was developed to allow development and production run modes to be easily monitored, including implementing an audible signal mechanism to indicate a successful read (see Fig. 5).
After the system undergoes testing onboard ship, robustness, repeatability, speed, and ease of use will be evaluated to determine additional requirements and upgrades. The most important change may be replacement of the system’s laptop PC with NI’s Compact Vision System to ensure more robust performance.
Steve Tomanovich is president of Compass Technical Consulting, Rochester, NY, USA; www.compasstechconsulting.com.
null
