By Andrew Wilson, Editor
One method for designing a color camera is to use a combination of prisms or filters coupled with three CCDs. These tri-CCD cameras can produce high-quality color images at high speed. However, they are more costly since they use multiple CCDs and associated optics. Although these cameras can produce high-resolution images, they can suffer from effects such as prism fringing.
As a cost-effective alternative, many manufacturers produce cameras that coat the CCD elements of their area arrays with a Bayer filter, named after Bryce E. Bayer, a scientist at Eastman Kodak who patented the concept in 1976. In Bayer-based color imagers, pixels on the image sensor are covered with a mosaic of red, green, and blue transmissive filters. In the Bayer pattern, 50% of the pixels are green, 25% are red, and 25% are blue (see image).
Bayer noticed that the human eye obtains most of its sharpness information from green light, which is why the pattern has more green pixels. By using two green-filtered pixels for every red or blue, the Bayer pattern is designed to maximize the perceived sharpness in the luminance channel, composed mostly of green data. However, since the image is undersampled, the full detail available is not attained.
Different CCDs
While all manufacturers produce identical Bayer color filter arrays, there are slight differences between the CCDs produced by different manufacturers. The first line is generally R and G, but devices from Kodak start with a G and those from Sony with an R. To create a full-color image, interpolation generates color values for each pixel, creating a full three-channel RGB color image.
Some manufacturers provide RGB data output by performing Bayer interpolation at the camera source. Examples include both the TMC-1000CL and TMC-6700CL cameras from JAI Pulnix. In other cameras, raw Bayer data must be converted to RGB data using interpolation methods. Numerous methods exist to perform this, including nearest-neighbor replication, bilinear interpolation, and smooth-hue transition interpolation. Most frame-grabber manufacturers and independent image-processing-software suppliers provide color-interpolation software specifically designed for this purpose (see table on p. D12). In choosing the correct type of software, systems developers should understand the trade-offs between the interpolation types.
In its BayView software, for example, BitFlow offers nearest-neighbor-replication, bilinear, and smooth-hue-transition interpolation methods. “Before any interpolation can take place,” says Reynold Dodson, president of BitFlow, “the software must be set for the specific Bayer matrix pattern and the way it is read from the camera.” After this, the systems developer can choose from a number of interpolation methods, each with different trade-offs.
“Although nearest-neighbor interpolation is the fastest way to produce color data, it produces lower-quality images than both bilinear interpolation and smooth-hue transition,” says Dodson. As one of the mostsophisticated algorithms, smooth-hue- transition interpolation produces the highest- quality images at the expense of computational power. A description of these different methods can be found on the Web: www-ise.
To understand the importance of hardware-assisted Bayer interpolation, BitFlow benchmarked the bilinear interpolation algorithm using its R64-CL frame grabber and host PC-based BayView application and again on its R64-CL-IP2 frame grabber with FPGA-assisted interpolation. For the benchmark, the 1600 × 1200 Dalstar 2M30 camera from Dalsa was used. Running a 3.02-GHz dual-Xenon PC with the R64-CL, the PC could capture images from the frame grabber, perform the color interpolation, and display RGB images at 30 frames/s.
“However,” says Dodson, “because 100% of the processing power was used for this task, the PC could only capture color data streams and could not process the images in real time. Using BitFlow’s R64-CL-IP2 Camera Link frame grabber with FPGA-enabled Bayer interpolation, 30-frames/s color images could also be captured. But, because the interpolation was being performed by the on-board FPGA, the CPU usage was reduced to just 10%, which is expected because the CPU is being used to display the 1600 × 1200 RGB images.
Third capture method
In the past, tri-CCD and Bayer-based cameras were the only devices to capture color images. Now, however, recent developments by Foveon have added a third method that uses a direct image sensor to capture red, green, and blue light at each point in an image during a single exposure. The company has introduced a sensor evaluation kit (SEK) to illustrate how to use a Foveon X3 direct image sensor to capture image data and prepare data for postprocessing.
The Foveon X3 sensor enables capture of RGB image data at every pixel location. Image-sensor output comprises three full layers of RGB image data, eliminating the need for image-reconstruction schemes such as Bayer interpolation. According to BitFlow’s Dodson, images produced by the camera do not suffer from the inherent problems of images derived from Bayer sensors.
Recently, Foveon announced that its F19 sensor-a 1/1.8-in., 4.5-Mpixel CMOS direct image sensor that incorporates Foveon’s X3 technology-has been incorporated into the HanVision HVDUO-5M Camera Link camera. Alternative Vision has announced that it has produced a development platform for camera design with the sensor.
Company Info
Alternative Vision, Los Gatos, CA, USA www.alt-vision.com
BitFlow, Woburn, MA, USA www.bitflow.com
Dalsa, Waterloo, ON, Canada www.dalsa.com
Eastman Kodak, Rochester, NY, USA www.kodak.com
Foveon, Santa Clara, CA, USA www.foveon.com
HanVision, Daejeon, Republic of Korea www.hanvision.com
JAI Pulnix, Sunnyvale, CA, USA www.jaipulnix.com
Sony, Park Ridge, NJ, USA www.sony.com
