Andrew Wilson, Editor, [email protected]
In machine vision security-and-defense applications, providing region-of-interest (ROI) processing within a single image provides numerous benefits. Using different-sized regions within a single image can allow different image-processing operations to be performed on each window, speeding the throughput of inspection tasks. Subsampling pixels within image sensor arrays also allows faster frame rates to be achieved while bandwidth requirements remain fixed.
A number of methods exist that allow visible imagers to provide ROI processing. CCDs offer ROI processing through partial scanning, discarding vertical parts of the image to increase bandwidth. Because CMOS imagers convert charge to voltage at every pixel, they do not suffer from this limitation, allowing both vertical and horizontal windowing of multiple ROIs, often at different clock rates. Companies such as Kamera Werk Dresden (Dresden, Germany; www.kamera-werk-dresden.de) have used this feature to permit different regions of a digital camera frame to be captured simultaneously and the exposure time, position of the ROIs, and trigger rates to be independently controlled (see Vision Systems Design, February 2005, p. 16).
Unlike monolithic CMOS imagers, it is more complex to fabricate focal-plane arrays (FPAs) whose CMOS readout multiplexers are flip-chip bonded to exotic detector materials such as InSb, InGaAs, and HgCdTe used in infrared (IR) sensing applications. In cameras based on such standard devices, signals from each detector site on the FPA are read individually using a readout integrated circuit (ROIC).
While many such ROICs read out pixel values of each element in the array or just pixel values in the ROI to achieve higher frame rates, the patented variable acuity superpixel imager (VASI) technology developed by Nova Sensors (Solvang, CA, USA; www.novasensors.com) permits simultaneous multiple ROI processing. Demonstrated at the recent SPIE Defense and Security Symposium (April 2006; Orlando, FL, USA), VASI technology allows multiple windows within an IR image to be sampled at high spatial resolution, while the remainder of the field of view is sampled at lower resolutions. “This sensor,” says Christopher Baxter, vice president of engineering, “provides high spatial resolution where required, maintains situational awareness in the rest of the FOV at lower resolution, while performing at kilohertz frame rates depending on the FPA’s spatial configuration.”
To achieve this functionality, Nova Sensors has developed a CMOS ROIC that can be flip-chip bonded to a number of different visible and IR detector arrays. In the ROIC, an array of programmable switches in each unit cell can be selectively programmed to create a spatial configuration of foveae (high-resolution areas) and “superpixels” (lower resolution areas; see figure).
“In effect, the combined integration capacitance for a given super-pixel is essentially the parallel capacitance of all the connected unit cells,” says Baxter. “The combined photocharge produces a voltage that represents the average signal value of a superpixel region. To achieve a high frame rate, essentially only one value per superpixel along with each fovea pixel is multiplexed off-chip every frame. The readout is dual mode in that it is capable of operating in both full resolution, like a standard FPA, or a variable acuity spatial configuration. The FPA’s spatial configuration can be changed at the frame rate.”
“Unlike some ROI processing methods where pixel data is discarded to achieve faster data rates, the super-pixel configuration of the VASI design integrates the charge from all of the pixels in the array. This allows IR-based systems to simultaneously read out both high-resolution and lower resolution regions from the same device,” says Baxter.
To demonstrate the capability of this advanced sensor technology, Nova Sensors has developed a collaborative relationship with a strategic partner to provide InSb detector material and hybridization services resulting in both large-format 1024 × 1024 and medium-format 320 × 256 mid-wavelength infrared (MWIR) FPAs. Nova’s MWIR camera, VASI-Cam 1K-ML, exploits the capability of the large-format variable-acuity FPA. It allows the operator to configure the camera into either a standard full-resolution mode operating at 30 frames/s or change the spatial configuration on-the-fly by placing numerous variable-sized foveae anywhere in the FOV and specifying the background superpixel size. High-resolution foveal regions in this figure are highlighted with blue bounding boxes; the remaining portion of the image is “massively parallel” superpixelated on the FPA, thus reducing the bandwidth required to transmit or store the image.
The VASI FPA imager also operates as a “standard” full-resolution 1k × 1k FPA. The on-chip foveation demonstrated provides an on-FPA compression feature that can represent the essential image content of the full resolution image with more than 90% compression, based on the instantaneous spatial configuration of the device.
A peak-intensity autotrack function is optional. This feature provides the user the ability to track up to four peak intensity targets in an autonomous manner by placing foveae over targets with reduced resolution in the background. “Future VASI camera designs,” says Baxter, “will use other detector materials to meet the need for foveal vision in other spectral bands.”
