Under the Skin

Multispectral stereo camera system uses a five-channel prism to produce 3-D medical images

Richard Meester, Quest Innovations

Although the efficiency of the human visual system is unparalleled in comparison with any piece of optical instrumentation ever invented, human vision has its limitations, especially in regard to the spectral bandwidth of light captured, its capability for multispectral vision, and its ability to perform certain image-processing functions. The human visual system is also incapable of storing accurate images and lacks the ability to perform zoom functions.

Click here to enlarge image

While low-resolution grayscale cameras are sufficient for many machine-vision applications, more specialized, custom-built cameras are often required to meet complex multispectral imaging applications. In these applications, OEM components including sensors, processors, frame grabbers, and motor controllers can be harnessed to develop systems for multispectral applications such as crop inspection and selection, fruit picking, and detection of fungi and diseases on plants. In medical applications, multispectral systems can be used for tasks such as skin and blood flow analysis. These applications also add the complexity of requiring three-dimensional images to be captured and displayed so physicians can make a correct diagnosis.

To develop and produce the next generation of medical visualization devices based on bio-photonics technology, O2view sought to develop an optoelectrical mechanical system that provides the continuously variable magnification of a high-end surgical microscope, changes its field of view, provides automatic stereoscopic correction, combines near-infrared information with visible light, and stores all raw data and performs multispectral calculations as needed.

To accomplish this task, the company enlisted the expertise of Quest Innovations in the development of the multispectral stereoscopic vision system that produces 3-D multispectral images in real time, empowering the surgeon with images that reveal hidden subdermal structures up to 2 or 3 cm under the skin.

Five-channel stereo

In developing the five-channel stereo imaging system, Quest set out to produce a line of multispectral cameras that range from a two-channel RGB and IR camera to a five-channel camera, designed to acquire five optical wavelengths simultaneously. The resulting five-channel camera, the Condor-1000 MS-5 used in the Artemis system, combines custom-made prisms and off-the-shelf board-level cameras and interfaces coupled to Quest’s Architector software development toolkit (see Fig. 1).

FIGURE 1. In developing a five-channel Artemis stereo imaging system, Quest has produced a camera designed to acquire five optical wavelengths simultaneously. The resulting five-channel camera, the Condor-1000 MS-5, combines custom-made prisms, and off-the-shelf board-level cameras and interfaces (left). The spectral response of the camera consists of three color channels and two IR bands (right).Click here to enlarge image

The camera version supplied to O2view uses three standard RGB spectral regions and one IR region at approximately 810 nm and one spectral region at 910 nm. In the design of the camera, a five-channel dichromatic prism is used to selectively split the incoming light beam into separate channels. After separation, light is captured by five 25 × 25-mm Ibis 5 camera boards from Cypress Semiconductor that are aligned to within 1/3 of a pixel.

With a spatial resolution of 1280 × 1024 at 10 bits/pixel and five image sensors, the camera generates more than 150 frames/s in parallel. Because of this, Quest has developed its own Eagle-100, a scalable processing interface board capable of attaching to the Ibis 5 sensor board and to other processing or interface boards via small, flexible cables. Each processing component in the multicomputer system has a separate memory space and memory bus (see Fig. 2).

FIGURE 2. With a spatial resolution of 1280 × 1024 at 10 bits/pixel and five image sensors, the camera generates more than 150 frames/s. Quest has developed its own scalable processing interface board capable of attaching to the sensor board and to other processing or interface boards, making the camera essentially a configurable multicomputer.Click here to enlarge image

Multiple Eagle-100 boards can be attached to each other to allow for a scalable parallel processing camera system with multiprocessor capability. The Eagle-100 can also be attached to either the Camera Link or the fiberoptic interface module. The Eagle-100 contains a 32-bit processor as well as the proprietary Quest image-processing core. A flash program memory as well as SRAM application memory is available on the board to allow for running different programs and storing camera configurations.

Processing this data online is an enormous task. Because of this, each Eagle-100 processor board contains an FPGA, programmable with VHDL that acts as a pixel processor for high-speed image-processing tasks. This allows signal processing to be performed on each individual channel to process images on the fly.

Camera interfaces

Several interfaces are available to allow captured images to be transferred over Camera Link, optical fiber, and Gigabit Ethernet. Because the camera uses a Camera Link output, off-the-shelf Camera Link frame grabbers can be used to transfer images from the camera to the host computer. Similarly, Camera Link to fiber interface converters such as those manufactured by Phrontier Technologies or Camera Link to GigE interfaces such as those manufactured by Pleora Technologies can also be used.

After images are transferred to the PC, a custom software toolkit known as Architector allows developers to program and configure the cameras and onboard processing modules. With this software, developers can design and program an application and stream image data to the PC (see Fig. 3). This SDK not only controls the five-channel camera but is used to develop image-processing algorithms and display and store images. This C++-based software is an application development toolkit built around a processing pipeline architecture that can be used for developing multithreaded image-processing systems. Because Architector’s core library is extensible, new algorithms can be added and third-party cameras supported. All the software used in the Artemis system is based on this SDK.

FIGURE 3. The Architector SDK allows developers to program and configure the cameras and onboard processing modules. Developers can design and program an application and stream image data to the PC.FIGURE 4. The Artemis system allows visualization of five different wavelengths captured by the camera system. Built around two identical optical axes, these can be moved separately under control of the Architector SDK.Click here to enlarge image

O2view’s Artemis system was required to allow medical professionals to visualize the five different wavelengths captured by the camera system. Built around two identical optical axes, these can be moved separately under control of the Architector SDK (see Fig. 4). In operation, the system reveals hidden subdermal structures penetrating up to 2 or 3 cm under the skin. This medical imaging device makes use of the fact that the hemoglobin in blood displays some very specific spectral bands in the wavelength range between 600 and 1000 nm.

FIGURE 4. The Artemis system allows visualization of five different wavelengths captured by the camera system. Built around two identical optical axes, these can be moved separately under control of the Architector SDK.Click here to enlarge image

Based on parameters such as hemoglobin concentration and oxygen saturation, these spectral bands change in intensity. Using an image of a patient’s arm in the visible region and near-infrared (NIR) region, oxygen saturation can be calculated within the veins and the heart as well as the respiration rate of the patient. Because of the stereoscopic acquisition, 3-D measurements can be made of specific features within the image.

Richard Meester is president and CEO of Quest Innovations, Middenmeer, The Netherlands; www.quest-innovations.com.