A discussion with Stanley Hack, ConsulTech Engineering
Stanley Hack is founder and president of ConsulTech Engineering (Manlius, NY, USA; www.consultechusa.net). Hack has more than 30 years of experience in software, image processing, and machine-vision engineering and management. He has served as an engineering professor, an industrial program manager, and a vice president of engineering and is author of more than 45 technical publications. Editor in chief Conard Holton spoke to him about specialized image-analysis challenges.
VSD: What systems or services does your company provide to developers of machine-vision and image-processing-based systems?
Hack: ConsulTech Engineering is a technology-based consulting and system-integration company. We provide a number of products and services including, but not limited to, short-term consulting, developing algorithms and software, and developing, installing, and supporting complete machine-vision systems. The emphasis is on developing algorithm and software solutions for difficult and specialized image-analysis and data-handling problems.
These software solutions are delivered to our customers using three business models. In the first model, we integrate the developed software into custom machine-vision systems that the company designs and builds to its customers’ specifications. In the second model, we deliver custom software modules to our customers, who then integrate the modules directly into their products. In the third model, we design and develop software to a customer’s specifications and then integrate it into the customer’s products. The second and third models allow us to offer our customers a level of machine-vision and image-processing expertise that they may not have in-house for their product development.
Our clients are from many fields. Their applications include x-ray imaging, ultrasonic imaging, microscopy analysis, semiconductor inspection, manufacturing quality assurance, train rail inspection, printed circuit board inspection, web inspection, and general metrology inspection.
VSD: Over the past ten years there have been radical changes in the types of hardware and software that are available to system integrators. What major differences do you see?
Hack: Solutions to most of the hardware problems encountered can be solved by judicious use of off-the-shelf products. This is much different than ten years ago when many hardware problems required custom solutions. From our business perspective, the most significant hardware advances have been the tremendous increases in computer processing power, computer storage capacity, camera performance, frame-grabber performance, and illumination systems’ capabilities.
The advances in computer processing power and storage capacity allow us to implement very complex algorithms to solve complex problems using standard PC platforms. Today, we are rarely limited by processing performance issues. When those issues arise, we are often able to find solutions using off-the-shelf multi-processing and closely coupled networking products. These off-the-shelf solutions greatly reduce the costs to our customers by eliminating the need to develop custom hardware.
Advances in camera resolution, field-of-view size, performance, interface standardization, frame-grabber performance, and solid-state illumination systems have simplified the integration tasks and reduced the risks. These risks, which were omnipresent ten years ago, are introduced when components are integrated in such a way as to produce system performance that may exceed the performance of individual components. For example, if a system requires a field of view of 2048 × 1024 pixels but cameras with fields of view of only 1024 × 1024 pixels are available, the system designer might incorporate two cameras instead of one. The two-camera solution poses risks of intercamera misalignment and performance mismatches. The use of a modern 2048 × 2048-pixel camera completely eliminates these risks.
The advent of standard camera interfaces such as Camera Link, Ethernet, Gigabit Ethernet, IEEE 1394, and USB along with compatible cameras and frame grabbers has reduced the camera-interface problem nearly to “plug and play.” These standards eliminate the need to custom engineer camera interfaces for each system. When standard interfaces are substituted for custom interfaces, system costs decrease and the risks of not meeting system specifications are reduced. This risk reduction is due to the extensive verification and validation that has been made to the standard interfaces in multiple environments.
Many of the newer solid-state illumination systems are physically small, amenable to dynamic control, and easy to mount. Advances in reducing package sizes and power requirements of cameras and illumination systems have reduced the system layout complexity, especially in those cases where environmental issues are present. Because of the advances made in hardware component performance, standardization, size, and power requirements, today’s integrated machine-vision systems include more functionality and performance, are more cost-effective, are more reliable and maintainable, and require smaller footprints and less power.
VSD: What software libraries do you use and why?
Hack: We maintain and continuously expand our ImageTech library of image-processing software. For our specialized software and integration applications, we have found it to be more cost-effective than for our clients to use applicable parts of our in-house library than to integrate commercial libraries. However, our expertise is in image-processing and machine-vision algorithms and software. We find it more cost-effective to use third-party libraries for low-level mathematical functions and numerical methods.
These libraries are generally extensive, well documented, and rigorously validated. Also, we make use of hardware vendors’ expertise in developing software drivers for their products. It is far more cost-effective for us to integrate their drivers into our software than for us to develop new drivers. ConsulTech Engineering is able to provide software products for its customers using this combination of in-house, third-party, and hardware vendor libraries that maximize value and performance.
VSD: What types of cameras, frame grabbers, CPUs, and network interfaces do you use in these applications?
Hack: We select components for integrated systems based primarily on component capability, performance, and price. However, other factors are often involved in the selection process. Customers sometime dictate which hardware components are to be used in a system. This often occurs when we are developing upgrades to existing systems. In other cases, work is brought to us by hardware vendors or their sales representatives. In these cases, we do our best to use hardware components manufactured by these vendors if they fit into the systems’ designs. We only use components whose performance meets or exceeds the systems’ specifications. There have been applications in which we could not and did not use components requested by our clients due to functional or performance issues.
VSD: In which areas do you see the most growth?
Hack: We see three areas of growth. The first is the level of integration of machine-vision systems into the manufacturing process. For example, the results of in-line inspection of the honed edges of scalpel blades are used to directly adjust the honing process.
The second area is extending machine vision to take measurements of features that only partially exist in the digitized image. An example is the scalpel blade. The inspection process includes calculating the depth of the honed area. The digitized image of a scalpel blade shows discrete honed marks that are perpendicular to the blade’s edge and cover only a small percentage of the honed area. It is necessary to estimate the extent of the honed area from this small number of sampled points. We are seeing more and more applications that require data extrapolation and estimation processing similar to this example.
The third area of growth is in making true 3-D surface measurements. For example, 3-D measurements give us the capability to precisely calibrate the pixel dimensions of an image throughout its depth of field. As machine-vision systems are being developed to make more accurate measurements, 3-D imaging will become an integral part of the system to help produce measurements of the highest accuracy and precision.
VSD: How will hardware and software targeted toward machine-vision applications have to change to meet future needs?
Hack: We see two areas of change. The first is a continuation of increasing computer processing power, affordable mass storage capacity, camera image resolution, camera image field-of-view size, imaging speed, and illumination power. The other area of change is in sensor fusion. If one considers the three-dimensional measurement problem, fusing an area camera with a laser-based non-contact measurement system could provide a high precision 3-D surface map of an object. Currently, the performance and resolution of each of these two types of sensors is very different. We see a convergence of these capabilities and true fusion of the various sensors’ visualization and measurement capabilities as the wave of the future in machine vision.
An application that would benefit from sensor fusion is borescope measurement. Unless the sensor-to-object distance is known, the size of each pixel acquired by a borescope is not known in real-world units. The use of telecentric optics is one solution to this problem. Another solution is to use stereoscopic imaging. However, the fusion of different sensors, each optimized for a specific task, would add tremendous capability to the borescope measurement application.
VSD: What new applications do you expect to emerge? What market changes are driving these applications?
Hack: There will be expansive growth in machine-vision systems used for security applications. These applications require advanced sensor fusion, feature extraction, feature recognition, and situational awareness capabilities.
Situational awareness capabilities are fairly new to the machine-vision community. These capabilities give a system, or a system of systems, the ability to predict behavior in a scene based on sequences of recognitions and measurements taken over time. For example, a security system used for border protection might predict an event of interest based on traffic patterns in an area several miles from the border. The technology developed for security applications will easily be transferred to other applications. For example, situational awareness and behavior prediction will find great applicability in traffic monitoring of our highways, our runways, and our airspace.
