By Ron Buck
RON BUCK is president and CEO of Tyzx, Menlo Park, CA, USA; [email protected].
Electroland, an artistic group based in Los Angles, CA, USA (electroland.net), that creates large-scale public art projects and electronic installations, was selected as one of 87 companies to participate in the 2006 National Design Triennial at the Smithsonian Institution Cooper-Hewitt National Design Museum in New York City. Electroland worked with 3-D vision technology to create LUMEN, a site-specific interactive installation that achieves sensing and visitor tracking throughout the Cooper-Hewitt Museum's main stairwell. LUMEN is a 70-ft-long translucent light box that immerses visitors in sound and light as they ascend and descend the stairs. A single light avatar follows each person, triggering various patterns at different points of passage and through interactive play with the light box.
The LUMEN 3-D network can individually track up to 30 visitors accurately with a very low error rate. This level of individual tracking allows for great precision (down to 4 cm) in translating individual motions. Person tracking with a 3-D stereo camera can provide the location and movement of each individual precisely, even in a crowded environment under changing lighting conditions. This 3-D technology allows people to experience the artwork simply by virtue of their presence and movements, rather than through any special hardware interface.
A single 3-D stereo camera compares an image pair, using stereocorrelation. The resulting range image is a fine-grained representation of the distance to and size of each pixel in the scene. In contrast, traditional multicamera tracking systems use discrete image features identified by conventional 2-D methods in monocular cameras, with subsequent triangulation. With the second approach, the subject must be in the view of two or more cameras, and the cameras do not have the benefits with respect to lighting, occlusions, and size information. The cameras must also remain well-registered to each another across a space. This is not required with a 3-D stereo camera because the imager's extrinsic parameters are fixed when the camera is built.
With the 3-D data produced by a stereo camera, accurate information on the subject's size and precise 3-D location with respect to a world reference frame is obtained, enabling 3-D person tracking. In operation, the 3-D range and intensity images are first used in a background model to filter the incoming data and identify pixels that are in motion, that is, those that do not match to the established background model. The resulting data, which are expressed as a binary foreground mask, are next passed to a postprocessing step called ProjectionSpace, which consolidates the massive amount of incoming 3-D data and presents the information to the end application in a more directly usable fashion.
The orientation and position of the camera computed during registration transform each 3-D foreground pixel coordinate into a coordinate system aligned with the floor and other cameras. Objects with a high surface area perpendicular to the floor stand out, and people are especially easy to track because they are easily distinguished from the floor and other backgrounds and objects. In a 3-D person tracking network, the segmented and tracked results from many tracking cameras are sent to a single PC system to produce a final tracking map, which can be used to clearly identify and follow all the individuals in a particular room or other given space.
Installing the LUMEN exhibit presented many challenges. Electroland was limited by the nature of the museum space, which had once been Andrew Carnegie's private mansion (the walls are still covered by walnut paneling). Electroland engineered a light-box structure that would line the walls of the staircase without any hard attachments that might harm the wood paneling. The entire project had to be freestanding with only one or two pressure points. Electroland designed a cage made of high-grade aluminum and acrylic and lined with fluorescent panels, which was fabricated in three 20-ft long sections.
Electroland used computer-controlled fluorescent lights as an interactive technology. Sylvania/OSRAM provided technical and material support including the company's T5 fluorescent light fixtures. The designers were able to use fewer of these lights because the T5s project extremely high light output compared with other potential sources such as LEDs. These fixtures do not have the flickering problems traditionally associated with fluorescent lights. Their brightness level can be controlled just like theatrical LEDs and can be dimmed down as far as 1% of output to create multiple light patterns.
Electroland used a distributed person-tracking system from TYZX (Menlo Park, CA, USA www.tyzx.com) that has two networked cameras. To protect the wood paneling in the stairwell, the 3-D cameras were attached to existing lighting features, one mounted high on the ceiling, looking down into the space, and another over the railing. These 3-D cameras were originally calibrated to work within a plane or flat environment, so the 20-ft drop of the staircase presented an interesting challenge for the TYZX technology. Second, the group could not change the conventional dark ambient lighting on the stairs, and the fluorescent lights that followed individuals up and down the stairs created a particularly challenging environment because of the high contrast between the lights and the dark paneling and because the lighting was in constant flux. TYZX worked with Electroland, adjusting the cameras to track individuals accurately, despite the challenging installation space, and customizing autoexposure and gain-setting on the cameras to accommodate the high-contrast environment.
Electroland's Cameron McNall and partner Damon Seeley wrote a customized java application to run the LUMEN installation on a standard Windows-based computer. The Java application receives data from the TYZX tracking system and calculates assumptions about the location of various individuals relative to the staircase. The application then reconciles the tracking data with the real-world situation of the stairwell lighting and assigns "light personalities," or avatars, to track each individual as they move up and down the stairs. The application creates special zones that can trigger ambient effects such as slowly pulsing lights or racing patterns. If an individual stays longer in a particular space, their influence over the light skin grows, for instance, creating a larger, more persistent area of light.
Electroland is now working on a project using an eight- to 12-camera array for the new Indianapolis airport