Multi-camera system helps Disney create panoramic video
To correct for parallax in video panoramas, Disney Research (Zurich, Switzerland; www.disneyresearch.com) approached NorPix (Montreal, QC, Canada; www.norpix.com) to create a system that could capture images with multiple cameras at 60fps.
To create such a panorama, the Disney team needed to stitch together multiple views from a camera array which had overlapping visual fields, but were not precisely positioned. Often when multiple images are stitched together, the video is blurred and distorted. Additionally, parallax errors and any image warping needed to be corrected.
In the system, sixteen different cameras were used and NorPix created a system for multi-camera recording using its SmallPix 2 portable computers and StreamPix software. Each camera was tethered to one of the computers and all nine computers were controlled through StreamPix remote software installed on a remote computer on a LAN.
The cameras used for the system were Lt425 cameras from Lumenera (Ottawa, ON, Canada; www.lumenera.com.) These USB 3.0 cameras feature a 4MPixel CMV4000 CMOS image sensor from CMOSIS (Antwerp, Belgium; www.cmosis.com) with a 5.5μm pixel size. With the workstation, an operator can observe live sample feeds from each of the cameras.
Uncompressed images from the cameras were transferred to solid state drives at a transfer rate of more than 240MBps or 1.92Gbps, with the cameras delivering these images at frame rates up to 90fps at full resolution. Disney’s research team noted that the panoramas were captured without particularly accurate placement of the cameras to demonstrate the flexibility of their approach.
Robots teach themselves how to grasp objects
Researchers from Brown University (Providence, RI, USA; www.brown.edu) have developed a technique which teaches automation robots how to pick up objects so they can relay the information to other robots. Stefanie Tellex, an assistant professor along with graduate student John Oberlin, have used two Baxter industrial robots from Rethink Robotics (Boston, MA, USA; www.rethinkrobotics.com) for their research.
Baxter is a dual-armed collaborative robot with three cameras and a Microsoft (Redmond, WA, USA; www.microsoft.com) Kinect IR sensor that is employed to detect whether a human operator is within a 1.5-15in range of a machine. Each arm of the robot has a standard gripper and a camera to allow the robot to see what it is grasping.
In an MIT (Cambridge, MA, USA; www.mit.edu) Technology Review article, Tellex said that robotics researchers are looking for more efficient ways of training robots to perform tasks such as manipulation.
"We have deep learning algorithms that can learn from large data sets, but these algorithms require data," she said. "Robot practice is a way to acquire the data that a robot needs for learning to robustly manipulate objects."
She also noted that there are 300-some odd Baxter robots currently in use in research labs throughout the world, and that if each of these robots were to use both arms to examine new objects, it would be possible for them to learn to grasp a million objects in 11 days.
"By having robots share what they’ve learned, it’s possible to increase the speed of data collection by orders of magnitude," she said.
To grasp an object, the Baxter robot scans it from various angles using one of its arm cameras and the IR sensors on its body, which allows it to identify possible locations at which to grasp. Tellex and Oberlin used a technique to optimize the process of practicing different grips so that the robot reliably picked up objects.
From there, the information acquired for each object, including images, 3D scans and the correct grip, is encoded in a format that allows it to be shared online and with the other robot. The ultimate goal, Tellex says, is to give robots similar abilities.
High-speed cameras reveal how bats land upside down
To roost upside down on cave ceilings or tree limbs, bats use their wings to perform an acrobatic maneuver that allows them to change direction quickly. To determine how this is possible, a team of researchers from Brown University (Providence, RI, USA; www.brown.edu) have used a special flight enclosure, high-speed cameras and computer modeling to study bats in motion.
"Bats land in a unique way," said Sharon Swartz, a biologist at Brown University. "They have to go from flying with their heads forward to executing an acrobatic maneuver that puts them head down and feet up. No other flying animal lands the same way as bats."
How these bats determine the necessary motion and forces to perform such maneuvers had not been obvious. When they come in to land, they are not moving quickly which makes it difficult to generate the aerodynamic forces needed to reorient themselves. The question to be answered is how the bats position themselves to land.
In their experiments, short-tailed bats and dog-faced fruit bats were studied. The bats were encouraged to land repeatedly in a single location in view of three high-speed cameras by covering the ceiling and walls of a flight corridor with plastic sheets and heavy-duty paper, except for small, square landing pads fabricated from white mesh. The cameras used were FASTCAM 1024 PCI cameras from Photron (San Diego, CA, USA; www.photron.com) which feature a 1024x1024 color CMOS image sensor with a 17μm pixel size that can run at frame rates up to 109,500fps.
Through training, the bats regularly landed on the landing pad, but to elicit "failed landing" maneuvers, the team modified the landing experiment by removing the pads. When they did this, bats would attempt to grasp the missing landing pad and on failure would begin to fall, quickly reorient and fly out of the test area. The cameras - equipped with 50 mm lenses - were placed orthogonally and used to record wing and body kinematics of the bats. All the images were recorded at 1,000fps, which allowed the team to acquire between 500-1,500 frames per maneuver, or approximately 125-175frames per wing beat. The scene was lit using diffused white light and to increase the lighting and contrast, the team covered the walls across from each camera with heavy-duty white paper.
3D wing and body kinematics of the bats were reconstructed from the high-speed video using a model-based tracking algorithm. This tracking system incorporates a Kalman filter and a priori biomechanical constraints to recover the complex motions of the flight maneuvers. With this method, the team extracted six degrees of freedom describing a bat’s body position and orientation, 23 joint angles describing the articulated pose of each of the bat’s wings, and 38 parameters describing the bat’s geometry i.e. - shoulder and hip positions, body dimensions, and bone lengths.
With these methods, the team found that when the bats approach their landing spot, they retract one of their wings slightly toward their bodies, while flipping the other at full extension. With each wing beat in that asymmetric configuration, the bats rotate a half turn, helping to put them in position to meet the mesh feet first. When the team removed the mesh, the bats performed a similar rolling maneuver using their wings to reorient for forward flight.
A computer simulation was then used to confirm that the effect they were observing was due to inertia rather than aerodynamics. The team used motion capture to record the bats’ movements and then replayed the movements through a computer simulation in which the effects of different forces could be switched on and off. When the simulation was run with aerodynamic forces turned off, the virtual bats could still recreate the motion of the real ones.
While the research helps to shed light on the biology that helps bats fly and land, the team suggests that it may be useful in the development of human-made flying machines. The full findings of this research were published in PLOS Biology in November 2015 (http://bit.ly/1PrKqJf).
Autonomous robot monitors inventory in grocery store aisles
Simbe Robotics (San Francisco, CA, USA; www.simberobotics.com) has introduced Tally - an autonomous vision-guided robot that navigates the aisles of grocery stores or other brick and mortar retail environments for purposes of shelf auditing. The system is designed to capture, report, and analyze the state and availability of merchandise and help ensure compliance with the store’s planogram - the ideal placement of products on shelves to maximize sales.
Tally is equipped with a number of sensors that enables it to operate on its own while capturing information on the state of merchandise in the store. The data captured by the robot is sent securely to the cloud for processing and analysis.
"In the retail industry, shopper experience is everything. If a product is unavailable, the retailer has missed an opportunity and disappointed their customer," says Brad Bogolea, CEO and Co-founder of Simbe Robotics. "Tally helps retailers address these challenges by providing more precise and timely analysis of the state of in-store merchandise and freeing up staff to focus on customer service."
In addition to its sensor system, Tally is equipped with a charging dock that it can autonomously navigate to allow for continuous operation. The robot stands 38in tall and weighs about 30lbs and is currently in trial with several North American retailers.
