A senior software developer at Beca Applied Technologies (Auckland, NZ) has used the Kinect sensor from Microsoft (Redmond, WA, USA) and the VBS2 simulation environment from Bohemia Interactive Simulations (Prague, Czech Republic) to explore whether the combination could be used to build a system to train personnel in aircraft marshalling.
To limit the scope of the problem, Stephen Witherden chose to focus his efforts on developing a helicopter marshalling system, since the set of helicopter marshalling signals is finite, well defined and well understood.
Witherden created his system to allow an aircraft marshall to land a Kaman SH-2 Seasprite helicopter on the deck of the Her Majesty's New Zealand Ship Canterbury. The Canterbury is a multi-role vessel with a helicopter deck, thus enabling it to support helicopter landing and take-off at sea.
In his research, he discovered that although the Kinect performed well, there were a number of limitations, namely, that the Kinect SDK skeletal tracking algorithm did not appear to perform well when joints cross over each other, and the Kinect sensor could not tell the difference between palms faced up and palms faced down.
In the future, he believes that machine learning techniques could be employed to improve the performance of the pose and gesture detection of the system, resulting in fewer errors.
More details can be found in Stephen Witherden’s paper "Air Marshalling with the Kinect" here.
-- Dave Wilson, Senior Editor, Vision Systems Design