Kinect-based sign language interpretation

Kinect-based sign language interpretation

What started out as a game has become so much more.

More Vision Articles

How accurate is the Kinect?

Enhanced 3D sensing capabilities in Microsoft Kinect

NASA's JPL researchers operate a robotic arm with Kinect 2 and Oculus Rift

Kinect -based sign language interpretation

Kinect helps researchers steer roaches to explore and map disaster sites

Okay, so by now you are well aware of some of the more unique applications of the Microsoft Kinect. If you aren’t, here are just a few:

The latest innovative use of the Kinect involves researchers from Microsoft Research Asia using the device’s body-tracking abilities to perform sign-language recognition. By tracking the position of the user’s hands, many signs can be detected instantly by using the group’s 3D trajectory matching algorithm. Check out the video here:

The system consists of two modes, the first of which is translation mode, which translates sign language into text or speech. This includes isolated word recognition and sentence recognition. The raising and putting down of hands is defined as the “start” and the “end” of each sign language word to be recognized. The system gives the recognized rank-5 candidate works in the bottom region of the interface, and if the rank-1 word is not correct, the signer can adjust the results by manual interaction.

The second mode is communication mode, in which a non-hearing impaired person can communicate with the signer through an avatar. The impaired person can make an immediate response through signing, and the system translates the answer into text.

Once again, an extremely cool and practical application of the Microsoft Kinect that makes you wonder if the developers of the initial model had any inclination that the product would branch out as it has. Another thing that may make you wonder—I am, anyway—is what courtroom sign language interpreters are thinking of this technology. Probably akin to the way that a farmhand might think of this.

Get All the Vision Systems Design News Delivered to Your Inbox

Subscribe to Vision Systems Design Magazine or email newsletter today at no cost and receive the latest news and information.

 Subscribe Now



How the newest machine vision standards will affect you

How will the latest developments in machine vision standards affect you? This webcast will answer that question by featuring updates on relevant standards by some of the industry’s most knowledgeab...
Date: July 15, 2015

How vision systems are changing automotive manufacturing

How do Ford and General Motors leverage machine vision for powertrain manufacturing? How does traditional vision application development serve today’s needs, and what machine vision capabilities co...
Date: June 24, 2015


PROFINET® and Multi-Language Support Now Available with Microscan Link

October 9, 2013 Microscan Link connectivity provides vision engineers, programmers, and other users with an easy ...

Microscan’s AutoVISION™ 2.0 Wins 2013 Ringier Technology Innovation Award for Food and Beverage

August 8, 2013 Microscan, a global technology leader in barcode, machine vision and lighting solutions, announce...


Click here to view archived Vision Systems Design articles