Researchers at Brown University (Providence, Rhode Island, USA) and the Technical University of Berlin (Berlin, Germany) have developed computer software that can identify simple abstract sketches of objects as they are being drawn.
It is the first computer application that enables "semantic understanding" of abstract sketches, the researchers say.
Computers are already good at matching sketches to objects as long as the sketches are accurate representations. But iconic or abstract sketches -- the kind that most people are able to easily produce -- are another matter.
James Hays, assistant professor of computer science at Brown, developed the new program with Mathias Eitz and Marc Alexa from the Technical University in Berlin.
The key to making the program work, Hays says, is a large database of sketches. To put the database together, the researchers first came up with a list of 250 everyday objects that people might be inclined to sketch.
Then the researchers used Mechanical Turk, a crowd-sourcing marketplace run by Amazon, to hire people to sketch objects from each category -- 20,000 sketches in all. Those data were then fed into existing recognition and machine learning algorithms to teach the program which sketches belonged to which categories.
From there, the team developed an interface where users input new sketches, and the computer tries to identify them in real time, as quickly as the user draws them.
Today, the program can successfully identify sketches with around 56 percent accuracy, as long as the object is included in one of the 250 categories.
The researchers presented their work at last month's Siggraph. The paper is now available online, together with a video, a library of sample sketches, and other materials.
-- Dave Wilson, Senior Editor, Vision Systems Design
