Acquired Electronics360

Emerging Display Technologies

DARPA Wants to Develop Technology Allowing for a New Level of Human-Computer Interaction

30 October 2015

The Defense Advanced Research Projects Agency (DARPA) has just awarded $2.1 million to Colorado State University (CSU) researchers to develop gesture technology that would allow for humans to interact with computers in a different way than ever before.

Typical computer interactions include typing, clicking and scrolling, but DARPA wants to create a level of communication that would make computers smart enough to reliably recognize non-verbal cues from humans in the most natural way possible.

DARPA Wants to Develop Technology Allowing for a New Level of Human-Computer InteractionDARPA Wants to Develop Technology Allowing for a New Level of Human-Computer Interaction

The CSU project, titled “Communication Through Gestures, Expression and Shared Perception”, is being led by Professor of Computer Science Bruce Draper. It falls under DARPA’s “Communicating with Computers” funding program.

“Current human-computer interfaces are still severely limited,” says Draper, who is also joined by CSU researchers from the computer science and mathematics departments. “First, they provide essentially one-way communication; users tell the computer what to do. This was fine when computers were crude tools, but more and more, computers are becoming our partners and assistants in complex tasks. Communication with computers needs to become a two-way dialogue.”

To achieve its goal, the team has proposed creating a library of elementary composable ideas (ECIs), which can be described as little packets of information recognizable to computers. Each ECI contains information about a gesture or facial expression, derived from human users, as well as a syntactical element that constrains how the information can be read.

The researchers have set up a Microsoft Kinect interface to test out different methods. A human subject sits down at a table with blocks, pictures and other stimuli. The researchers try to communicate with and record the person’s natural gestures for concepts like “stop,” or, “huh?”

“We don’t want to say what gestures you should use,” Draper explains. “We want people to come in and tell us what gestures are natural. Then, we take those gestures and say, ‘OK, if that’s a natural gesture, how do we recognize it in real time, and what are its semantics? What roles does it play in the conversation? When do you use it? When do you not use it?’”

According to the project proposal, the work could someday allow people to communicate more easily with computers in noisy settings, or when a person is deaf or hard of hearing, or speaks another language and could revolutionize computer-human interaction by getting the computer to recognize non-verbal cues such as gestures, body language and facial expressions.

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter


Date Event Location
18-22 Jun 2018 Honolulu, Hawaii
12-16 Aug 2018 Vancouver, Canada
11-13 Sep 2018 Novi, Michigan
27 Sep 2018 The Reef, Los Angeles
26 Oct 2018 Old Billingsgate
Find Free Electronics Datasheets