Researchers at the Army Research Laboratory (ARL) have developed a dialog capability to help transform soldier-robot interactions to perform joint tasks at operational speeds.
The technology called Joint Understanding and Dialog Interface (JUDI) enables bi-directional conversations between soldiers and autonomous systems such as robots or drones.
ARL said that dialog is a critical step in autonomous systems that operate across multiple operations for land, air, sea and information spaces to maintain situational awareness in the field. The technology allows soldiers to interact with autonomous systems through bidirectional speech and dialog in tactical operations where verbal task instructions can be used for command and control of a mobile robot. It also gives the robot the ability to ask for clarification or provide status updates as tasks are completed.
Dialog processing is based on a statistical classification method that interprets a soldier’s intent from their spoken language. The technology was trained on a small dataset of human-robot dialog where human experiments stood in for the robot’s autonomy during initial phases of the research.
"JUDI's ability to leverage natural language will reduce the learning curve for soldiers who will need to control or team with robots, some of which may contribute different capabilities to a mission, like scouting or delivery of supplies," said Matthew Marge, a research scientist at the laboratory.
The goal of the project is to shift the soldier-robot interaction from a heads-down, hands-full joystick operation of robots to a hands-free mode of interaction where the soldier can team with one or more robots while maintaining situational awareness.
JUDI is designed for tasks that require reasoning when data is sparse compared to other systems that only retrieve factual knowledge or perform specialized tasks such as smart assistants like Siri or Alexa. JUDI in contrast allows access to multiple sources such as soldier speech and a robot’s perception system for collaborative decision-making.
The system will be integrated into the Combat Capabilities Development Command (CCDC) ARL autonomy stack, which is a suite of software algorithms, libraries and software components. These components perform specific functions for intelligent systems such as navigation, planning, perception, control and reasoning. Innovations in the ARL stack will be rolled into the CCDC Ground Vehicle System Center’s (GVSC) robotics technology kernel.
"Once ARL develops a new capability that is built into the autonomy software stack, it is spiraled into GVSC's Robotics Technology Kernel where it goes through extensive testing and hardening and is used in programs such as the Combat Vehicle Robotics, or CoVeR, program," said Dr. John Fossaceca, AIMM ERP program manager. "Ultimately, this will end up as Army owned intellectual property that will be shared with industry partners as a common architecture to ensure that Next Generation Combat Vehicles are based on best of breed technologies with modular interfaces."
The next steps include evaluating the robustness of JUDI with a physical mobile robot platform at a field test planned for September.
