TY - GEN
T1 - Improving grounded natural language understanding through human-robot dialog
AU - Thomason, Jesse
AU - Padmakumar, Aishwarya
AU - Sinapov, Jivko
AU - Walker, Nick
AU - Jiang, Yuqian
AU - Yedidsion, Harel
AU - Hart, Justin
AU - Stone, Peter
AU - Mooney, Raymond J.
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/5/1
Y1 - 2019/5/1
N2 - Natural language understanding for robotics can require substantial domain- and platform-specific engineering. For example, for mobile robots to pick-and-place objects in an environment to satisfy human commands, we can specify the language humans use to issue such commands, and connect concept words like red can to physical object properties. One way to alleviate this engineering for a new domain is to enable robots in human environments to adapt dynamically - continually learning new language constructions and perceptual concepts. In this work, we present an end-to-end pipeline for translating natural language commands to discrete robot actions, and use clarification dialogs to jointly improve language parsing and concept grounding. We train and evaluate this agent in a virtual setting on Amazon Mechanical Turk, and we transfer the learned agent to a physical robot platform to demonstrate it in the real world.
AB - Natural language understanding for robotics can require substantial domain- and platform-specific engineering. For example, for mobile robots to pick-and-place objects in an environment to satisfy human commands, we can specify the language humans use to issue such commands, and connect concept words like red can to physical object properties. One way to alleviate this engineering for a new domain is to enable robots in human environments to adapt dynamically - continually learning new language constructions and perceptual concepts. In this work, we present an end-to-end pipeline for translating natural language commands to discrete robot actions, and use clarification dialogs to jointly improve language parsing and concept grounding. We train and evaluate this agent in a virtual setting on Amazon Mechanical Turk, and we transfer the learned agent to a physical robot platform to demonstrate it in the real world.
UR - http://www.scopus.com/inward/record.url?scp=85071504285&partnerID=8YFLogxK
U2 - 10.1109/ICRA.2019.8794287
DO - 10.1109/ICRA.2019.8794287
M3 - Conference contribution
AN - SCOPUS:85071504285
T3 - Proceedings - IEEE International Conference on Robotics and Automation
SP - 6934
EP - 6941
BT - 2019 International Conference on Robotics and Automation, ICRA 2019
PB - Institute of Electrical and Electronics Engineers
T2 - 2019 International Conference on Robotics and Automation, ICRA 2019
Y2 - 20 May 2019 through 24 May 2019
ER -