A Natural Language-enabled Virtual Assistant for Human-Robot Interaction in Industrial Environments

This paper introduces a natural language-enabled virtual assistant (VA), called Max, developed to enhance human-robot interaction (HRI) with industrial robots. Regardless of the numerous natural language interfaces already available for commercial use and social robots, most VAs remain tightly bound to a specific robotic system. Besides, they lack a natural and efficient human-robot communication protocol to advance the user experience and the required robustness for use on the industrial floor. Therefore, the proposed framework is designed based on three key elements. A Client-Server style architecture that provides a centralised solution for managing and controlling various types of robots deployed on the shop floor. A communication protocol inspired by human-human conversation strategies, i.e., lexical-semantic strategy and general diversion strategy, is used to guide Max's response generation. These conversation strategies are embedded in Max's architecture to improve the engagement of the operators during the execution of industrial tasks. Finally, the state-of-the-art pre-trained model, Bidirectional Encoder Representations from Transformers (BERT), is fine-tuned to support a highly accurate prediction of requested intents from the operator and robot services. Multiple experiments were conducted for validating Max's performance in a real industrial environment.