Natural-language Assisted Human-Multirobot Interaction and Planning (NAMUR)

Multi-UAV systems offer numerous potential applications. However, their use currently requires manual control of each drone or the use of intricate, specialized planning software. We aim to revolutionize human-multi-UAV interaction by developing novel, dynamic planning algorithms and specialized user interfaces that integrate with large language models (LLMs). We are focusing on three research challenges: (i) achieving effective plans through the integration of precise, logic-based multi-robot planning in closed-loop LLM-based reasoning; (ii) exploring user interfaces that enable both low-level and natural-language definition and adaptation of multi-UAV missions; and (iii) establishing robust situational awareness, error-handling, and feedback mechanisms to adapt to changes in the operational environment. We will develop and evaluate prototypes in cooperation with the company Robotto and with internationally recognized researchers in human-computer interaction and disaster robotics.

This is a DFF Project 2 -- funding is equally shared with Syddansk Universitet (SDU) and provides for the hiring of a PhD student at SDU and a postdoc at AAU.

Imagine being able to control a whole team of drones as easily as giving simple instructions. This project aims to revolutionize multi-drone control by developing smarter planning tools and user-friendly interfaces that let you simply speak or type what you want the drones to do, eliminating the need for complicated manual controls or technical software. It combines detailed, logic-based planning with advanced language models—similar to those behind chatbots—so that drones can understand and automatically execute complex tasks, while also allowing users to adjust settings in everyday language. In addition, the project is creating systems that continuously monitor the situation to quickly address errors and adapt to changes in real time. Prototypes will be developed in collaboration with the company Robotto and experts in human-computer interaction and disaster robotics from Virginia Tech in USA and Tohoku University in Japan, with funding shared equally between two universities to support both a new PhD student and a postdoctoral researcher.