Two-Stage Reinforcement Learning for Planetary Rover Navigation: Reducing the Reality Gap with Offline Noisy Data

Anton Bjørndahl Mortensen, Emil Tribler Pedersen, Laia Vives Benedicto, Lionel Burg, Mads Rossen Madsen, Simon Bøgh

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

Abstract

We address the challenge of enhancing navigation autonomy for planetary space rovers using reinforcement learning (RL). The ambition of future space missions necessitates advanced autonomous navigation capabilities for rovers to meet mission objectives. RL’s potential in robotic autonomy is evident, but its reliance on simulations poses a challenge. Transferring policies to real-world scenarios often encounters the "reality gap", disrupting the transition from virtual to physical environments. The reality gap is exacerbated in the context of mapless navigation on Mars and Moon-like terrains, where unpredictable terrains and environmental factors play a significant role. Effective navigation requires a method attuned to these complexities and real-world data noise. We introduce a novel two-stage RL approach using offline noisy data. Our approach employs a teacher-student policy learning paradigm, inspired by the "learning by cheating" method. The teacher policy is trained in simulation. Subsequently, the student policy is trained on noisy data, aiming to mimic the teacher’s behaviors while being more robust to real-world uncertainties. Our policies are transferred to a custom-designed rover for real-world testing. Comparative analyses between the teacher and student policies reveal that our approach offers improved behavioral performance, heightened noise resilience, and more effective sim-to-real transfer.Videos, simulation environment, source code, and datasets: https://github.com/abmoRobotics/isaac_rover_2.0, https://github.com/abmoRobotics/isaac_rover_2.0_learning_by_cheating
Original languageEnglish
Title of host publication2024 International Conference on Space Robotics (iSpaRo)
Number of pages7
PublisherIEEE (Institute of Electrical and Electronics Engineers)
Publication date2024
Pages266-272
ISBN (Electronic)979-8-3503-6723-2
DOIs
Publication statusPublished - 2024

Keywords

  • Reinforcement Learning
  • Space Robotics
  • Navigation
  • planetary exploration
  • Mars
  • Moon
  • space rover navigation
  • Safe

Fingerprint

Dive into the research topics of 'Two-Stage Reinforcement Learning for Planetary Rover Navigation: Reducing the Reality Gap with Offline Noisy Data'. Together they form a unique fingerprint.

Cite this