The presented work proposes a novel system that combines Extended Reality (XR) and Artificial Intelligence (AI) to enhance the teleoperation of lunar rovers. The system consists of three main subsystems: the lunar rover, the ROS PC, and the XR PC.
The first phase involves collecting sensory data from the rover's sensors, including RGB-D information, color images, and depth data. In the second phase, this data is processed using the YOLOv5 CNN algorithm to detect rocks in 2D images, and a 3D mesh of the rover's surroundings is generated based on RTAB-MAP and 3D point cloud environment generation.
The third phase visualizes the processed data in a 3D reconstructed XR environment, providing the operator with a dynamic view of the rover's surroundings, including a 3D model of the rover and marked 3D visual indicators for identified rocks and their positions. This comprehensive approach enhances the operator's ability to discern the positions of rocks proximate to the rover in a lunar environment.
The system was validated through experiments conducted in an analogue lunar laboratory, the LunaLab, at the University of Luxembourg. The findings from the experiments demonstrate the significant impact of the XR system in minimizing the cognitive load of operators and improving their perception of the environment compared to traditional 2D-based teleoperation approaches.
The authors highlight the importance of expanding the range and functionality of the XR system to include enhanced ranging and reporting capabilities in the virtual lunar environment, which will increase the rover's safety when navigating challenging terrain and provide operators with critical data to make informed operational adjustments and optimize exploration routes.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Sofí... at arxiv.org 04-23-2024
https://arxiv.org/pdf/2404.14095.pdfDeeper Inquiries