Multi-modal perception

insight - Multi-modal perception

Robust Multi-Modal Semantic Segmentation with Modality-Incomplete Scenarios

A comprehensive task, Modality-Incomplete Scene Segmentation (MISS), is studied to cover both system-level modality missing and sensor-level modality errors in multi-modal semantic segmentation. A Missing-aware Modal Switch (MMS) training strategy and a Fourier Prompt Tuning (FPT) method are proposed to address these challenges, enabling efficient and robust multi-modal perception.

Sigma: A Siamese Mamba Network for Efficient and Accurate Multi-Modal Semantic Segmentation

Sigma, a Siamese Mamba network, effectively fuses information from multiple modalities like RGB, thermal, and depth to achieve superior performance in semantic segmentation tasks, while maintaining high computational efficiency.

Robust Multi-Modal 3D Object Detection with Uniform BEV Encoders

UniBEV, a multi-modal 3D object detection framework, is designed to be robust against missing sensor modalities by using uniform BEV encoders and a fusion module that can handle varying input combinations.

About

Products | Resources

Insights