In this study, we introduce an efficient and robust LiDAR-camera 3D object detector, referred to as MoME, which can achieve robust performance through a mixture of experts approach. Our MoME fully decouples modality dependencies using three parallel expert decoders, which use camera features, LiDAR features, or a combination of both to decode object queries, respectively. We propose Multi-Expert Decoding (MED) framework, where each query is decoded selectively using one of three expert decoders. MoME utilizes an Adaptive Query Router (AQR) to select the most appropriate expert decoder for each query based on the quality of camera and LiDAR features. This ensures that each query is processed by the best-suited expert, resulting in robust performance across diverse sensor failure scenarios. We evaluated the performance of MoME on the nuScenes-R benchmark. Our MoME achieved state-of-the-art performance in extreme weather and sensor failure conditions, significantly outperforming the existing models across various sensor failure scenarios.

We Evaluate MoME on nuScenes-R and nuScenes-C

• Installation
• Prepare Dataset
• Train and Eval

MoME is based on MEFormer. It is also greatly inspired by the following outstanding contributions to the open-source community: mmdetection3d, CMT.

If you find MoME is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{MoME,
  title={Resilient Sensor Fusion under Adverse Sensor Failures via Multi-Modal Expert Fusion},
  author={Park, Konyul and Kim, Yecheol and Kim, Daehun and Choi, Jun Won},
  journal={arXiv preprint arXiv:2503.19776},
  year={2025}
}