Abstract:To address the loss of geometric information of distant objects caused by the sparsity of light detection and ranging (LiDAR) point clouds and the problems of projection noise interference and non-standardized feature interaction in existing modal fusion methods, this paper proposed an end-to-end multi-modal three-dimensional detection network, VAF-Net. First, this paper constructed a noise-resistant virtual point enhancement framework, which utilized images to generate virtual points to densify the point clouds and employed VirConvNet as the backbone network to extract global geometric features while effectively filtering the noise introduced by the virtual points. Second, this paper designed a color semantic feature extraction branch to precisely extract local textures and contextual semantics of images aligned with the three-dimensional space through a semantic lifting module. Finally, this paper proposed a bidirectional cross-attention fusion module based on standard Transformer principles. Through a symmetric dual-stream architecture and a gating mechanism, it achieved deep, stable, and bidirectional interaction between LiDAR geometric features and image semantic features. Experiments on the KITTI three-dimensional object detection benchmark indicate that VAF-Net achieves state-of-the-art performance. Particularly in the detection of hard samples, its 3D average precision reaches 82.74%, significantly outperforming mainstream methods such as Voxel-RCNN. The research results demonstrate that VAF-Net effectively realizes the complementary enhancement of multi-modal information through the standardized bidirectional fusion architecture and significantly improves the detection robustness in complex scenarios.