| 引用本文: | 邬开俊,张园园,张巧艳.跨尺度交叉注意力的红外和可见光图像融合[J].哈尔滨工业大学学报,2026,58(9):135.DOI:10.11918/202410008 |
| Wu Kaijun,Zhang Yuanyuan,Zhang Qiaoyan.Infrared and visible image fusion with cross-scale and cross-attention[J].Journal of Harbin Institute of Technology,2026,58(9):135.DOI:10.11918/202410008 |
|
| 摘要: |
| 为改善红外和可见光图像融合过程中信息丢失及特征融合局限性等问题,提出一种跨尺度交叉注意力的Transformer网络。首先,设计跨尺度视觉注意力模块以加强弱光照条件下特征提取能力,重点关注低对比度区域的信息;其次,构造交叉约束模块,通过嵌套连接进行全局特征的交互与融合;最后,提出一种基于Transformer的融合策略,进一步提升图像融合性能,增强细节信息保留。与其他9种先进融合网络的对比实验表明:在客观评价方面,跨尺度交叉注意力的Transformer网络在TNO数据集上的8项评价指标分别提升了10.5%~27.8%, 在Road scene数据集上的8项评价指标分别提升了8.6%~30.3%;在主观评价方面,跨尺度交叉注意力的Transformer网络展现出更优的视觉表现,显著提升了融合图像的可解释性与实用性,能有效保留红外图像中的热目标信息与可见光图像中的细节纹理。所提网络能够充分提取源图像信息,生成纹理细节更清晰、对比度更佳的融合结果,为提升图像融合性能及支持下游任务提供有益参考。 |
| 关键词: 红外和可见光 图像融合 交叉注意力机制 跨尺度交互式Transformer |
| DOI:10.11918/202410008 |
| 分类号:TP391.41 |
| 文献标识码:A |
| 基金项目:甘肃省自然科学基金(23JRRA913);内蒙古自治区重点研发与成果转化计划项目(2023YFSH3,3YFDZ0054);兰州交通大学重点研发项目(ZDYF2304) |
|
| Infrared and visible image fusion with cross-scale and cross-attention |
|
Wu Kaijun1,Zhang Yuanyuan1,Zhang Qiaoyan2
|
|
(1.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China; 2.Data and Equipment Department, China Railway Information Technology Group Co. Ltd., Beijing 100080, China)
|
| Abstract: |
| To address issues such as information loss and limitations in feature fusion during infrared and visible image fusion, a cross-scale cross-attention Transformer network is proposed. First, a cross-scale visual attention module is designed to enhance feature extraction capabilities under low-light conditions, focusing on information in low-contrast regions. Second, a cross-domain constrain module is constructed to facilitate interaction and fusion of global features through nested connections. Finally, a Transformer-based fusion strategy is proposed to further enhance image fusion performance and improve detail retention. Comparative experiments with nine other advanced fusion networks showed that in terms of objective evaluation, the cross-scale cross-attention Transformer network improved 8 evaluation indicators by 10.5% to 27.8% on the TNO dataset and by 8.6% to 30.3% on the Road scene dataset; In terms of subjective evaluation, the cross-scale cross-attention Transformer network exhibits better visual performance, significantly improving the interpretability and practicality of fused images, and effectively preserving thermal target information in infrared images and detailed textures in visible images. The proposed network can fully extract source image information, generate fusion results with clearer texture details and better contrast, providing useful references for improving image fusion performance and supporting downstream tasks. |
| Key words: infrared and visible image fusion cross-attention mechanism cross-scale interactive Transformer |