跨尺度交叉注意力的红外和可见光图像融合
CSTR:
作者:
作者单位:

(1.兰州交通大学 电子与信息工程学院,兰州 730070;2.中国铁路信息科技集团有限公司 数据和装备处,北京 100080)

作者简介:

邬开俊(1978—),男,教授,博士生导师

通讯作者:

邬开俊,wkj@mail.lzjtu.cn

中图分类号:

TP391.41

基金项目:

甘肃省自然科学基金(23JRRA913);内蒙古自治区重点研发与成果转化计划项目(2023YFSH3,3YFDZ0054);兰州交通大学重点研发项目(ZDYF2304)


Infrared and visible image fusion with cross-scale and cross-attention
Author:
Affiliation:

(1.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China; 2.Data and Equipment Department, China Railway Information Technology Group Co. Ltd., Beijing 100080, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    为改善红外和可见光图像融合过程中信息丢失及特征融合局限性等问题,提出一种跨尺度交叉注意力的Transformer网络。首先,设计跨尺度视觉注意力模块以加强弱光照条件下特征提取能力,重点关注低对比度区域的信息;其次,构造交叉约束模块,通过嵌套连接进行全局特征的交互与融合;最后,提出一种基于Transformer的融合策略,进一步提升图像融合性能,增强细节信息保留。与其他9种先进融合网络的对比实验表明:在客观评价方面,跨尺度交叉注意力的Transformer网络在TNO数据集上的8项评价指标分别提升了10.5%~27.8%, 在Road scene数据集上的8项评价指标分别提升了8.6%~30.3%;在主观评价方面,跨尺度交叉注意力的Transformer网络展现出更优的视觉表现,显著提升了融合图像的可解释性与实用性,能有效保留红外图像中的热目标信息与可见光图像中的细节纹理。所提网络能够充分提取源图像信息,生成纹理细节更清晰、对比度更佳的融合结果,为提升图像融合性能及支持下游任务提供有益参考。

    Abstract:

    To address issues such as information loss and limitations in feature fusion during infrared and visible image fusion, a cross-scale cross-attention Transformer network is proposed. First, a cross-scale visual attention module is designed to enhance feature extraction capabilities under low-light conditions, focusing on information in low-contrast regions. Second, a cross-domain constrain module is constructed to facilitate interaction and fusion of global features through nested connections. Finally, a Transformer-based fusion strategy is proposed to further enhance image fusion performance and improve detail retention. Comparative experiments with nine other advanced fusion networks showed that in terms of objective evaluation, the cross-scale cross-attention Transformer network improved 8 evaluation indicators by 10.5% to 27.8% on the TNO dataset and by 8.6% to 30.3% on the Road scene dataset; In terms of subjective evaluation, the cross-scale cross-attention Transformer network exhibits better visual performance, significantly improving the interpretability and practicality of fused images, and effectively preserving thermal target information in infrared images and detailed textures in visible images. The proposed network can fully extract source image information, generate fusion results with clearer texture details and better contrast, providing useful references for improving image fusion performance and supporting downstream tasks.

    参考文献
    相似文献
    引证文献
引用本文

邬开俊,张园园,张巧艳.跨尺度交叉注意力的红外和可见光图像融合[J].哈尔滨工业大学学报,2026,58(9):135. DOI:10.11918/202410008

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-10-08
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2026-10-11
  • 出版日期:
文章二维码