采用多层卷积稀疏表示的红外与可见光图像融合

Infrared and visible image fusion via multi-layer convolutional sparse representation

摘要:

为提升光学图像在低照度条件下的可用性,对红外图像与可见光图像进行融合从而结合两者的优势是一种有效的技术手段。稀疏表示理论在红外与可见光图像融合领域已经得到了广泛的应用,然而基于稀疏表示理论的图像融合方法所采用的局部建模方式易于导致语义信息损失和对误匹配的低容忍度两大缺陷。卷积稀疏表示的全局建模能力对克服上述不足具有巨大的优势,本研究借鉴卷积神经网络的结构设计了一种前馈式基于多层卷积稀疏表示的红外与可见光图像融合网络,该网络包含5层:第1、2层为卷积稀疏层,通过预训练的字典滤波器获取源图像的卷积稀疏响应；第3层为融合层,通过活性图评价以获取融合结果；第4、5层为重建层,基于融合结果结合字典滤波器重建融合图像。实验结果表明,所提出的图像融合方法有效抑制了稀疏表示理论应用于图像融合的两大不足,在客观评价指标方面明显优于基于稀疏表示、基于单层卷积稀疏表示和基于卷积神经网络的图像融合算法,在算法的计算复杂度和运行时间方面优于基于稀疏表示和基于卷积神经网络的图像融合算法。

Abstract:

Integrating the advantages of infrared and visible images by image fusion is an effective means to enhance the applicability of optical images in low illumination conditions. Despite the wide application of sparse representation (SR) theory in the field of infrared and visible image fusion, the drawbacks including detail loss and low toleration with mis-registration caused by the local patch representation nature of SR have never been effectively solved. Different from SR, the global representation capability of the recently emerged convolutional sparse representation (CSR) model reveals huge potential to overcome the above mentioned deficiencies. Drawing on the convolutional neural network (CNN) architecture, a multi-layer CSR model was designed for pixel level image fusion. The image fusion model was constructed with five layers in a forward-feeding manner: the first two layers are CSR layers which acquire sparse coefficient maps with response to the pre-learned dictionary filter sets; the third layer is fusion layer which obtains fused results of the sparse coefficient maps; the last two are reconstruction layers which reconstruct the fused image step by step, and the fusion results are thus obtained. Experimental results indicate that the image fusion method proposed in this paper can effectively overcome the two drawbacks of SR. The method outperforms SR, CSR, and CNN in the aspect of objective assessment metrics, and outperforms SR and CNN in terms of computation complexity and computation time.