融合边缘和关键点的遮挡人脸修复网络

Occluded face inpainting network fusing edges and key points

摘要:

人脸图像修复技术通过修复人脸被遮挡区域以生成完整的人脸图像,其在刑事侦查、安全防护等领域具有重要的应用价值,但现有方法的修复结果常常会出现纹理模糊、人脸结构扭曲等伪影问题。为此,在生成对抗网络框架的基础上,提出了一种融合边缘和关键点的遮挡人脸修复网络。首先,利用结构森林边缘修复网络来补全遮挡人脸图像的结构森林边缘图,以获得更精细的人脸细节描述信息；其次,利用关键点预测网络对遮挡人脸的68个关键点进行定位,以获得人脸图像的拓扑结构信息；最后,将上述两个网络输出的结构森林边缘图和人脸关键点作为先验信息,通过人脸图像修复网络对遮挡人脸区域进行修复并生成完整人脸图像。在CelebA-HQ数据集上的实验结果表明:所提算法修复的人脸图像纹理细节更精细、人脸拓扑结构更合理；在不同遮挡比例下,所提算法的PSNR和SSIM均高于对比算法；在掩码占比为50%时,与GatedConv、EdgeConnect、 LaFIn算法相比, 所提算法的PSNR分别提升了36.8%、 25.8%、 29.3%, 而SSIM分别提升了19.5%、12.2%、12.2%。

Abstract:

The face image inpainting technology can generate a complete face image by repairing the occluded area of the face, which has important application value in fields such as criminal investigation and security protection. However, the inpainting results of the existing methods often exhibit artifacts such as fuzzy texture and distorted face structure. Therefore, based on the generative adversarial network (GAN) framework, this paper proposed an occluded face restoration network fusing edges and key points. Firstly, the proposed network used the structural forest edge restoration network to complete the structural forest edge map occluding the face image to obtain more description information of the face details. Then, it used the key point prediction network to locate 68 key points of the occluded face to obtain the topological structure information of the face image. Finally, it took the structural forest edge map and the key points of face obtained by the above two networks as prior information, restored the occluded face area by the face image inpainting network, and generated a complete face image. The experimental results on the CelebA-HQ dataset show that the face images restored by the proposed algorithm have finer texture details and more reasonable topological structures of faces. Under different occluded areas, the PSNR and SSIM of the proposed algorithm are higher than those of the comparison algorithm. Compared with that of GatedConv, EdgeConnect, and LaFIn algorithms, when the mask ratio is 50%, the PSNR of the proposed algorithm increases by 36.8%, 25.8%, and 29.3%, respectively, while the SSIM increases by 19.5%, 12.2%, and 12.2%.