Occluded face inpainting network fusing edges and key points

Abstract:

The face image inpainting technology can generate a complete face image by repairing the occluded area of the face, which has important application value in fields such as criminal investigation and security protection. However, the inpainting results of the existing methods often exhibit artifacts such as fuzzy texture and distorted face structure. Therefore, based on the generative adversarial network (GAN) framework, this paper proposed an occluded face restoration network fusing edges and key points. Firstly, the proposed network used the structural forest edge restoration network to complete the structural forest edge map occluding the face image to obtain more description information of the face details. Then, it used the key point prediction network to locate 68 key points of the occluded face to obtain the topological structure information of the face image. Finally, it took the structural forest edge map and the key points of face obtained by the above two networks as prior information, restored the occluded face area by the face image inpainting network, and generated a complete face image. The experimental results on the CelebA-HQ dataset show that the face images restored by the proposed algorithm have finer texture details and more reasonable topological structures of faces. Under different occluded areas, the PSNR and SSIM of the proposed algorithm are higher than those of the comparison algorithm. Compared with that of GatedConv, EdgeConnect, and LaFIn algorithms, when the mask ratio is 50%, the PSNR of the proposed algorithm increases by 36.8%, 25.8%, and 29.3%, respectively, while the SSIM increases by 19.5%, 12.2%, and 12.2%.