Search by item HOME > Access full text > Search by item

JBE, vol. 27, no. 5, pp.619-628, September, 2022

DOI: https://doi.org/10.5909/JBE.2022.27.5.619

Multi-attribute Face Editing using Facial Masks

Laudwika Ambardi, In Kyu Park, and Sungeun Hong

C.A E-mail: csehong@inha.ac.kr

Abstract:

Although face recognition and face generation have been growing in popularity, the privacy issues of using facial images in the wild have been a concurrent topic. In this paper, we propose a face editing network that can reduce privacy issues by generating face images with various properties from a small number of real face images and facial mask information. Unlike the existing methods of learning face attributes using a lot of real face images, the proposed method generates new facial images using a facial segmentation mask and texture images from five parts as styles. The images are then trained with our network to learn the styles and locations of each reference image. Once the proposed framework is trained, we can generate various face images using only a small number of real face images and segmentation information. In our extensive experiments, we show that the proposed method can not only generate new faces, but also localize facial attribute editing, despite using very few real face images.



Keyword: Face editing, Image synthesis, Facial mask, Data privacy

Reference:
[1] Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. "Generative adversarial nets," Proceedings of Advances in Neural Information Processing Systems, 2014. doi: https://doi.org/10.48550/arXiv.1406.2661
[2] Wang, X. and Gupta, A., “Generative image modeling using style and structure adversarial networks,“ Proceedings of European Conference on Computer Vision, pp. 318-335, 2016. doi: https://doi.org/10.1007/978-3-319-10578-9_24
[3] Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434, 2015. doi: https://doi.org/10.48550/arXiv.1511.06434
[4] Choi, Yunjey, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8789-8797, 2018. doi: https://doi.org/10.1109/CVPR.2018.00916
[5] Zhu, Yuhao, Qi Li, Jian Wang, Cheng-Zhong Xu, and Zhenan Sun. "One shot face swapping on megapixels," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4834-4844, 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00480
[6] Korshunova, Iryna, Wenzhe Shi, Joni Dambre, and Lucas Theis. "Fast face-swap using convolutional neural networks," Proceedings of the IEEE International Conference on Computer Vision, pp. 3677-3685, 2017. doi: https://doi.org/10.1109/ICCV.2017.397
[7] Xu, Yangyang, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, and Shengfeng He. "High-resolution face swapping via latent semantics disentanglement," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7642-7651, 2022. doi: https://doi.org/10.48550/arXiv.2203.15958
[8] Gu, Shuyang, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen, and Lu Yuan. "Mask-guided portrait editing with conditional gans," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3436-3445, 2019. doi: https://doi.org/10.1109/CVPR.2019.00355
[9] Zhu, Peihao, Rameen Abdal, Yipeng Qin, and Peter Wonka. "Sean: Image synthesis with semantic region-adaptive normalization," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5104-5113, 2020. doi: https://doi.org/10.48550/arXiv.1911.12861
[10] Kim, Hyunsu, Yunjey Choi, Junho Kim, Sungjoo Yoo, and Youngjung Uh. "Exploiting spatial dimensions of latent in gan for real-time image editing," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 852-861, 2021. doi: https://doi.org/10.48550/arXiv.2104.14754
[11] Zhou, Hang, Jihao Liu, Ziwei Liu, Yu Liu, and Xiaogang Wang. "Rotate-and-render: Unsupervised photorealistic face rotation from single-view images," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5911-5920, 2020. doi: https://doi.org/10.48550/arXiv.2003.08124
[12] Wang, Ting-Chun, Arun Mallya, and Ming-Yu Liu. "One-shot free-view neural talking-head synthesis for video conferencing," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10039-10049, 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00991
[13] Vitadhani, Agastya, Kalamullah Ramli, and Prima Dewi Purnamasari. "Detection of Clickbait Thumbnails on YouTube Using Tesseract- OCR, Face Recognition, and Text Alteration," Proceedings of International Conference on Artificial Intelligence and Computer Science Technology (ICAICST), pp. 56-61, 2021. doi: https://doi.org/10.1109/ICAICST53116.2021.9497811
[14] Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784, 2014. doi: https://doi.org/10.48550/arXiv.1411.1784
[15] Isola, Phillip, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125-1134, 2017. doi: https://doi.org/10.1109/CVPR.2017.632
[16] Wang, Ting-Chun, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. "High-resolution image synthesis and semantic manipulation with conditional gans," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8798-8807, 2018. doi:https://doi.org/10.1109/CVPR.2018.00917
[17] Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2414-2423, 2016. doi:https://doi.org/10.1109/CVPR.2016.265
[18] Huang, Xun, and Serge Belongie. "Arbitrary style transfer in real-time with adaptive instance normalization," Proceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017. doi: https://doi.org/10.48550/arXiv.1703.06868
[19] Karras, Tero, Timo Aila, Samuli Laine, and Jaakko Lehtinen. "Progressive Growing of GANs for Improved Quality, Stability, and Variation," Proceedings of International Conference on Learning Representations, 2018. doi: https://doi.org/10.48550/arXiv.1710.10196
[20] Grassucci, Eleonora, Edoardo Cicero, and Danilo Comminiello. "Quaternion generative adversarial networks," In Generative Adversarial Learning: Architectures and Applications, pp. 57-86. Springer, 2022. doi: https://doi.org/10.1007/978-3-030-91390-8_4

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved