Search by item HOME > Access full text > Search by item

JBE, vol. 27, no. 5, pp.638-653, September, 2022


3D Clothes Modeling of Virtual Human for Metaverse

Hyun Woo Kim, Dong Eon Kim, Yujin Kim, and In Kyu Park

C.A E-mail:


In this paper, we propose the new method of creating 3D virtual-human reflecting the pattern of clothes worn by the person in the high-resolution whole body front image and the body shape data about the person. To get the pattern of clothes, we proceed Instance Segmentation and clothes parsing using Cascade Mask R-CNN. After, we use Pix2Pix to blur the boundaries and estimate the background color and can get UV-Map of 3D clothes mesh proceeding UV-Map base warping. Also, we get the body shape data using SMPL-X and deform the original clothes and body mesh. With UV-Map of clothes and deformed clothes and body mesh, user finally can see the animation of 3D virtual-human reflecting user’s appearance by rendering with the state-of-the game engine, i.e. Unreal Engine.

Keyword: Metaverse, virtual human, 3D cloth modeling

[1] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL: A Skinned multi-person linear model,” ACM Trans. on Graphics, vol. 34, no. 6, pp. 1-16, 2015. doi:
[2] J. Deng, S. Cheng, N. Xue, Y. Zhou, and S. Zafeiriou, “UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition,” arXiv preprint arXiv:1712.04695, 2017. doi:
[3] X. Han, Z. Wu, Z. Wu, R. Yu, and L. S. Davis, “VITON: An image- based virtual try-on network,” arXiv preprint arXiv:1711.08447, 2017. doi:
[4] Metahuman Creator, (accessed June 13, 2022)
[5] Unreal Engine, (accessed June 13, 2022)
[6] Blender, (accessed June 13, 2022)
[7] G. Pavlakos, V. Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black, “Expressive body capture: 3D hands, face, and body from a single image,” Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019. doi:
[8] K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open MMLab detection toolbox and benchmark,” arXiv preprint arXiv:1906.07155, 2019. doi:
[9] Z. Cai and N. Vasconcelos, “Cascade R-CNN: High quality object detection and instance segmentation,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 43, no. 5, pp. 1483-1498, 2019. doi:
[10] Y. Ge, R. Zhang, X. Wang, X. Tang, and P. Luo, “DeepFashion2: A versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images,” Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2019. doi:
[11] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. doi:
[12] P. Isola, J. -Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2017. doi:
[13] Fashion Product Images Dataset, paramaggarwal/fashion-product-images-dataset (accessed June 13, 2022)


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved