Search by item HOME > Access full text > Search by item

JBE, vol. 27, no. 4, pp.477-486, July, 2022

DOI: https://doi.org/10.5909/JBE.2022.27.4.477

Accurate Prediction of VVC Intra-coded Block using Convolutional Neural Network

Hye-Sun Jeong and Je-Won Kang

C.A E-mail: jewonk@ewha.ac.kr

Abstract:

In this paper, we propose a novel intra-prediction method using convolutional neural network (CNN) to improve a quality of a predicted block in VVC. The proposed algorithm goes through a two-step procedure. First, an input prediction block is generated using one of the VVC intra-prediction modes. Second, the prediction block is further refined through a CNN model, by inputting the prediction block itself and reconstructed reference samples in the boundary. The proposed algorithm outputs a refined block to reduce residual signals and enhance coding efficiency, which is enabled by a CU-level flag. Experimental results demonstrate that the proposed method achieves improved rate-distortion performance as compared a VVC reference software, I.e., VTM version 10.0.



Keyword: Intra Prediction, Convolutional Neural Network, Versatile Video Coding

Reference:
[1] Versatile Video Coding Test Model (VTM) 10.0 : https://vcgit.hhi. fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-10.0
[2] T. Barnett, et al. "Cisco visual networking index (vni) complete forecast update, 2017–2022." Americas/EMEAR Cisco Knowledge Network (CKN) Presentation (2018).
[3] Moving Picture Experts Group (MPEG) : https://mpeg.chiariglione. org
[4] JVET of ITU-T and ISO/IEC, “Versatile Video Coding (Draft 10)”, Documents JEVT-S2001, June 2020.
[5] B. Bross et al., “Overview of the Versatile Video Coding (VVC) Standard and its Applications”IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3101953
[6] J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3072430
[7] J. Li, et al. "Intra prediction using fully connected network for video coding." 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017. doi: https://doi.org/10.1109/ICIP.2017.8296231
[8] JVET of ITU-T and ISO/IEC, “AHG11 Neural Network-based Intra Prediction with Transform selection in VVC”, Document JVET-T0073, Oct. 2020.
[9] Y. Hu, et al. "Enhanced intra prediction with recurrent neural network in video coding." 2018 Data Compression Conference. IEEE, 2018. doi: https://doi.org/https://doi.org/10.1109/DCC.2018.00066
[10] Y. Hu, et al. "Optimized spatial recurrent network for intra prediction in video coding." 2018 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2018. doi: https://doi.org/10.1109/VCIP.2018.8698658
[11] Y. Hu, et al. "Progressive spatial recurrent neural network for intra prediction." IEEE Transactions on Multimedia 21.12 (2019): 3024-3037. doi: https://doi.org/10.1109/TMM.2019.2920603
[12] Y. Wang, et al. "Multi-scale convolutional neural network-based intra prediction for video coding." IEEE Transactions on Circuits and Systems for Video Technology 30.7 (2019): 1803-1815. doi: https://doi.org/10.1109/TCSVT.2019.2934681
[13] L. Zhu, et al. "Generative adversarial network-based intra prediction for video coding." IEEE transactions on multimedia 22.1 (2019): 45-58. doi: https://doi.org/10.1109/TMM.2019.2924591
[14] F. Brand, S. Jürgen, and K. André. "Intra frame prediction for video coding using a conditional autoencoder approach." 2019 Picture Coding Symposium (PCS). IEEE, 2019. doi: https://doi.org/10.1109/PCS48520.2019.8954546
[15] M. G. Blanch, et al. "Chroma intra prediction with attention-based CNN architectures." 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020. doi: https://doi.org//ICIP40778.2020.9191050
[16] JVET of ITU-T and ISO/IEC, "AHG11: Neural Network based cross-component Prediction model", Deocument JVET-W0111, July 2021
[17] Sang-hyo Park and Je-Won Kang, "Fast Multi-type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network", IEEE Transactions on Multimedia, 2021. doi: https://doi.org/10.1109/TMM.2020.3042062
[18] Jyung-Kyung Lee, Nayoung Kim, Seunghyun Cho, and Je-Won Kang, "Deep Video Prediction Network Based Inter-Frame Coding in HEVC," IEEE Access, 2020. doi: https://doi.org/10.1109/ACCESS.2020.2993566
[19] Sookyung Ryu and Je-Won Kang, "Machine Learning-Based Fast Angular Prediction Mode Decision Technique in Video Coding," IEEE Transaction on Image Processing, Nov. 2018. doi: https://doi.org/10.1109/TIP.2018.2857404
[20] Je-Won Kang, Soo-Kyung Ryu, Na-Young Kim, Minjoo Kang, "Efficient Residual DPCM using an L-1 Robust Linear Prediction in Screen Content Video Coding," IEEE Transaction on Multimedia, vol. 18, no. 10, pp.2054-2065, Oct. 2016. doi: https://doi.org/10.1109/TMM.2016.2595259
[21] Jyung-Kyung Lee, Nayoung Kim, and Je-Won Kang, "Rate-distortion optimized temporal segmentation using reinforcement leaning for video coding," APSIPA, 2021.
[22] Je-Won Kang, Gabbouj, M., and Jay Kuo, C. C. “Sparse/DCT (S/DCT) two-layered representation of prediction residuals for video coding”, IEEE Transactions on Image Processing, 22(7), 2711-2722. doi: https://doi.org/10.1109/TIP.2013.2256917
[23] Radu Timofte, Eirikur Agustsson, Luc Van Gool, MingHsuan Yang, and Lei Zhang, “Ntire 2017 challenge on single image super- resolution: Methods and results,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 114–125. doi: https://doi.org/10.1109/cvprw.2017.150
[24] H. Jeong, Je-Won Kang “Improvements of intra-prediction in VVC”, Summer conference in the Korean Society of Broadcast and Media Engineers, 2022
[25] H. Jeong, Je-Won Kang, “Intra prediction through block refinement”, 34th Image processing and understanding, 2022.

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved