|Search by item||HOME > Access full text > Search by item|
JBE, vol. 27, no. 4, pp.477-486, July, 2022
Accurate Prediction of VVC Intra-coded Block using Convolutional Neural Network
Hye-Sun Jeong and Je-Won Kang
C.A E-mail: firstname.lastname@example.org
In this paper, we propose a novel intra-prediction method using convolutional neural network (CNN) to improve a quality of a predicted block in VVC. The proposed algorithm goes through a two-step procedure. First, an input prediction block is generated using one of the VVC intra-prediction modes. Second, the prediction block is further refined through a CNN model, by inputting the prediction block itself and reconstructed reference samples in the boundary. The proposed algorithm outputs a refined block to reduce residual signals and enhance coding efficiency, which is enabled by a CU-level flag. Experimental results demonstrate that the proposed method achieves improved rate-distortion performance as compared a VVC reference software, I.e., VTM version 10.0.
Keyword: Intra Prediction, Convolutional Neural Network, Versatile Video Coding
 Versatile Video Coding Test Model (VTM) 10.0 : https://vcgit.hhi. fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-10.0
 T. Barnett, et al. "Cisco visual networking index (vni) complete forecast update, 2017–2022." Americas/EMEAR Cisco Knowledge Network (CKN) Presentation (2018).
 Moving Picture Experts Group (MPEG) : https://mpeg.chiariglione. org
 JVET of ITU-T and ISO/IEC, “Versatile Video Coding (Draft 10)”, Documents JEVT-S2001, June 2020.
 B. Bross et al., “Overview of the Versatile Video Coding (VVC) Standard and its Applications”IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3101953
 J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3072430
 J. Li, et al. "Intra prediction using fully connected network for video coding." 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017. doi: https://doi.org/10.1109/ICIP.2017.8296231
 JVET of ITU-T and ISO/IEC, “AHG11 Neural Network-based Intra Prediction with Transform selection in VVC”, Document JVET-T0073, Oct. 2020.
 Y. Hu, et al. "Enhanced intra prediction with recurrent neural network in video coding." 2018 Data Compression Conference. IEEE, 2018. doi: https://doi.org/https://doi.org/10.1109/DCC.2018.00066
 Y. Hu, et al. "Optimized spatial recurrent network for intra prediction in video coding." 2018 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2018. doi: https://doi.org/10.1109/VCIP.2018.8698658
 Y. Hu, et al. "Progressive spatial recurrent neural network for intra prediction." IEEE Transactions on Multimedia 21.12 (2019): 3024-3037. doi: https://doi.org/10.1109/TMM.2019.2920603
 Y. Wang, et al. "Multi-scale convolutional neural network-based intra prediction for video coding." IEEE Transactions on Circuits and Systems for Video Technology 30.7 (2019): 1803-1815. doi: https://doi.org/10.1109/TCSVT.2019.2934681
 L. Zhu, et al. "Generative adversarial network-based intra prediction for video coding." IEEE transactions on multimedia 22.1 (2019): 45-58. doi: https://doi.org/10.1109/TMM.2019.2924591
 F. Brand, S. Jürgen, and K. André. "Intra frame prediction for video coding using a conditional autoencoder approach." 2019 Picture Coding Symposium (PCS). IEEE, 2019. doi: https://doi.org/10.1109/PCS48520.2019.8954546
 M. G. Blanch, et al. "Chroma intra prediction with attention-based CNN architectures." 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020. doi: https://doi.org//ICIP40778.2020.9191050
 JVET of ITU-T and ISO/IEC, "AHG11: Neural Network based cross-component Prediction model", Deocument JVET-W0111, July 2021
 Sang-hyo Park and Je-Won Kang, "Fast Multi-type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network", IEEE Transactions on Multimedia, 2021. doi: https://doi.org/10.1109/TMM.2020.3042062
 Jyung-Kyung Lee, Nayoung Kim, Seunghyun Cho, and Je-Won Kang, "Deep Video Prediction Network Based Inter-Frame Coding in HEVC," IEEE Access, 2020. doi: https://doi.org/10.1109/ACCESS.2020.2993566
 Sookyung Ryu and Je-Won Kang, "Machine Learning-Based Fast Angular Prediction Mode Decision Technique in Video Coding," IEEE Transaction on Image Processing, Nov. 2018. doi: https://doi.org/10.1109/TIP.2018.2857404
 Je-Won Kang, Soo-Kyung Ryu, Na-Young Kim, Minjoo Kang, "Efficient Residual DPCM using an L-1 Robust Linear Prediction in Screen Content Video Coding," IEEE Transaction on Multimedia, vol. 18, no. 10, pp.2054-2065, Oct. 2016. doi: https://doi.org/10.1109/TMM.2016.2595259
 Jyung-Kyung Lee, Nayoung Kim, and Je-Won Kang, "Rate-distortion optimized temporal segmentation using reinforcement leaning for video coding," APSIPA, 2021.
 Je-Won Kang, Gabbouj, M., and Jay Kuo, C. C. “Sparse/DCT (S/DCT) two-layered representation of prediction residuals for video coding”, IEEE Transactions on Image Processing, 22(7), 2711-2722. doi: https://doi.org/10.1109/TIP.2013.2256917
 Radu Timofte, Eirikur Agustsson, Luc Van Gool, MingHsuan Yang, and Lei Zhang, “Ntire 2017 challenge on single image super- resolution: Methods and results,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 114–125. doi: https://doi.org/10.1109/cvprw.2017.150
 H. Jeong, Je-Won Kang “Improvements of intra-prediction in VVC”, Summer conference in the Korean Society of Broadcast and Media Engineers, 2022
 H. Jeong, Je-Won Kang, “Intra prediction through block refinement”, 34th Image processing and understanding, 2022.