Search by item HOME > Access full text > Search by item

JBE, vol. 27, no. 4, pp.527-537, July, 2022

DOI: https://doi.org/10.5909/JBE.2022.27.4.527

Survey on Deep learning-based Content-adaptive Video Compression Techniques

Changwoo Han, Hongil Kim, Hyun-ku Kang, Hyoungjin Kwon, Sung-Chang Lim, and Seung-Won Jung

C.A E-mail: swjung83@korea.ac.kr

Abstract:

As multimedia contents demand and supply increase, internet traffic around the world increases. Several standardization groups are striving to establish more efficient compression standards to mitigate the problem. In particular, research to introduce deep learning technology into compression standards is actively underway. Despite the fact that deep learning-based technologies show high performance, they suffer from the domain gap problem when test video sequences have different characteristics of training video sequences. To this end, several methods have been made to introduce content-adaptive deep video compression. In this paper, we will look into these methods by three aspects: codec information-aware methods, model selection methods, and information signaling methods.



Keyword: Contents adaptive filtering, Deep-learning, In-loop filtering, Post-processing, Video compression

Reference:
[1] Cisco, Cisco Annual Internet Report (2018-2023) White Paper, Mar. 2020.
[2] A. Skodras, C. Christopoulos and T. Ebrahimi, "The JPEG 2000 still image compression standard," Signal Processing Magazine, Vol.18,, No.5, pp 36-58, 2001. doi: https://doi.org/10.1109/79.952804
[3] B. Bross, J. Chen, S. Liu and Y.-K. Wang, “Versatile video coding (Draft 10),” JVET-S2001, Jul. 2020.
[4] Wei Jia, et al “Residual-guided In-loop Filter Using Convolution Neural Network,“ ACM Trans. Multimedia Comput. Communications, and Applications, 2021 doi: https://doi.org/10.1145/3460820
[5] Li, Daowen, and Lu Yu. "An in-loop filter based on low-complexity CNN using residuals in intra video coding," IEEE International Symposium on Circuits And Systems 2019. doi: https://doi.org/10.1109/ISCAS.2019.8702443
[6] Dai, Yuanying, Dong Liu, and Feng Wu. "A convolutional neural network approach for post-processing in HEVC intra coding," International Conference on Multimedia Modeling, Springer, Cham, pp. 28-39, 2017. doi: https://doi.org/10.1007/978-3-319-51811-4_3
[7] Huang, Zhijie, et al. "An efficient QP variable convolutional neural network based in-loop filter for intra coding." IEEE Data Compression Conference, pp. 33-42, 2021. doi: https://doi.org/10.1109/dcc50243.2021.00011
[8] Y. Li, L. Zhang, K. Zhang, “Conditional in-loop filter with parameter selection”, JVET-V0101, Apr. 2021.
[9] Wang, Ming-Ze, et al. "Attention-based dual-scale CNN in-loop filter for versatile video coding," IEEE Access, Vol.7, pp. 145214-145226, 2019. doi: https://doi.org/10.1109/access.2019.2944473
[10] Xu, Xiaoyu, et al. "Dense inception attention neural network for in-loop filter," IEEE Picture Coding Symposium, pp. 1-5, 2019. doi: https://doi.org/10.1109/pcs48520.2019.8954499
[11] Z. Dai, et al, “AHG11: Neural network-nased adaptive model selection for CNN in-loop filtering”, JVET-X0126, Oct. 2021.
[12] Jia, Chuanmin, et al. "Content-aware convolutional neural network for in-loop filtering in high efficiency Video coding," IEEE Transactions on Image Processing, Vol.28, No.7, 2019. doi: https://doi.org/10.1109/tip.2019.2896489
[13] Li, Yue, Li Zhang, and Kai Zhang. "IDAM: Iteratively trained deep in-loop filter with adaptive model selection," ACM Transaction on Multimedia Computing, Communications, and Application, 2022. doi: https://doi.org/10.1145/3529107
[14] Y. Li, K. Zhang, and L. Zhang. “EE1-1.2: Test on deep in-loop filter with adaptive model selection and external attention,” JVET-X0065, Oct, 2021.
[15] L. Wang, X. Xu, and S. Liu, “AHG11: Neural network based in-loop filter with adaptive model selection,” JVET-X0054, Oct. 2021.
[16] W. Lin, et al. “Partition-aware adaptive switching neural networks for post-processing In HEVC,” IEEE Transactions on Multimedia, Vol.22, No.11, pp. 2749-2763, 2019. doi: https://doi.org/10.1109/tmm.2019.2962310
[17] L. van Der Maaten, and G. Hinton. "Visualizing data using t-SNE," Journal of Machine Learning Research, Vol.9, No.11, 2008.
[18] Lam, Yat-Hong, et al. "Efficient adaptation of neural network filter for video compression." Adaptive Model Selection," ACM International Conference on Multimedia, pp. 358-366, 2020. doi: https://doi.org/10.1145/3394171.3413536
[19] M. Santamaria, et al. “AHG11: Hannuksela, Content-adaptive post-processing filter,” JVET-Y0059, Jan. 2022.
[20] M. Santamaria, et al. “AHG11: Content-adaptive neural network post-filte,” JVET-Z0082, Apr. 2022.
[21] Lee. So Yoon, et al. "Offset-based in-loop filtering with a deep network in HEVC," IEEE Access, Vol.8, pp. 213958-213967, 2020. doi: https://doi.org/10.1109/access.2020.3040751
[22] Kong, Lingyi, et al. "Guided CNN restoration with explicitly signaled linear combination," IEEE International Conference on Image Processing, pp. 3379-3383, 2020. doi: https://doi.org/10.1109/icip40778.2020.9190807
[23] Bordes, Philippe, et al. "Revisiting the sample adaptive offset post-filter of VVC with neural-networks," IEEE Picture Coding Symposium, pp. 1-5, 2021. doi: https://doi.org/10.1109/pcs50896.2021.9477457

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved