|Search by item||HOME > Access full text > Search by item|
JBE, vol. 27, no. 4, pp.527-537, July, 2022
Survey on Deep learning-based Content-adaptive Video Compression Techniques
Changwoo Han, Hongil Kim, Hyun-ku Kang, Hyoungjin Kwon, Sung-Chang Lim, and Seung-Won Jung
C.A E-mail: email@example.com
As multimedia contents demand and supply increase, internet traffic around the world increases. Several standardization groups are striving to establish more efficient compression standards to mitigate the problem. In particular, research to introduce deep learning technology into compression standards is actively underway. Despite the fact that deep learning-based technologies show high performance, they suffer from the domain gap problem when test video sequences have different characteristics of training video sequences. To this end, several methods have been made to introduce content-adaptive deep video compression. In this paper, we will look into these methods by three aspects: codec information-aware methods, model selection methods, and information signaling methods.
Keyword: Contents adaptive filtering, Deep-learning, In-loop filtering, Post-processing, Video compression
 Cisco, Cisco Annual Internet Report (2018-2023) White Paper, Mar. 2020.
 A. Skodras, C. Christopoulos and T. Ebrahimi, "The JPEG 2000 still image compression standard," Signal Processing Magazine, Vol.18,, No.5, pp 36-58, 2001. doi: https://doi.org/10.1109/79.952804
 B. Bross, J. Chen, S. Liu and Y.-K. Wang, “Versatile video coding (Draft 10),” JVET-S2001, Jul. 2020.
 Wei Jia, et al “Residual-guided In-loop Filter Using Convolution Neural Network,“ ACM Trans. Multimedia Comput. Communications, and Applications, 2021 doi: https://doi.org/10.1145/3460820
 Li, Daowen, and Lu Yu. "An in-loop filter based on low-complexity CNN using residuals in intra video coding," IEEE International Symposium on Circuits And Systems 2019. doi: https://doi.org/10.1109/ISCAS.2019.8702443
 Dai, Yuanying, Dong Liu, and Feng Wu. "A convolutional neural network approach for post-processing in HEVC intra coding," International Conference on Multimedia Modeling, Springer, Cham, pp. 28-39, 2017. doi: https://doi.org/10.1007/978-3-319-51811-4_3
 Huang, Zhijie, et al. "An efficient QP variable convolutional neural network based in-loop filter for intra coding." IEEE Data Compression Conference, pp. 33-42, 2021. doi: https://doi.org/10.1109/dcc50243.2021.00011
 Y. Li, L. Zhang, K. Zhang, “Conditional in-loop filter with parameter selection”, JVET-V0101, Apr. 2021.
 Wang, Ming-Ze, et al. "Attention-based dual-scale CNN in-loop filter for versatile video coding," IEEE Access, Vol.7, pp. 145214-145226, 2019. doi: https://doi.org/10.1109/access.2019.2944473
 Xu, Xiaoyu, et al. "Dense inception attention neural network for in-loop filter," IEEE Picture Coding Symposium, pp. 1-5, 2019. doi: https://doi.org/10.1109/pcs48520.2019.8954499
 Z. Dai, et al, “AHG11: Neural network-nased adaptive model selection for CNN in-loop filtering”, JVET-X0126, Oct. 2021.
 Jia, Chuanmin, et al. "Content-aware convolutional neural network for in-loop filtering in high efficiency Video coding," IEEE Transactions on Image Processing, Vol.28, No.7, 2019. doi: https://doi.org/10.1109/tip.2019.2896489
 Li, Yue, Li Zhang, and Kai Zhang. "IDAM: Iteratively trained deep in-loop filter with adaptive model selection," ACM Transaction on Multimedia Computing, Communications, and Application, 2022. doi: https://doi.org/10.1145/3529107
 Y. Li, K. Zhang, and L. Zhang. “EE1-1.2: Test on deep in-loop filter with adaptive model selection and external attention,” JVET-X0065, Oct, 2021.
 L. Wang, X. Xu, and S. Liu, “AHG11: Neural network based in-loop filter with adaptive model selection,” JVET-X0054, Oct. 2021.
 W. Lin, et al. “Partition-aware adaptive switching neural networks for post-processing In HEVC,” IEEE Transactions on Multimedia, Vol.22, No.11, pp. 2749-2763, 2019. doi: https://doi.org/10.1109/tmm.2019.2962310
 L. van Der Maaten, and G. Hinton. "Visualizing data using t-SNE," Journal of Machine Learning Research, Vol.9, No.11, 2008.
 Lam, Yat-Hong, et al. "Efficient adaptation of neural network filter for video compression." Adaptive Model Selection," ACM International Conference on Multimedia, pp. 358-366, 2020. doi: https://doi.org/10.1145/3394171.3413536
 M. Santamaria, et al. “AHG11: Hannuksela, Content-adaptive post-processing filter,” JVET-Y0059, Jan. 2022.
 M. Santamaria, et al. “AHG11: Content-adaptive neural network post-filte,” JVET-Z0082, Apr. 2022.
 Lee. So Yoon, et al. "Offset-based in-loop filtering with a deep network in HEVC," IEEE Access, Vol.8, pp. 213958-213967, 2020. doi: https://doi.org/10.1109/access.2020.3040751
 Kong, Lingyi, et al. "Guided CNN restoration with explicitly signaled linear combination," IEEE International Conference on Image Processing, pp. 3379-3383, 2020. doi: https://doi.org/10.1109/icip40778.2020.9190807
 Bordes, Philippe, et al. "Revisiting the sample adaptive offset post-filter of VVC with neural-networks," IEEE Picture Coding Symposium, pp. 1-5, 2021. doi: https://doi.org/10.1109/pcs50896.2021.9477457