JBE, vol. 27, no. 4, pp.561-568, July, 2022

DOI: https://doi.org/10.5909/JBE.2022.27.4.561

Efficient Memory Update Module for Video Object Segmentation

Junho Jo and Nam Ik Cho

C.A E-mail: Video Object Segmentation, Memory, Update, Adaptive

Abstract:

Most deep learning-based video object segmentation methods perform the segmentation with past prediction information stored in external memory. In general, the more past information is stored in the memory, the better results can be obtained by accumulating evidence for various changes in the objects of interest. However, all information cannot be stored in the memory due to hardware limitations, resulting in performance degradation. In this paper, we propose a method of storing new information in the external memory without additional memory allocation. Specifically, after calculating the attention score between the existing memory and the information to be newly stored, new information is added to the corresponding memory according to each score. In this way, the method works robustly because the attention mechanism reflects the object changes well without using additional memory. In addition, the update rate is adaptively determined according to the accumulated number of matches in the memory so that the frequently updated samples store more information to maintain reliable information.

Keyword: Video Object Segmentation, Memory, Update, Adaptive

Reference:
[1] Oh, S. W., Lee, J. Y., Xu, N., & Kim, S. J., "Video object segmentation using space-time memory networks." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. doi: https://doi.org/10.1109/ICCV.2019.00932
[2] Li, Yu, Zhuoran Shen, and Ying Shan., "Fast video object segmentation using the global context module." European Conference on Computer Vision. Springer, Cham, 2020. doi: https://doi.org/10.1007/978-3-030-58607-2_43
[3] Liang, Y., Li, X., Jafari, N., & Chen, J., "Video object segmentation with adaptive feature bank and uncertain-region refinement." Advances in Neural Information Processing Systems 33: 3430-3441., 2020.
[4] Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine-Hornung, A., & Van Gool, L, "The 2017 davis challenge on video object segmentation." arXiv preprint arXiv:1704.00675, 2017.
[5] Ning Xu, Linjie Yang, Dingcheng Yue, Jianchao Yang, Brian Price, Jimei Yang, Scott Cohen, Yuchen Fan, Yuchen Liang, and Thomas Huang., “Youtube-vos: Sequence-to-sequence video object segmentation.“ In European Conference on Computer Vision (ECCV), 2018. doi: https://doi.org/10.1007/978-3-030-01228-1_36
[6] Yao, R., Lin, G., Xia, S., Zhao, J., & Zhou, Y., "Video object segmentation and tracking: A survey." ACM Transactions on Intelligent Systems and Technology (TIST) 11.4 ,1-47p, 2020. doi: http://dx.doi.org/10.1145/3391743
[7] Wang, H., Jiang, X., Ren, H., Hu, Y., & Bai, S., "Swiftnet: Real-time video object segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00135
[8] Hu, Yuan-Ting, Jia-Bin Huang, and Alexander G. Schwing. "Video- match: Matching based video object segmentation." Proceedings of the European conference on computer vision (ECCV). 2018. doi: https://doi.org/10.1007/978-3-030-01237-3_4
[9] He, K., Zhang, X., Ren, S., & Sun, J., “Deep residual learning for image recognition.“, Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770-778, 2016. doi: https://doi.org/10.1109/CVPR.2016.90
[10] T. -Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, "Feature Pyramid Networks for Object Detection.", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936-944, 2017. doi: https://doi.org/10.1109/CVPR.2017.106

The Effect of Accommodation Cue Manipulation at Stereoscopic Display on Binocular Fusion

SHVC-based V-PCC Content ISOBMFF Encapsulation and DASH Configuration Method