Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 4, pp.419-428, July, 2021

DOI: https://doi.org/10.5909/JBE.2021.26.4.419

Shadow Removal based on the Deep Neural Network Using Self Attention Distillation

Jinhee Kim and Wonjun Kim

C.A E-mail: wonjkim@konkuk.ac.kr

Abstract:

Shadow removal plays a key role for the pre-processing of image processing techniques such as object tracking and detection. With the advances of image recognition based on deep convolution neural networks, researches for shadow removal have been actively conducted. In this paper, we propose a novel method for shadow removal, which utilizes self attention distillation to extract semantic features. The proposed method gradually refines results of shadow detection, which are extracted from each layer of the proposed network, via top-down distillation. Specifically, the training procedure can be efficiently performed by learning the contextual information for shadow removal without shadow masks. Experimental results on various datasets show the effectiveness of the proposed method for shadow removal under real world environments.



Keyword: shadow removal, shadow detection, convolutional neural networks, self attention distillation

Reference:
[1] G. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 59–68, Nov. 2006.
[2] R. Guo, Q. Dai, and D. Hoiem, “Paired regions for shadow detection and removal,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, pp. 2956–2967, Oct. 2013.
[3] J. Wang, X. Li, and J. Yang, “Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 1788–1797.
[4] X. Hu, C. Fu, L. Zhu, J. Qin, and P. Heng, “Direction-aware spatial context features for shadow detection and removal,'' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 11, pp. 2795-2808, May. 2019.
[5] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Advances in Neural Information Processing Systems, Dec. 2017, pp. 1097–1105.
[6] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 1493–1500.
[7] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Representation, May 15, 2015, pp. 1–14.
[8] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image super-resolution,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 2472–2481.
[9] L. Qu, J. Tian, S. He, Y. Tang, and R. W. Lau, “DeshadowNet: A multi-context embedding deep network for shadow removal,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 4067–4075.
[10] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. Devito, Lin, A. Desmaison, L. Antiga, A. Lerer, “Automatic differentiation in pytorch”. in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), pp. 1-4, 2017.
[11] H. Gong and D. P. Cosker, “Interactive shadow removal and ground truth for variable scene categories,” in Proc. British Machine Vision Conference, Sep. 2014, pp. 1–11.
[12] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), Dec. 2015, pp. 1-9.
[13] F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, “Residual attention network for image classification,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 3156–3164.
[14] S. Zagoruyko and N. Komodakis, “Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer,” in Proc. International Conference on Learning Representations, Sep. 2017, pp. 1–13.

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved