Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 1, pp.3-13, January, 2021

DOI: https://doi.org/10.5909/JBE.2021.26.1.3

Screen-shot Image Demorieing Using Multiple Domain Learning

Hyunkook Park, An Gia Vien, and Chul Lee

C.A E-mail: chullee@dongguk.edu

Abstract:

We propose a moire artifacts removal algorithm for screen-shot images using multiple domain learning. First, we estimate clean preliminary images by exploiting complementary information of the moire artifacts in pixel value and frequency domains. Next, we estimate a clean edge map of the input moire image by developing a clean edge predictor. Then, we refine the pixel and frequency domain outputs to further improve the quality of the results using the estimated edge map as the guide information. Finally, the proposed algorithm obtains the final result by merging the two refined results. Experimental results on a public dataset demonstrate that the proposed algorithm outperforms conventional algorithms in quantitative and qualitative comparison.



Keyword: Moire artifacts removal, demoireing, convolutional neural network (CNN), multiple domain learning

Reference:
[1] M. Schöberl, W. Schnurrer, A. Oberdörster, S. Fössel, and A. Kaup, “Dimensioning of optical birefringent anti-alias filters for digital cameras,” in Proceeding of IEEE International Conference on Image Processing, Sep. 2010.
[2] I. Pekkucuksen and Y. Altunbasak, “Multiscale gradients based color filter array interpolation,” IEEE Transactions Image Processing, vol, 22, no. 1, pp. 157-165, Jan. 2013.
[3] Y. Sun, Y. Yu, and W. Wang, “Moiré photo restoration using multiresolution convolutional neural networks,” IEEE Transactions Image Processing, vol. 27, no. 8, pp. 4160-4172, Aug. 2018.
[4] X. Cheng, Z. Fu, and J. Yang, “Multi-scale dynamic feature encoding network for image demoiréing,” in Proceedings of IEEE/CVF International Conference on Computer Vision Workshops, Oct. 2019, pp. 3486–3493.
[5] B. He, C. Wang, B. Shi, and L. Duan, “Mop moiré patterns using MopNet,” in Proceedings of IEEE International Conference on Computer Vision, Oct./Nov. 2019, pp. 2424-2432.
[6] B. Zheng, S. Yuan, G. Slabaugh, and A. Leonardis, “Image demoireing with learnable bandpass filters,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2020, pp. 3636–3645.
[7] L. Cavigelli, P. Hager, and L. Benini, “CAS-CNN: A deep convolutional neural network for image compression artifact suppression,” in Proceedings of International Joint Conference on Neural Networks, May 2017, pp. 752–759.
[8] A. G. Vien, H. Park, and C. Lee, “Dual-domain deep convolutional neural networks for image demoireing,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Jun. 2020, pp. 1934–1942.
[9] J. Yang, F. Liu, H. Yue, X. Fu, C. Hou, and F. Wu, “Textured image demoireing via signal decomposition and guided filtering,” IEEE Transactions Image Processing, vol. 26, no. 7, pp. 3528–3541, Jul. 2017.
[10] B. He, C. Wang, B. Shi, and L.-Y. Duan, “FHDe2Net: Full high definition demoireing network.” in Proceedings of European Conference on Computer Vision, Sep. 2020, pp. 713-729.
[11] B. Liu, X. Shu, and X. Wu, “Demoireing of camera-captured screen images using deep convolutional neural network,” arXiv preprint arXiv:1804.03809, 2018.
[12] P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, “Multi-level wavelet CNN for image restoration,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Jun. 2018, pp. 886–895.
[13] B. Zheng, Y. Chen, X. Tian, F. Zhou, and X. Liu, “Implicit dual-domain convolutional network for robust color image compression artifact reduction,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 11, pp. 3982-3994, Nov. 2020.
[14] X. Luo, J. Zhang, M. Hong, Y. Qu, Y. Xie, and C. Li, “Deep wavelet network with domain adaptation for single image demoireing,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Jun. 2020, pp. 1687-1694.
[15] L. Liu, J. Liu, S. Yuan, G. Slabaugh, A. Leonardis, W. Zhou, and Q. Tian, “Wavelet-based dual-branch network for image demoiréing,” in Proceedings of European Conference on Computer Vision, Aug. 2020, pp 86-102.
[16] X. Cheng, Z. Fu, and J. Yang, “Multi-scale dynamic feature encoding network for image demoireing,” in Proceedings of IEEE/CVF International Conference on Computer Vision Workshops, Oct. 2019, pp. 3486–3493.
[17] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image super-resolution,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 2472-2481.
[18] W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video superresolution using an efficient sub-pixel convolutional neural network,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2016, pp. 1874–1883.
[19] F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” in Proceedings of International Conference Learning Representations, May 2016.
[20] S. Yuan, R. Timofte, G. Slabaugh, and A. Leonardis, “AIM 2019 challenge on image demoireing: Dataset and study,” in Proceedings of IEEE/CVF International Conference on Computer Vision Workshops, Oct. 2019, pp. 3526-3533.
[21] G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2017, pp. 2261-2269.
[22] T. Tong, G. Li, X. Liu, and Q. Gao, “Image super-resolution using dense skip connections,” in Proceedings of IEEE International Conference on Computer Vision, Jul. 2017, pp. 4809-4817.
[23] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” in Proceedings of European Conference on Computer Vision, Sep. 2018, pp. 3-19.
[24] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Proceedings of Medical Image Computing and Computer-Assisted Intervention, Nov. 2015, pp. 234.241.
[25] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks.” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jul. 2018, pp. 7794-7803.
[26] I. Loshchilov and F. Hutter, Decoupled weight decay regularization. in Proceedings in International Conference on Learning Representations, 2018.
[27] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved