Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 2, pp.167-174, March, 2021


Deep Learning-based Real-Time Super-Resolution Architecture Design

Saehyun Ahn and Suk-Ju Kang

C.A E-mail:


Recently, deep learning technology is widely used in various computer vision applications, such as object recognition, classification, and image generation. In particular, the deep learning-based super-resolution has been gaining significant performance improvement. Fast super-resolution convolutional neural network (FSRCNN) is a well-known model as a deep learning-based super-resolution algorithm that output image is generated by a deconvolutional layer. In this paper, we propose an FPGA-based convolutional neural networks accelerator that considers parallel computing efficiency. In addition, the proposed method proposes Optimal-FSRCNN, which is modified the structure of FSRCNN. The number of multipliers is compressed by 3.47 times compared to FSRCNN. Moreover, PSNR has similar performance to FSRCNN. We developed a real-time image processing technology that implements on FPGA.

Keyword: hardware accelerator, super-resolution, FPGA, deep learning

1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” In NIPS, pp. 1097-1105, 2012.
[2] A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” In IJCNN, pp. 2047-2052, 2005.
[3] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” In CVPR, pp. 580-587, 2014.
[4] R. Girshick, “Fast R-CNN,” In ICCV, 2015.
[5] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” In NIPS, pp. 91-99, 2015.
[6] K. He, X. Zhang, S. Ren, and J. Sun, “Mask R-CNN,” In ICCV, pp. 2980-2988, 2017.
[7] C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” In Proc. ECCV, 2014. pp.184-199.
[8] Dong Chao, Chen Change Loy, and Xiaoou Tang, “Accelerating the super-resolution convolutional neural network,” In ECCV, 2016.
[9] A. Radford et al., “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv, 2015.
[10] S. Williams et al., “Roofline: an insightful visual performance model for multicore architectures,” Commun, ACM, 52(4):65-76, Apr. 2009.
[11] J.-W. Chang, K.-W. Kang, and S.-J Kang, “SDCNN: An efficient sparse deconvolutional neural network accelerator on FPGA,” Proceedings of Design, Automation & Test in Europe (DATE), March. 2019.
[12] Dong C., Loy C. C., He K., and Tang X., “Image superresolution using deep convolutional networks,” In TPAMI, pp.295-307, 2015.
[13] J. Kim , J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” In CVPR, 2016.
[14] A. Yazdanbakhsh, K. Samadi, N. S. Kim, and H. Esmaeilzadeh, “GANAX: A unified mimd-simd acceleration for generative adversarial networks,” In ISCA, pp. 650-661, 2018.
[15] M. Song, J. Zhang, H. Chen, and T. Li, “Towards efficient microarchitectural design for accelerating unsupervised gan-based deep learning,” In HPCA, pp. 66-77, 2018.
[16] D. Xu, K. Tu, Y. Wang, C. Liu, B. He, and H. Li, “FCN-engine: Accelerating deconvolutional layers in classic cnn processors,” In ICCAD, 2018.


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved