Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 2, pp.125-131, March, 2021

DOI: https://doi.org/10.5909/JBE.2021.26.2.125

Compression of CNN Using Low-Rank Approximation and CP Decomposition Methods

HyeonCheol Moon, Gihwa Moon, and Jae-Gon Kim

C.A E-mail: jgkim@kau.ac.kr

Abstract:

In recent years, Convolutional Neural Networks (CNNs) have achieved outstanding performance in the fields of computer vision such as image classification, object detection, visual quality enhancement, etc. However, as huge amount of computation and memory are required in CNN models, there is a limitation in the application of CNN to low-power environments such as mobile or IoT devices. Therefore, the need for neural network compression to reduce the model size while keeping the task performance as much as possible has been emerging. In this paper, we propose a method to compress CNN models by combining matrix decomposition methods of LR (Low-Rank) approximation and CP (Canonical Polyadic) decomposition. Unlike conventional methods that apply one matrix decomposition method to CNN models, we selectively apply two decomposition methods depending on the layer types of CNN to enhance the compression performance. To evaluate the performance of the proposed method, we use the models for image classification such as VGG-16, RestNet50 and MobileNetV2 models. The experimental results show that the proposed method gives improved classification performance at the same range of 1.5 to 12.1 times compression ratio than the existing method that applies only the LR approximation.



Keyword: NN, Neural Network Compression, Low-Rank Approximation, Canonical polyadic decomposition

Reference:
[1] S. Jung, C. Son, S. Lee, J. Han, Y. Kwak, and S. Hwang, “Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss,” In Proc. Computer Vision and Pattern Recognition (CVPR), 2019.
[2] W. Bailer, et al, “Text of ISO/IEC DIS 15938-17 Compression of Neural Networks for Multimedia Content Description and Analysis,” ISO/IEC/JTC1/SC29/WG04, N0016, Oct. 2020.
[3] X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Exteremey Efficient Convolutional Neural Network for Mobile Devices,” In Proc. Computer Vision and Patter Recognition (CVPR), 2018.
[4] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv preprint arXiv:1704.04861, 2017
[5] C. Aytekin, F. Cricri, T. Wang, E. Aksu, “Response to the Call for Proposals on Neural Network Compression: Training Highly Compressible Neural Networks,” ISO/IEC JTC1/SC29/WG11, m47379, Mar. 2019.
[6] H. Moon, H. Lee, and J. Kim, “Acceleration of CNN Model Using Neural Network Compression and its Performance Evaluation on Embedded Boards,” In Proc. KIBME Annual Fall Conf., Nov. 2019.
[7] S. Han, et al, “Deep Compression: Compressing Deep Neural Net- works with pruning, trained quantization and Huffman coding,” Computer Vision and Patter Recognition, In Proc. ICLR 2016, May 2016.
[8] M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up Convolu- tional Neural Networks with Low Rank Expansions,” In Proc. CVPR, Jun. 2014.
[9] V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lemptisky, “Sppeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition,” In Proc. CVPR, Jun. 2015.
[10] H. Moon, G. Moon, and J. Kim, “Compression of CNN Using Low-Rank Approximation and CP Decomposition Methods,” In Proc. KIBME Annual Fall Conf., Nov. 2020.
[11] H. Moon, J. Kim, S. Kim, S. Jang, and B. Choi, “KAU/KETI Response to the CE-1 on Neural Network Compression: CP Decomposition of Convolutional Layers (Method 5),” ISO/IEC JTC1/SC29/WG04, m55053, Oct. 2020.
[12] Large Scale Visual Recognition Challenge 2012 (ILSVRC 2012), [Available at Online] http://www.image-net.org/challenges/LSVRC/ 2012/

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved