Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 1, pp.79-87, January, 2021


The Analysis of Semi-supervised Learning Technique of Deep Learning-based Classification Model

Jae Hyeon Park and Sung In Cho

C.A E-mail:


In this paper, we analysis the semi-supervised learning (SSL), which is adopted in order to train a deep learning-based classification model using the small number of labeled data. The conventional SSL techniques can be categorized into consistency regularization, entropy-based, and pseudo labeling. First, we describe the algorithm of each SSL technique. In the experimental results, we evaluate the classification accuracy of each SSL technique varying the number of labeled data. Finally, based on the experimental results, we describe the limitations of SSL technique, and suggest the research direction to improve the classification performance of SSL.

Keyword: convolutional neural network, image classification, semi-supervised learning

[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Adv. Neural Inf. Process. Syst., pp. 1097–1105, 2012.
[2] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent., pp. 1–14, 2015.
[3] C. Szegedy et al., “Going deeper with convolutions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–9, 2015.
[4] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 770-778, 2016.
[5] P. Bachman, O. Alsharif, and D. Precup, “Learning with pseudo ensembles,” in Proc. Advances Neural Inf. Process. Syst., pp. 3365–3373, 2014.
[6] M. Sajjadi, M. Javanmardi, and T. Tasdizen, “Regularization with stochastic transformations and perturbations for deep semi-supervised learning,” in Proc. 30th Int. Conf. Neural Inf. Process. Syst., pp. 1171–1179, 2016.
[7] S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” in Proc. Int. Conf. Learn. Represent., pp. 1-13, 2017.
[8] A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” in Proc. Adv. Neural Inf. Process. Syst., pp. 1195–1204, 2017.
[9] T. Miyato, S.-I. Maeda, M. Koyama, and S. Ishii, ‘‘Virtual adversarial training: A regularization method for supervised and semi-supervised learning,’’ IEEE Trans. Pattern Anal. Mach. Intell., Vol. 41, No. 8, pp. 1979–1993, Aug 2019.
[10] Y. Grandvalet and Y. Bengio, “Semi-supervised learning by entropy minimization,” in Proc. Adv. Neural Inf. Process. Syst., pp. 529–536, 2004.
[11] D.-H. Lee, “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in Proc. Workshop Challenges Represent. Learn. (ICML), pp. 2-7, 2013.
[12] A. Oliver, A. Odena, C. Raffel, E. Cubuk, and I. Goodfellow, “Realistic Evaluation of Deep Semi-Supervised Learning Algorithms,” in Adv. in Neural Inf. Process. Syst., pp. 3235-3246, 2018.
[13] A. Krizhevsky and G. Hinton, “Learning Multiple Layers of Features from Tiny Images,” technical report, Univ. of Toronto, 2009.
[14] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[15] S. Zagoruyko and N. Komodakis, “Wide residual networks,” in Proc. Brit. Mach. Vis. Conf., pp. 87.1–87.12, 2016.
[16] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition.” In Proc. IEEE Conf. Comput. Vis. Pattern Rcognit., pp. 770-778, 2016.
[17] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. Int. Conf. Mach. Learn., pp. 448–456, 2015.
[18] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in Proc. ICML, Vol. 30, No. 1, p. 3, Jun. 2013.


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved