Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 1, pp.26-38, January, 2021

DOI: https://doi.org/10.5909/JBE.2021.26.1.26

Deep Learning-based Keypoint Filtering for Remote Sensing Image Registration

Jun-Young Sung, Woo-Ju Lee, and Seoung-Jun Oh

C.A E-mail: sjoh@kw.ac.kr

Abstract:

In this paper, DLKF (Deep Learning Keypoint Filtering), the deep learning-based keypoint filtering method for the rapidization of the image registration method for remote sensing images is proposed. The complexity of the conventional feature-based image registration method arises during the feature matching step. To reduce this complexity, this paper proposes to filter only the keypoints detected in the artificial structure among the keypoints detected in the keypoint detector by ensuring that the feature matching is matched with the keypoints detected in the artificial structure of the image. For reducing the number of keypoints points as preserving essential keypoints, we preserve keypoints adjacent to the boundaries of the artificial structure, and use reduced images, and crop image patches overlapping to eliminate noise from the patch boundary as a result of the image segmentation method. the proposed method improves the speed and accuracy of registration. To verify the performance of DLKF, the speed and accuracy of the conventional keypoints extraction method were compared using the remote sensing image of KOMPSAT-3 satellite. Based on the SIFT-based registration method, which is commonly used in households, the SURF-based registration method, which improved the speed of the SIFT method, improved the speed by 2.6 times while reducing the number of keypoints by about 18%, but the accuracy decreased from 3.42 to 5.43. Became. However, when the proposed method, DLKF, was used, the number of keypoints was reduced by about 82%, improving the speed by about 20.5 times, while reducing the accuracy to 4.51.



Keyword: Computer Vision, Deep Learning, Image Registration, Remote Sensing Image, Image Segmentation

Reference:
[1] B. Zitova, J. Flusser, “Image registration methods: a survey” Image and Vision Computing, Vol.21, No.11, pp.977-1000, June 2003.
[2] S. Suri, P. Reinartz, “Mutual-information-based registration of terrasar-x and ikonos imagery in urban areas” IEEE Transactions on Geoscience and Remote Sensing, Vol. 48, No.2, pp.939-949, November 2009.
[3] J.P. Kern, M.S. Pattichis, “Robust multispectral image registration using mutual-information models” IEEE Transactions on Geoscience and Remote Sensing, Vol.45, No.5, pp.1494–1505, April 2007.
[4] H. Bay, A. Ess, T. Tuytelaars, L.V. Gool, “Speeded-up robust features (surf)” Computer Vision and Image Understanding, Vol.110, No3, pp.346-359, June 2008.
[5] D. Lowe, “Distinctive image features from scale-invariant keypoints” International Journal of Computer Vision, Vol.60, No.2, pp.91-110, January 2004.
[6] M.I. Patel, V.K. Thakar, "Speed Improvement in Image Registration using Maximum Likelihood based Mutual Information" International Conference on Advanced Computing and Communication Systems, Coimbatore, India, pp.1-3, 2015.
[7] L.C. Chiu, T.S. Chang, J.Y. Chen, N.Y.C. Chang, "Fast SIFT Design for Real-Time Visual Feature Extraction” IEEE Transactions on Image Processing, Vol. 22, No. 8, pp.3158-3167, August 2013.
[8] Y. LeCun, Y. Bengion and G. Hinton, "Deep learning," Nature, Vol. 521, pp.436-444, May 2015.
[9] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation” Medical Image Computing and Computer-Assisted Intervention, Vol.9351. pp.234-241, November 2015.
[10] C. Harris, M. Stephens, "A combined corner and edge detector" Alvey Vision Conference, Manchester, UK, pp. 147-151, 1988.
[11] T. Lindeberg, "Feature detection with automatic scale selection", International Journal of Computer Vision, Vol. 30, No.2, pp.79-116, Novemver 1998.
[12] K. Mikolajczyk, C. Schmid, "Indexing based on scale invariant interest points" International Conference on Computer Vision, Vancouver, Canada, pp.525-531, 2001.
[13] D. Lowe, "Object recognition from local scale-invariant features" International Conference on Computer Vision, Kerkyra, Greece, pp.1150-1157, 1999.
[14] T. Kadir, M. Brady, "Scale, saliency and image description" International Journal of Computer Vision, Vol. 45, No.2, pp.83-105, Novemver 2001.
[15] F. Jurie, C. Schmid, "Scale-invariant shape features for recognition of object categories" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp.90-96, 2004.
[16] L.M.J. Florack, B.M. ter Haar Romeny, J.J. Koenderink, M.A. Viergever, "General intensity transformations and differential invariants" Journal of Mathematical Imaging and Vision, Vol.4, No.2, pp.171-187, May 1994.
[17] F. Mindru, T. Tuytelaars, L. Van Gool, T. Moons, "Moment invariants for recognition under changing viewpoint and illumination" Computer Vision and Image Understanding, Vol.94 No.1–3, pp.3-27, April 2004.
[18] A. Baumberg, "Reliable feature matching across widely separated views" IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA, pp.774-781, 2000.
[19] W.T. Freeman, E.H. Adelson, "The design and use of steerable filters" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.13, No.9, pp.891-906, September 1991.
[20] G. Carneiro, A.D. Jepson, "Multi-scale phase-based local features" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp.736-743, 2003.
[21] Y. Ke, R. Sukthankar, "PCA-SIFT: a more distinctive representation for local image descriptors" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp.506-513, 2004.
[22] K. Mikolajczyk, C. Schmid, "A performance evaluation of local descriptors” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp.1615-1630, October 2003.
[23] H. Goncalves, J.A. Goncalves, L. Corte-Real, “Measures for an objective evaluation of the geometric correction process quality” IEEE Geoscience and Remote Sensing Letters, Vol.6, No.2, pp.292-296, April 2009.

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved