Hadoop-based Parallel Algorithm for Data Mining in Remote Sensing Images

doi:10.23940/ijpe.19.11.p4.28602870

Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (11): 2860-2870.doi: 10.23940/ijpe.19.11.p4.28602870

Previous Articles Next Articles

Hadoop-based Parallel Algorithm for Data Mining in Remote Sensing Images

Yanhua Wang, Yaqiu Liu^*, and Weipeng Jing

School of Information and Computer Engineering, Northeast Forestry University, Harbin, 150040, China

Submitted on ; Revised on ; Accepted on
Contact: * E-mail address: Yaqiuliu@126.com
About author:Yanhua Wang is a Ph.D. student in the School of Information and Computer Engineering at Northeast Forestry University. Her research interests include cloud computing and software engineering. Yaqiu Liu is a professor in the School of Information and Computer Engineering at Northeast Forestry University. His research interests include cloud computing and software engineering.Weipeng Jing is an associate professor in the School of Information and Computer Engineering at Northeast Forestry University. His research interests include cloud computing and software engineering.

Abstract

Abstract: As a typical distributed parallel computing model, cloud computing can greatly reduce the execution time of computing tasks. Remote sensing image data mining, an important part of data mining, plays a significant role in meteorological analysis and earthquake prediction. By constructing a Hadoop cloud computing platform, this paper studies the Hadoop-based parallel algorithm for remote sensing image data mining. In accordance with the Hadoop distributed computing framework, the parallel algorithm for remote sensing image data mining is realized through data preprocessing, image feature extraction, and clustering analysis. The main work of this paper includes image preprocessing, Hadoop-based parallelization of remote sensing image feature extraction, and a Hadoop-based parallel algorithm for remote sensing image data mining.

Key words: cloud computing, data mining, feature extraction

Yanhua Wang, Yaqiu Liu, and Weipeng Jing. Hadoop-based Parallel Algorithm for Data Mining in Remote Sensing Images [J]. Int J Performability Eng, 2019, 15(11): 2860-2870.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] M. S. Chen, J. Han,P. S. Yu, “Data Mining: An Overview from a Database Perspective,” IEEE Transactions on Knowledge and Data Engineering, Vol. 8, No. 6, pp. 866-883, June 2002
[2] M. H. Dunham, “Data Mining Introductory and Advanced Topics,” Tsinghua University Press, Beijing, China, 2005
[3] Y. T.Zhang and L. Gong, “Data Mining Principles and Technology,” Electronic Industry Press, Beijing, China, 2004
[4] Q. J.Wang and L. F. Xue, “Data Mining Technology and Its Application in Geoscience,” World Geology, Vol. 19, No. 3, pp. 235-239, March 2000
[5] Z. Y. Li, H. Z. Wang, W. Shao, J. Z. Li,H. Gao, “Repairing Data through Regular Expressions,” PVLDB, Vol. 9, No. 5, pp. 432-443, 2016
[6] R. Agrawal, T. Imielinski,A. Swami, “Database Mining: A Performance Perspective,” IEEE Transactions on Knowledge and Data Engineering, Vol. 5, No. 6, pp. 914-925, June 2002
[7] “The Apache Software Foundation. Apache Hadoop 2.9.2,” (http://hadoop.apache.org/docs/stable/, accessed November 13, 2018
[8] Q. L. Han, S. Liang,H. L. Zhang, “Mobile Cloud Sensing, Big Data, and 5G Networks Make an Intelligent and Smart World,” IEEE Network, Vol. 29, No. 2, pp. 40-45, 2015
[9] Y. H. Huang, “In-Depth Understanding of Big Data: Big Data Processing and Programming Practice,” Machinery Industry Press, Beijing, China, 2014
[10] G. Holmes, A. Donkin,I. H. Witten, “WEKA: A Machine Learning Workbench,” inProceedings of the 2nd Australian and New Zealand Conference on Intelligent Information Systems, pp. 357-361, Brisbane, Australia, December 1994
[11] D. Talia, P. Trunfio,O. Verta, “Weka4WS: A WSRF-Enabled Weka Toolkit for Distributed Data Mining on Grids,” inProceedings of European Conference on Principles and Practice of Knowledge Discovery in Databases, pp. 309-320, Berlin, Germany, September 2005
[12] G. Liu, B. Hou,Z. W. Zhai, “Hadoop Open Source Cloud Computing Platform,” Beijing University of Posts and Telecommunications Press, Beijing, China, 2011
[13] Q. He, F. Z. Zhuang, L. Zeng, W. Z. Zhao,Q. Tan, “PDMiner: A Cloud Computing based Parallel and Distributed Data Mining Toolkit Platform,” Scientia Sinica, Vol. 44, No. 7, pp. 871-885, July 2014
[14] L. J. Yang, “Application of Data Mining Technology in Weather Data based on Hadoop Cloud Platform,” Beijing University of Posts and Telecommunications, 2015
[15] W. B. Pan, “Research and Application of Parallel K-Means Meteorological Data Mining based on Cloud Computing,” Nanjing University of Information Science and Technology, 2013
[16] L. M. Bao, “Research on Parallelization of Dynamic K-Means Algorithm in Remote Sensing Image Mining,” Nanjing University of Posts and Telecommunications, 2017
[17] Y. W. Li, “Research on File Copy Storage Improvement and Small File Merge and Access Optimization on Hadoop Platform,” Wuhan University of Technology, 2015

Hadoop-based Parallel Algorithm for Data Mining in Remote Sensing Images

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended 0

[1]	P. Antony Seba and J. V. Bibal Benifa. Hybrid Outlier Detection Strategy and Weighted Decision Matrix Ordinal Classifier for CKD Severity Prediction [J]. Int J Performability Eng, 2023, 19(2): 144-154.
[2]	Sushant Jhingran, Mayank Kumar Goyal, and Nitin Rakesh. DQLC: A Novel Algorithm to Enhance Performance of Applications in Cloud Environment [J]. Int J Performability Eng, 2023, 19(12): 771-778.
[3]	Priti Kumari and Parmeet Kaur. An Adaptable Approach to Fault Tolerance in Cloud Computing [J]. Int J Performability Eng, 2023, 19(1): 43-54.
[4]	Sagnik Pal, Rutvik Patel, Vijayasherly V., and Ramani Selvanambi. Hashtag Recommendation System for Instagram Posts using Transfer Learning with EfficientNet and ALS Model [J]. Int J Performability Eng, 2022, 18(8): 552-558.
[5]	Divya Singhal, Laxmi Ahuja, and Ashish Seth. An Insight into Combating Security Attacks for Smart Grid [J]. Int J Performability Eng, 2022, 18(7): 512-520.
[6]	Roop Preet Kaur, Anshu Sharma, Inderpal Singh, and Rahul Malhotra. Deep Learning-Based Pneumonia Recognition from Chest X-Ray Images [J]. Int J Performability Eng, 2022, 18(5): 380-386.
[7]	Sukruta Pardeshi, chetana Khairnar, and Khalid Alfatmi. Analysis of Data Handling Challenges in Edge Computing [J]. Int J Performability Eng, 2022, 18(3): 176-187.
[8]	Sanjay Razdan, Himanshu Gupta, and Ashish Seth. A Multi-Layer Feed Forward Network Intrusion Detection System using Individual Component Optimization Methodology for Cloud Computing [J]. Int J Performability Eng, 2022, 18(11): 781-790.
[9]	Palak Girdhar, Prashant Johri, and Deepali Virmani. An Improved Empirical Hyper-Parameter Tuned Supervised Model for Human Activity Recognition based on Motion Flow and Deep Learning [J]. Int J Performability Eng, 2022, 18(11): 808-816.
[10]	D. Bhavana, K. Kishore Kumar, Medasani Bipin Chandra, P.V. Sai Krishna Bhargav, D. Joy Sanjanaa, and G. Mohan Gopi. Hand Sign Recognition using CNN [J]. Int J Performability Eng, 2021, 17(3): 314-321.
[11]	Gayathri D and S.P. Shantharajah. A Survey on Fusion of Internet of Things and Cloud Computing [J]. Int J Performability Eng, 2021, 17(11): 946-954.
[12]	D. Sakthivel and B. Radha. Adaptive Model to Detect Anomaly and Real-Time Attacks in Cloud Environment Using Data Mining Algorithm [J]. Int J Performability Eng, 2021, 17(10): 889-899.
[13]	Kun Li, Liwei Jia, and Xiaoming Shi. IPSOMC: An Improved Particle Swarm Optimization and Membrane Computing based Algorithm for Cloud Computing [J]. Int J Performability Eng, 2021, 17(1): 135-142.
[14]	Anita Agárdi, László Kovács, and Tamás Bányai. Using Time Series and Classification in Vehicle Routing Problem [J]. Int J Performability Eng, 2021, 17(1): 14-25.
[15]	Xiaoxiang Fan. Cloud Computing Task Scheduling based on Improved Bird Swarm Algorithm [J]. Int J Performability Eng, 2021, 17(1): 85-94.