Semantic Segmentation Method based on Super-Resolution

doi:10.23940/ijpe.20.05.p4.711719

Abstract

Abstract:

Convolutional neural network is an important method to solve most computer vision problems nowadays. Although increasing the computing cost and the scale of model will make most tasks achieve satisfactory results, the difficulty of increasing the computing cost and high-quality data also limit the increase of model scale. In this paper, when using neural network to segment the remote sensing image, aiming at the problem that the classification effect of the internal pixels of the target is not ideal, a multi-scale fusion structure about the dimension of the feature map is proposed as the classifier module of the model. In order to further improve the performance of semantic segmentation model, inspired by the Generative adversarial nets, combined with super-resolution, generative semantic segmentation architecture is proposed. In order to verify the effect of the two methods, the kappa coefficient was selected as the evaluation to conduct the semantic segmentation experiment of the remote sensing image of seaculture. With little to no increase in the scale of the model, the classification ability of the model is improved, and the effect is compared intuitively from the segmentation image.

Key words: semantic segmentation, generative, single image super-resolution, remote sensing image

Dulei Zheng, You Fu, Hao Zhang, Minghao Gao, and Jianzhi Yu. Semantic Segmentation Method based on Super-Resolution [J]. Int J Performability Eng, 2020, 16(5): 711-719.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 25.

1.	L. Cui, P. Zhang,J. Che, “Overview of Deep Neural Network based Classification Algorithms for Remote Sensing Images,” Computer Science, Vol. 45, No. S1, pp. 50-53, 2018
2.	Y. Zhang, H. T. Yang,C. H. Yuan, “A Survey of Remote Sensing Image Classification Methods,” Journal of Ordnance Equipment Engineering, Vol. 39, No. 8, pp. 108-111, 2018
3.	C. Peng, X. Y. Zhang, G. Yu, G. M. Luo,J. Sun, “Large Kernel Matters-Improve Semantic Segmentation by Global Convolutional Network,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4353-4361, 2017
4.	J. Long, E. Shelhamer,T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431-3440, 2015
5.	Y. C.Wei and Y. Zhao, “A Review on Image Semantic Segmentation based on DCNN,” Journal of Beijing Jiaotong University, Vol. 40, No. 4, pp. 82-91, 2016
6.	H. S. Zhao, J. P. Shi, X. J. Qi, X. G. Wang,J. Y. Jia, “Pyramid Scene Parsing Network,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881-2890, 2017
7.	O. Ronneberger, P. Fischer,T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” inProceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241, 2015
8.	V. Badrinarayanan, A. Kendall,R. Cipolla, “Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495, 2017
9.	Y. Lecun, L. Bottou, Y. Bengio,P. Haffner, “Gradient-based Learning Applied to Document Recognition,” Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998
10.	A. Krizhevsky, I. Sutskever,G. E. Hinton, “Imagenet Classification with Deep Convolutional Neural Networks,” inProceedings of the 25th International Conference on ImageNet Classification with Deep Convolutional Neural Netwoks, pp. 1097-1105, 2012
11.	K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv preprint arXiv:1409.1556, 2014
12.	C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., “Going Deeper with Convolutions,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015
13.	K. He, X. Y. Zhang, S. Q. Ren,J. Sun, “Deep Residual Learning for Image Recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016
14.	I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., “Generative Adversarial Nets,” inProceedings of the 27th International Conference on Neural Information Processing Systems, pp. 2672-2680, 2014
15.	X. C.Zhu and G. J. Tang, “A Survey on Generative Adversarial Networks in Image Processing,” Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition), Vol. 39, No. 3, pp. 1-12, 2019
16.	M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” arXiv preprint arXiv:1411.1784, 2014
17.	E. L. Denton, S. Chintala, A. Szlam,R. Fergus, “Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks,” inProceedings of Advances in Neural Information Processing Systems, pp. 1486-1494, 2015
18.	L. Zhu, Y. S. Chen, P. Ghamisi,J. A. Benediktsson, “Generative Adversarial Networks for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 56, No. 9, pp. 5046-5063, 2018
19.	F. Z. Nan, Y. R. Qian, Y. N. Xing,J. X. Zhao, “Survey of Single Image Super Resolution based on Deep Learning,”Application Research of Computers, 2019
20.	C. Dong, C. C. Loy, K. M. He,X. O. Tang, “Image Super-Resolution using Deep Convolutional Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, No. 2, pp. 295-307, 2016
21.	C. Dong, C. C. Loy,X. Tang, “Accelerating the Super-Resolution Convolutional Neural Network,” inProceedings of European Conference on Computer Vision, pp. 391-407, 2016
22.	G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, 2017
23.	T. Tong, G. Li, X. J. Liu,Q. Q. Gao, “Image Super-Resolution using Dense Skip Connections,” inProceedings of the IEEE International Conference on Computer Vision, pp. 4799-4807, 2017
24.	C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, et al., “Photo-Realistic Single Image Super-Resolution using a Generative Adversarial Network,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681-4690, 2017
25.	C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens,Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2826, 2016

[1]	Shou-Yu Lee, Yu-Sheng Chu, Tzu-Wei Hsu, I-Hsiang Yu, and W. Eric Wong. Enhanced Recognition Approach for Herb Medicine using YOLOv8 in Medical Information Systems [J]. Int J Performability Eng, 2024, 20(12): 713-722.
[2]	Ritu Gupta. Modelling and Comparative Assessment of Two Reliability Models of a Clinker Manufacturing System Working in Cement Plants [J]. Int J Performability Eng, 2024, 20(1): 18-23.
[3]	M. J. Delsey and J. V. Bibal Benifa. UWGAN-EnhaNet: Conditional Generative Adversarial Network Inspired Network for Enhancing the Quality of Underwater Images [J]. Int J Performability Eng, 2023, 19(8): 499-506.
[4]	Angel Arul Jothi J and Razia Sulthana A. A Review on the Literature of Fashion Recommender System using Deep Learning [J]. Int J Performability Eng, 2021, 17(8): 695-702.
[5]	Zhifeng Zhang, Xiao Cui, Pu Li, Jintao Jiang, and Xiaohui Ji. Hyperspectral Data Analysis based on Integrated Deep Learning [J]. Int J Performability Eng, 2020, 16(8): 1225-1234.
[6]	Xincheng Cao, Wanshan Liu, Bin Yao, Qixin Lan, Weifang Sun. Detection and Classification of Surface Defects of Magnetic Tile based on SE-U-Net [J]. Int J Performability Eng, 2020, 16(12): 1910-1920.
[7]	Qinggang Wu and Xueming Zhai. Remote Sensing Object Detection via an Improved YOLO Network [J]. Int J Performability Eng, 2020, 16(11): 1803-1813.
[8]	Qinggang Wu, Yilan Zhao, Qiuwen Zhang, and Bin Jiang. Remote Sensing Image Classification based on Fusion of ATLTP and Tamura Texture Features [J]. Int J Performability Eng, 2020, 16(1): 59-66.
[9]	Aili Wang, Ying Wang, Xiaoying Song, and Yuji Iwahori. Remote Sensing Image Super-Resolution Reconstruction based on Generative Adversarial Network [J]. Int J Performability Eng, 2019, 15(7): 1783-1791.
[10]	Guanyu Chen, Zhihua Cai, and Xiang Li. Classification of Remote Sensing Images based on Distributed Convolutional Neural Network Model [J]. Int J Performability Eng, 2019, 15(6): 1508-1517.
[11]	Guanyu Chen, Zhihua Cai, and Xiang Li. Recognition and Classification of High Resolution Remote Sensing Image based on Convolutional Neural Network [J]. Int J Performability Eng, 2018, 14(11): 2852-2863.