A Novel Image Inpainting Method for Object Removal based on Structure Sparsity

doi:10.23940/ijpe.18.11.p24.27772788

Int J Performability Eng ›› 2018, Vol. 14 ›› Issue (11): 2777-2788.doi: 10.23940/ijpe.18.11.p24.27772788

Previous Articles Next Articles

A Novel Image Inpainting Method for Object Removal based on Structure Sparsity

Lei Zhang^* and Minhui Chang

School of Mathematics and Information Technology, Yuncheng University, Yuncheng, 044000, China

Submitted on ;
Contact: * E-mail address: inpainting@126.com
About author:Lei Zhang received his B.S. degree in computer science and technology from Shanxi Normal University, Linfen, China in 2003, his M.S. degree in computer application technology from Shanxi University, Taiyuan, China in 2008, and his Ph.D. in computer application technology from Northwest University, Xi'an, China in 2016. His research interests include image processing, pattern recognition, and machine learning. E-mail: inpainting@126.com. Minhui Chang received her B.S. degree and M.S. degree in basic mathematics from Shaanxi Normal University, Xi'an, China in 2003 and 2009, respectively. Her research interests include wavelet analysis and machine learning. E-mail: changminhui1120@126.com

Abstract

Abstract: In the traditional image inpainting method for object removal, for each target patch, the entire source region must be traversed to search for the exemplar patch, which may make the restoration process time-consuming and affect the restoration efficiency. Even worse, the target patch may be replaced by an inappropriate exemplar patch during the process, which will introduce some unexpected objects in the restored image and make the result unable to meet the requirements of visual consistency. In view of these problems, we propose a novel image inpainting method for object removal based on structure sparsity. First, we calculate the structure sparsity of the target patch, and then identify the local characteristics of the region where the target patch is located. Then, we set different search regions for the target patches according to different regional characteristics. Finally, we find the exemplar patch in the search region and restore the target patch. Experiments on a number of natural images show that the proposed method can reduce the restoration time and improve the restoration efficiency. Additionally, it can prevent the mismatch to some extent and improve the restoration effect.

Key words: Image Inpainting, object removal, structure sparsity, search region

Lei Zhang and Minhui Chang. A Novel Image Inpainting Method for Object Removal based on Structure Sparsity [J]. Int J Performability Eng, 2018, 14(11): 2777-2788.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] C. Guillemot and O. M. Le, “Image Inpainting: Overview and Recent Advances,” IEEE Signal Processing Magazine, Vol. 31, No. 1, pp. 127-144, 2014
[2] W. Wang and Y. Jia, “Damaged Region Filling and Evaluation by Symmetrical Exemplar-based Image Inpainting for Thangka,”Eurasip Journal on Image & Video Processing, Vol. 38, pp. 1-13, 2017
[3] N. Bhardwaj and S. Agarwal, “Review of Image Defect Detection and Inpainting Techniques and Scope for Improvements,” International Journal of Emerging Sciences, Vol. 5, No. 1, pp. 49-67, 2015
[4] I. C. Chang, J. C. Yu,C. C. Chang, “A Forgery Detection Algorithm for Exemplar-based Inpainting Images Using Multi-region Relation,” Image and Vision Computing, Vol. l, No. 1, pp. 57-71, 2013
[5] L. Zhang, B. Kang, B. Liu,Z. Bao, “A New Inpainting Method for Object Removal based on Patch Local Feature and Sparse Representation,” International Journal of Innovative Computing, Information and Control, Vol. 12, No. 1, pp. 113-124, 2016
[6] M. Bertalmio, G. Sapiro, V. Caselles,C. Ballester, “Image Inpainting,” inProceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417-424, 2000
[7] T. F. Chan, J. Shen,H. M. Zhou, “Total Variation Wavelet Inpainting,” Journal of Mathematical Imaging and Vision, Vol. 25, No. 1, pp. 107-125, 2006
[8] T. F.Chan and J. Shen, “Nontexture Inpainting by Curvature-Driven Diffusions,” Journal of Visual Communication and Image Representation, Vol. 12, No. 4, pp. 436-449, 2001
[9] L. Zhang, B. Kang, B. Liu,F. Zhang, “Image Inpainting based on Exemplar and Sparse Representation,” International Journal of Signal Processing, Image Inpainting and Pattern Recognition, Vol. 9, No. 9, pp. 177-188, 2016
[10] L. Dai, D. H. Jiang, B. Ding,J. K. Hahn, “Improved Digital Image Restoration Algorithm based on Criminisi,” Journal of Digital Information Management, Vol. 14, No. 5, pp. 302-310, 2016
[11] M. S. Ishi, L. Singh,M. Agrawal, “A Review on Image Inpainting to Restore Image,” IOSR Journal of Computer Engineering, Vol. 13, No. 6, pp. 8-13, 2013
[12] A. Criminisi, P. Pérez,K. Toyama, “Region Filling and Object Removal by Exemplar-based Image Inpainting,” IEEE Transactions on Image Processing, Vol. 13, No. 9, pp. 1200-1212, 2004
[13] A. Wong and J. Orchard, “A Nonlocal-means Approach to Exemplar-based Inpainting,” inProceedings of 15th IEEE International Conference on Image Processing, pp. 2600-2603, 2008
[14] A. A.Efros and T. K. Leung, “Texture Synthesis by Non-parametric Sampling,” inProceedings of IEEE International Conference on Computer Vision, pp. 1033-1038, 1999
[15] L. J. Deng, T. Z. Huang,X. L. Zhao, “Exemplar-based Image Inpainting Using a Modified Priority Definition,” PLOS One, Vol. 10, No. 10, pp. 1-18, 2015
[16] J. Lin, D. Deng, J. Yan,X. Lin, “Self-Adaptive Group based Sparse Representation for Image Inpainting,” Journal of Computer Applications, Vol. 37, No. 4, pp. 1169-1173, 2017
[17] M. Aharon, M. Elad,A. Bruckstein, “K-SVD: An Algorithm for Designing Over-Complete Dictionaries for Sparse Representation,” IEEE Transactions on Signal Processing, Vol. 54, No. 11, pp. 4311-4322, 2006
[18] M. Elad, J. L. Starck, P. Querre,D. L. Donoho, “Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis (MCA),” Applied and Computational Harmonic Analysis, Vol. 19, No. 3, pp. 340-358, 2005
[19] Y. Huang, K. Li,M. Yang, “An Improved Image Inpainting Algorithm based on Image Segmentation,”Procedia Computer Science, No. 107, pp. 796-801, 2017
[20] V. K.Alilou and F. Yaghmaee, “Exemplar-based Image Inpainting Using Svd-based Approximation Matrix and Multi-Scale Analysis,” Multimedia Tools and Applications, Vol. 76, No. 5, pp. 7213-7234, 2017
[21] J. Wang, K. Lu, D. Pan,B. K. Bao, “Letters: Robust Object Removal with an Exemplar-based Image Inpainting Approach,”Neurocomputing, No. 123, pp. 150-155, 2014
[22] X. Xi, F. Wang,Y. Liu, “Improved Criminisi Algorithm based on a New Priority Function with the Gray Entropy,” inProceedings of 2013 9th International Conference on Computational Intelligence and Security, 2013
[23] Q. Zhang, J. Lin,Y. Liu, “A Fast Image Inpainting Scheme Using Local Average Gray Entropy,” Computer Applications and Software, Vol. 31, No. 10, pp. 206-208, 2014
[24] W. Zou, Z. Zhou,Y. Wang, “Image Inpainting Algorithm based on Non-Subsampled Contourlet Transform,” Journal of Computer Applications, Vol. 37, No. 2, pp. 553-558, 2017
[25] A. Nan and X. Xi, “An Improved Criminisi Algorithm based on a New Priority Function and Updating Confidence,” inProceedings of International Conference on Biomedical Engineering and Informatics, pp. 885-889, 2014
[26] Y. F. Liu, F. L. Wang,X. Y. Xi, “Enhanced Algorithm for Exemplar-based Image Inpainting,” in Proceedings of 2013 9th International Conference on Computational Intelligence and Security (CIS), pp. 209-213, 2013
[27] B. Shen, W. Hu, Y. Zhang,Y. J. Zhang, “Image Inpainting via Sparse Representation,” inProceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 697-700, 2009
[28] Z. Xu and J. Sun, “Image Inpainting by Patch Propagation Using Patch Sparsity,” IEEE Transactions on Image Processing, Vol. 19, No. 5, pp. 1153-1165, 2010
[29] P. Arbelaez, M. Maire, C. Fowlkes,J. Malik, “Contour Detection and Hierarchical Image Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 5, pp. 898-916, 2011

A Novel Image Inpainting Method for Object Removal based on Structure Sparsity

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended 0