Image inpainting based on feature clustering and locality-sensitive sparse representation

Xue Juntao; Ni Chenyang; Yang Sixue

doi:10.3788/IRLA201847.1126001

A novel image inpainting method based on sparse representation which combined image clustering and dictionary learning was proposed to solve the problems of long iteration time, bad adaptation and non-ideal results when using one single dictionary. Firstly, the broken image was divided into blocks and generated index matrix. Then Steering Kernel Regression Weight (SKRW) algorithm was used for image clustering. By exploring the inner structures of image and the information of intact area, blocks were sorted into categories based on their similarities of SKRW. Then each category had their own overcomplete dictionary by self-adaptive locality-sensitive dictionary learning. By building a self-adaptive local adaptor, the rate of convergence and the adaptability of sparse dictionary were improved. Multi-dictionaries were matched with different image structures, so the image would have a more accurate sparse representation. The dictionaries were updated until convergence, along with sparse coefficients as well. The image was finally restored after replacing patches back. Experimental results show that the proposed algorithm can repair the damaged images better than the state-of-the-art algorithms in both visual effect and objective evaluations. In addition, the time consumption is greatly reduced in comparison with the other algorithms.

Image inpainting based on feature clustering and locality-sensitive sparse representation

1. School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China

Received Date: 2018-06-05

Rev Recd Date: 2018-07-03

Publish Date: 2018-11-25

Key words:

Abstract: A novel image inpainting method based on sparse representation which combined image clustering and dictionary learning was proposed to solve the problems of long iteration time, bad adaptation and non-ideal results when using one single dictionary. Firstly, the broken image was divided into blocks and generated index matrix. Then Steering Kernel Regression Weight (SKRW) algorithm was used for image clustering. By exploring the inner structures of image and the information of intact area, blocks were sorted into categories based on their similarities of SKRW. Then each category had their own overcomplete dictionary by self-adaptive locality-sensitive dictionary learning. By building a self-adaptive local adaptor, the rate of convergence and the adaptability of sparse dictionary were improved. Multi-dictionaries were matched with different image structures, so the image would have a more accurate sparse representation. The dictionaries were updated until convergence, along with sparse coefficients as well. The image was finally restored after replacing patches back. Experimental results show that the proposed algorithm can repair the damaged images better than the state-of-the-art algorithms in both visual effect and objective evaluations. In addition, the time consumption is greatly reduced in comparison with the other algorithms.

HTML

Reference (18)

[1]	Elad M, Figueiredo M A T, Ma Y. On the role of sparse and redundant representations in image processing[J]. Proceedings of the IEEE, 2010, 98(6):972-982.
[2]	Michal Aharon, Michael Elad, Alfred Bruckstein. K-SVD:An algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.
[3]	Beckouche J, Starck J L, Fadili J. Astronomical image denoising using dictionary learning[J]. Astronomy Astrophysics, 2013, 556(7):132.
[4]	Hu Gaolong, Xiong Ling. Criminisi-based sparse representation for image inpainting[C]//2017 IEEE Third International Conference on Multimedia Big Data, 2017:389-393.
[5]	Huang Linjiang, Liu Hong, Tao Shaojie. An improved inpainting algorithm based on K-SVD dictionary[J]. Journal of Anhui University(Natural Science Edition), 2013, 37(3):69-74. (in Chinese)
[6]	Han Yulan, Zhao Yongping, Wang Qisong, et al. Reconstruction of super resolution for noise image under the sparse representation[J]. Optics and Precision Engineering, 2017, 25(6):1619-1626. (in Chinese)
[7]	Xu Kai, Wang Nannan, Gao Xinbo. Image inpainting based on sparse representation with dictionary pre-clustering[C]//7th Chinese Conference on Pattern Recognition, 2016:245-258.
[8]	Mairal Julien, Bach Francis, Ponce Jean, et al. Online dictionary learning for sparse coding[C]//26th Annual International Conference on Machine Learning, 2009:689-696.
[9]	Naderahmadian Yashar, Beheshti Soosan, Tinati Mohammad Ali. Correlation based online dictionary learning algorithm[J]. IEEE Transactions on Signal Processing, 2015, 64(3):592-602.
[10]	Marquez M A, Mojica E, Arguello H. Data sinogram sparse reconstruction based on steering kernel regression and filtering strategies[C]//SPIE, 2016, 9847:98470Z.
[11]	Sahoo S K, Makur A. Replacing K-SVD with SGK:Dictionary training for sparse representation of images[C]//2015 IEEE International Conference on Digital Signal Processing, 2015(2):614-617.
[12]	Verhack R, Sikora T, Lange, L, et al. A universal image coding approach using sparse steered Mixture-of-Experts regression[C]//2016 IEEE International Conference on Image Processing, 2016:2142-2146.
[13]	Marquez M A, Mojica E, Arguello H. Data sinogram sparse reconstruction based on steering kernel regression and filtering strategies[C]//SPIE, 2016, 9847:98470Z.
[14]	Zhang Kaibing, Gao Xinbo. Single image super-resolution with non-local means and steering Kernel regression[J]. IEEE Transactions on Image Processing, 2012, 21(11):4544-4556.
[15]	Wei Chiapo, Chao Yuwei, Yeh Yiren, et al. Locality-sensitive dictionary learning for sparse representation based classification[J]. Pattern, 2013, 46(5):1277-1287.
[16]	Guo Jun, Guo Yangqing, Li Yi, et al. Locality sensitive discriminative dictionary learning[C]//IEEE International Conference on Image Processing, 2015:1558-1562.
[17]	Rao T J V S, Rao M V G, Aswini T V N L. Image inpainting with group based sparse representation using self adaptive dictionary learning[C]//International Conference on Signal Processing Communication Engineering Systems, 2015:301-305.
[18]	Zhang Lin, Zhang Lei, Mou Xuanqin, et al. FSIM:a feature similarity index for image qualtiy assessment[J]. IEEE Transactions on Image Processing, 2011, 20(8):2378-2386.

Image inpainting based on feature clustering and locality-sensitive sparse representation

doi: 10.3788/IRLA201847.1126001

Abstract

References

Proportional views

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Related

Proportional views