research-article

Noise-robust semi-supervised learning via fast sparse coding

Authors:

Liwei WangAuthors Info & Claims

Pattern Recognition, Volume 48, Issue 2

Pages 605 - 612

https://doi.org/10.1016/j.patcog.2014.08.019

Published: 01 February 2015 Publication History

Abstract

This paper presents a novel noise-robust graph-based semi-supervised learning algorithm to deal with the challenging problem of semi-supervised learning with noisy initial labels. Inspired by the successful use of sparse coding for noise reduction, we choose to give new L1-norm formulation of Laplacian regularization for graph-based semi-supervised learning. Since our L1-norm Laplacian regularization is explicitly defined over the eigenvectors of the normalized Laplacian matrix, we formulate graph-based semi-supervised learning as an L1-norm linear reconstruction problem which can be efficiently solved by sparse coding. Furthermore, by working with only a small subset of eigenvectors, we develop a fast sparse coding algorithm for our L1-norm semi-supervised learning. Finally, we evaluate the proposed algorithm in noise-robust image classification. The experimental results on several benchmark datasets demonstrate the promising performance of the proposed algorithm. HighlightsWe propose novel semi-supervised learning based on fast sparse coding.Our algorithm achieves promising results in noise-robust image classification.Our algorithm can readily be extended to many other challenging problems.

References

[1]

K. Lu, J. Zhao, D. Cai, An algorithm for semi-supervised learning in image retrieval, Pattern Recognit., 39 (2006) 717-720.

Digital Library

[2]

D. Xu, S. Yan, Semi-supervised bilinear subspace learning, IEEE Trans. Image Process., 18 (2009) 1671-1676.

[3]

Z. Lu, H. Ip, Combining context, consistency, and diversity cues for interactive image categorization, IEEE Trans. Multimed., 12 (2010) 194-203.

Digital Library

[4]

R. Fergus, Y. Weiss, A. Torralba, Semi-supervised learning in gigantic image collections, in: Advances in Neural Information Processing Systems, vol. 22, 2010, pp. 522-530.

[5]

J. Tang, S. Yan, R. Hong, G.-J. Qi, T.-S. Chua, Inferring semantic concepts from community-contributed images and noisy tags, in: Proceedings of ACM Multimedia, 2009, pp. 223-232.

[6]

J. Liu, M. Li, Q. Liu, H. Lu, S. Ma, Image annotation via graph learning, Pattern Recognit., 42 (2009) 218-228.

Digital Library

[7]

A. Blum, T. Mitchell, Combining labeled and unlabeled data with co-training, in: Proceedings of COLT, 1998.

[8]

X. Zhu, Z. Ghahramani, J. Lafferty, Semi-supervised learning using Gaussian fields and harmonic functions, in: Proceedings of ICML, 2003, pp. 912-919.

[9]

D. Zhou, O. Bousquet, T. Lal, J. Weston, B. Sch�lkopf, Learning with local and global consistency, in: Advances in Neural Information Processing Systems, vol. 16, 2004, pp. 321-328.

[10]

M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization, J. Mach. Learn. Res., 7 (2006) 2399-2434.

Digital Library

[11]

F. Wang, C. Zhang, Label propagation through linear neighborhoods, IEEE Trans. Knowl. Data Eng., 20 (2008) 55-67.

Digital Library

[12]

S. Yan, H. Wang, Semi-supervised learning by sparse representation, in: Proceedings of SDM, 2009, pp. 792-801.

[13]

B. Cheng, J. Yang, S. Yan, Y. Fu, T. Huang, Learning with �1-graph for image analysis, IEEE Trans. Image Process., 19 (2010) 858-866.

Digital Library

[14]

W. Liu, J. He, S.-F. Chang, Large graph construction for scalable semi-supervised learning, in: Proceedings of ICML, 2010, pp. 679-686.

[15]

B.-K. Bao, B. Ni, Y. Mu, S. Yan, Efficient region-aware large graph construction towards scalable multi-label propagation, Pattern Recognit., 44 (2011) 598-606.

Digital Library

[16]

L. Zhuang, H. Gao, Z. Lin, Y. Ma, X. Zhang, N. Yu, Non-negative low rank and sparse graph for semi-supervised learning, in: Proceedings of CVPR, 2012, pp. 2328-2335.

[17]

S. Sun, Z. Hussain, J. Shawe-Taylor, Manifold-preserving graph reduction for sparse semi-supervised learning, Neurocomputing, 124 (2014) 13-21.

Digital Library

[18]

R. Ando, T. Zhang, Learning on graph with Laplacian regularization, in: Advances in Neural Information Processing Systems, vol. 19, 2007, pp. 25-32.

[19]

Z. Lu, Y. Peng, Exhaustive and efficient constraint propagation, Int. J. Comput. Vis., 103 (2013) 306-325.

[20]

M. Elad, M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Trans. Image Process., 15 (2006) 3736-3745.

[21]

J. Mairal, M. Elad, G. Sapiro, Sparse representation for color image restoration, IEEE Trans. Image Process., 17 (2008) 53-69.

Digital Library

[22]

J. Wright, A. Yang, A. Ganesh, S. Sastry, Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2009) 210-227.

Digital Library

[23]

Y. Xiao, T. Zeng, J. Yu, M.K. Ng, Restoration of images corrupted by mixed gaussian-impulse noise via l1-l0 minimization, Pattern Recognit., 44 (2011) 1708-1720.

Digital Library

[24]

A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009) 183-202.

Digital Library

[25]

D. Donoho, For most large underdetermined systems of linear equations the minimal �1-norm solution is also the sparsest solution, Commun. Pure Appl. Math., 59 (2004) 797-829.

[26]

M. Osborne, B. Presnell, B. Turlach, A new approach to variable selection in least squares problems, IMA J. Numer. Anal., 20 (2000) 389-403.

[27]

M. Figueiredo, R. Nowak, S. Wright, Gradient projection for sparse reconstruction, IEEE J. Sel. Top. Signal Process., 1 (2007) 586-597.

[28]

H. Lee, A. Battle, R. Raina, A. Ng, Efficient sparse coding algorithms, in: Advances in Neural Information Processing Systems, vol. 19, 2007, pp. 801-808.

[29]

R. Tibshirani, Regression shrinkage and selection via the Lasso, J. R. Stat. Soc. Ser. B, 58 (1996) 267-288.

[30]

M. Slawski, W. zu Castell, G. Tutz, Feature selection guided by structural information, Ann. Appl. Stat., 4 (2010) 1056-1080.

[31]

Z. Lu, Y. Peng, Heterogeneous constraint propagation with constrained sparse representation, in: Proceedings of ICDM, 2012, pp. 1002-1007.

[32]

S. Melacci, M. Belkin, Laplacian support vector machines trained in the primal, J. Mach. Learn. Res., 12 (2011) 1149-1184.

Digital Library

[33]

M. Karlen, J. Weston, A. Erkan, R. Collobert, Large scale manifold transduction, in: Proceedings of ICML, 2008, pp. 448-455.

[34]

S. Sun, J. Shawe-Taylor, Sparse semi-supervised learning using conjugate functions, J. Mach. Learn. Res., 11 (2010) 2423-2455.

Digital Library

[35]

G. Huang, S. Song, J.N. Gupta, C. Wu, A second order cone programming approach for semi-supervised learning, Pattern Recognit., 46 (2013) 3548-3558.

Digital Library

[36]

K.-H. Kim, S. Choi, Label propagation through minimax paths for scalable semi-supervised learning, Pattern Recognit. Lett., 45 (2014) 17-25.

[37]

T. Kobayashi, Kernel-based transition probability toward similarity measure for semi-supervised learning, Pattern Recognit., 47 (2014) 1994-2010.

Digital Library

[38]

K. Sinha, M. Belkin, Semi-supervised learning using sparse eigenfunction bases, in: Advances in Neural Information Processing Systems, vol. 22, 2010, pp. 1687-1695.

[39]

S. Gao, I. Tsang, L.-T. Chia, P. Zhao, Local features are not lonely-Laplacian sparse coding for image classification, in: Proceedings of CVPR, 2010, pp. 3555-3561.

[40]

X. Chen, Q. Lin, S. Kim, J.G. Carbonell, E.P. Xing, Smoothing proximal gradient method for general structured sparse learning, in: Proceedings of UAI, 2011, pp. 105-114.

[41]

S. Petry, C. Flexeder, G. Tutz, Pairwise fused Lasso, Technical Report 102, Department of Statistics, University of Munich, 2011.

[42]

D. Zhou, B. Scholk�pf, Regularization on discrete spaces, in: Proceedings of DAGM, 2005, pp. 361-368.

[43]

Y. Zhang, J.G. Schneider, A. Dubrawski, Learning compressible models, in: Proceedings of SDM, 2010, pp. 872-881.

[44]

Convex Optimization, in: Convex Optimization, Cambridge University Press, New York, 2004.

[45]

Semi-Supervised Learning, in: Semi-Supervised Learning, MIT Press, Cambridge, MA, 2006.

[46]

V. Sindhwani, D. Rosenberg, An RKHS for multi-view learning and manifold coregularization, in: Proceedings of ICML, 2008, pp. 976-983.

[47]

Z. Lu, H. Ip, Spatial Markov kernels for image categorization and annotation, IEEE Trans. Syst. Man Cybern. Part B, 41 (2011) 976-989.

Digital Library

[48]

O. Chapelle, A. Zien, Semi-supervised classification by low density separation, in: Proceedings of AISTATS, 2005, pp. 57-64.

Cited By

Yang XSong ZKing IXu Z(2023)A Survey on Deep Semi-Supervised LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322021935:9(8934-8954)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3220219
Gu NFan PFan MWang D(2019)Structure regularized self-paced learning for robust semi-supervised pattern classificationNeural Computing and Applications10.1007/s00521-018-3478-131:10(6559-6574)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s00521-018-3478-1
Kim M(2018)Dynamic sparse coding for sparse time-series modeling via first-order smooth optimizationApplied Intelligence10.5555/3288064.328809648:11(3889-3901)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.5555/3288064.3288096
Show More Cited By

Noise-robust semi-supervised learning via fast sparse coding
1. Computing methodologies

Recommendations

Iterative Semi-Supervised Sparse Coding Model for Image Classification

The scarcity of labeled data and the high-dimensionality of multimedia data are the major obstacles for image classification. Due to these concerns, this paper proposes a novel algorithm, Iterative Semi-supervised Sparse Coding (ISSC), which jointly ...
Image Classification by Iterative Semi-Supervised Sparse Coding
Advances in Multimedia Information Processing – PCM 2013
Abstract
In this paper, we propose a novel algorithm, Iterative Semi-supervised Sparse Coding(ISSC), which combines sparse coding and graph-based semi-supervised learning into a unified framework, in order to learn the discriminative sparse codes as well ...
L1-norm Laplacian support vector machine for data reduction in semi-supervised learning
Abstract
As a semi-supervised learning method, Laplacian support vector machine (LapSVM) is popular. Unfortunately, the model generated by LapSVM has a poor sparsity. A sparse decision model has always been fascinating because it could implement data ... $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$

Comments

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 48, Issue 2

February 2015

325 pages

ISSN:0031-3203

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 February 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang XSong ZKing IXu Z(2023)A Survey on Deep Semi-Supervised LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.322021935:9(8934-8954)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TKDE.2022.3220219
Gu NFan PFan MWang D(2019)Structure regularized self-paced learning for robust semi-supervised pattern classificationNeural Computing and Applications10.1007/s00521-018-3478-131:10(6559-6574)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s00521-018-3478-1
Kim M(2018)Dynamic sparse coding for sparse time-series modeling via first-order smooth optimizationApplied Intelligence10.5555/3288064.328809648:11(3889-3901)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.5555/3288064.3288096
Gu JJiao LLiu FYang SWang RChen PCui YXie JZhang Y(2018)Random subspace based ensemble sparse representationPattern Recognition10.1016/j.patcog.2017.09.01674:C(544-555)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1016/j.patcog.2017.09.016
Zhang ZZhang YLi FZhao MZhang LYan S(2017)Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagationPattern Recognition10.1016/j.patcog.2016.07.04261:C(492-510)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1016/j.patcog.2016.07.042
Yang TFu DLi X(2017)Semi-supervised classification of multiple kernels embedding manifold informationCluster Computing10.1007/s10586-017-1123-x20:4(3417-3426)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s10586-017-1123-x
Gan HLuo ZSun YXi XSang NHuang R(2016)Towards designing risk-based safe Laplacian Regularized Least SquaresExpert Systems with Applications: An International Journal10.1016/j.eswa.2015.09.01745:C(1-7)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1016/j.eswa.2015.09.017
Tang YChen YXu NJiang AZhou L(2016)Image denoising via sparse coding using eigenvectors of graph LaplacianDigital Signal Processing10.1016/j.dsp.2015.12.01350:C(114-122)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1016/j.dsp.2015.12.013

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents