research-article

A sparsity-inducing formulation for evolutionary co-clustering

Authors:

Jun LiuAuthors Info & Claims

KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 334 - 342

https://doi.org/10.1145/2339530.2339586

Published: 12 August 2012 Publication History

Abstract

Traditional co-clustering methods identify block structures from static data matrices. However, the data matrices in many applications are dynamic; that is, they evolve smoothly over time. Consequently, the hidden block structures embedded into the matrices are also expected to vary smoothly along the temporal dimension. It is therefore desirable to encourage smoothness between the block structures identified from temporally adjacent data matrices. In this paper, we propose an evolutionary co-clustering formulation for identifying co-cluster structures from time-varying data. The proposed formulation encourages smoothness between temporally adjacent blocks by employing the fused Lasso type of regularization. Our formulation is very flexible and allows for imposing smoothness constraints over only one dimension of the data matrices, thereby enabling its applicability to a large variety of settings. The optimization problem for the proposed formulation is non-convex, non-smooth, and non-separable. We develop an iterative procedure to compute the solution. Each step of the iterative procedure involves a convex, but non-smooth and non-separable problem. We propose to solve this problem in its dual form, which is convex and smooth. This leads to a simple gradient descent algorithm for computing the dual optimal solution. We evaluate the proposed formulation using the Allen Developing Mouse Brain Atlas data. Results show that our formulation consistently outperforms methods without the temporal smoothness constraints.

Supplementary Material

JPG File (311b_m_talk_6.jpg)

Download
16.27 KB

MP4 File (311b_m_talk_6.mp4)

Download
145.81 MB

References

[1]

A. Ahmed and E. P. Xing. Recovering time-varying networks of dependencies in social and biological studies. Proceedings of the National Academy of Sciences, 106(29):11878--11883, 2009.

[2]

J. W. Bohland, H. Bokil, C. B. Allen, and P. P. Mitra. The brain atlas concordance problem: Quantitative comparison of anatomical parcellations. PLoS ONE, 4(9):e7200, 09 2009.

[3]

J. W. Bohland et al. Clustering of spatial gene expression patterns in the mouse brain and comparison with classical neuroanatomy. Methods, 50(2):105--112, 2010.

[4]

S. Busygin, O. Prokopyev, and P. M. Pardalos. Biclustering in data mining. Computers and Operations Research, 35:2964--2987, September 2008.

Digital Library

[5]

D. Chakrabarti, R. Kumar, and A. Tomkins. Evolutionary clustering. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 554--560, 2006.

Digital Library

[6]

Y. Cheng and G. M. Church. Biclustering of expression data. In Proceedings of the Eighth International Conference on Intelligent Systems for Molecular Biology, pages 93--103, 2000.

Digital Library

[7]

Y. Chi, X. Song, D. Zhou, K. Hino, and B. L. Tseng. On evolutionary spectral clustering. ACM Transactions on Knowledge Discovery from Data, 3:17:1--17:30, December 2009.

Digital Library

[8]

H. Cho, I. S. Dhillon, Y. Guan, and S. Sra. Minimum sum-squared residue co-clustering of gene expression data. In Proceedings of the Fourth SIAM International Conference on Data Mining, 2004.

[9]

I. S. Dhillon. Co-clustering documents and words using bipartite spectral graph partitioning. In Proceedings of the seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 269--274, 2001.

Digital Library

[10]

I. S. Dhillon, S. Mallela, and D. S. Modha. Information-theoretic co-clustering. In Proceedings of the ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 89--98, 2003.

Digital Library

[11]

C. Ding, T. Li, W. Peng, and H. Park. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 126--135, 2006.

Digital Library

[12]

C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211--218, 1936.

[13]

J. Friedman, T. Hastie, H. H�fling, and R. Tibshirani. Pathwise coordinate optimization. Annals of Applied Statistics, 1(2):302--332, 2007.

[14]

E. Giannakidou, V. Koutsonikola, A. Vakali, and Y. Kompatsiaris. Co-clustering tags and social data sources. In Proceedings of the 2008 The Ninth International Conference on Web-Age Information Management, pages 317--324, 2008.

Digital Library

[15]

N. Guan, D. Tao, Z. Luo, and B. Yuan. Non-negative patch alignment framework. IEEE Transactions on Neural Networks, 22(8):1218--1230, 2011.

Digital Library

[16]

N. Guan, D. Tao, Z. Luo, and B. Yuan. NeNMF: An optimal gradient method for nonnegative matrix factorization. IEEE Transactions on Signal Processing, 60(6):2882--2898, 2012.

Digital Library

[17]

J. A. Hartigan. Direct clustering of a data matrix. Journal of the American Statistical Association, 67(337):123--129, 1972.

[18]

H. H�fling. A path algorithm for the fused Lasso signal approximator. Journal of Computational and Graphical Statistics, 19(4):984--1006, 2010.

[19]

S. Ji. Computational network analysis of the anatomical and genetic organizations in the mouse brain. Bioinformatics, 27(23):3293--3299, 2011.

Digital Library

[20]

Y. Kluger, R. Basri, J. T. Chang, and M. Gerstein. Spectral biclustering of microarray data: Coclustering genes and conditions. Genome Research, 13(4):703--716, 2003.

[21]

M. Lee, H. Shen, J. Z. Huang, and J. S. Marron. Biclustering via sparse singular value decomposition. Biometrics, 66:1087--1095, 2010.

[22]

E. S. Lein et al. Genome-wide atlas of gene expression in the adult mouse brain. Nature, 445(7124):168--176, 2007.

[23]

Y.-R. Lin, Y. Chi, S. Zhu, H. Sundaram, and B. L. Tseng. Analyzing communities and their evolutions in dynamic social networks. ACM Transactions on Knowledge Discovery from Data, 3:8:1--8:31, April 2009.

Digital Library

[24]

J. Liu, L. Yuan, and J. Ye. An efficient algorithm for a class of fused Lasso problems. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 323--332, 2010.

Digital Library

[25]

S. C. Madeira and A. L. Oliveira. Biclustering algorithms for biological data analysis: A survey. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 1:24--45, January 2004.

Digital Library

[26]

A. Nemirovski. Efficient methods in convex programming, 1994. Lecture Notes.

[27]

Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, 2003.

Digital Library

[28]

C. Soneson and M. Fontes. A method for visual identification of small sample subgroups and potential biomarkers. Annals of Applied Statistics, 5(3):2131--2149, 2011.

[29]

R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58(1):267--288, 1996.

[30]

R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight. Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1):91--108, 2005.

[31]

R. Tibshirani and P. Wang. Spatial smoothing and hot spot detection for CGH data using the fused lasso. Biostatistics, 9(1):18--29, 2008.

[32]

H. Tong, S. Papadimitriou, P. S. Yu, and C. Faloutsos. Proximity tracking on time-evolving bipartite graphs. In Proceedings of the SIAM International Conference on Data Mining, pages 704--715, 2008.

[33]

P. Tseng. Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109:475--494, June 2001.

Digital Library

[34]

F. Wang, H. Tong, and C.-Y. Lin. Towards evolutionary nonnegative matrix factorization. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.

Digital Library

[35]

J. Yoo and S. Choi. Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds. Information Processing and Management, 46(5):559--570, 2010.

Digital Library

[36]

H. Zha, X. He, C. Ding, H. Simon, and M. Gu. Bipartite graph partitioning and data clustering. In Proceedings of the tenth International Conference on Information and Knowledge Management, pages 25--32, 2001.

Digital Library

Cited By

Li WZhu HLi SWang HDai HWang CJin Q(2021)Evolutionary community discovery in dynamic social networks via resistance distanceExpert Systems with Applications10.1016/j.eswa.2020.114536171(114536)Online publication date: Jun-2021
https://doi.org/10.1016/j.eswa.2020.114536
Sun WLi L(2019)Dynamic Tensor ClusteringJournal of the American Statistical Association10.1080/01621459.2018.1527701114:528(1894-1907)Online publication date: 18-Mar-2019
https://doi.org/10.1080/01621459.2018.1527701
Ma XDong D(2017)Evolutionary Nonnegative Matrix Factorization Algorithms for Community Detection in Dynamic NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.265775229:5(1045-1058)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TKDE.2017.2657752
Show More Cited By

Index Terms

A sparsity-inducing formulation for evolutionary co-clustering
1. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Sparsity Learning Formulations for Mining Time-Varying Data
Traditional clustering and feature selection methods consider the data matrix as static. However, the data matrices evolve smoothly over time in many applications. A simple approach to learn from these time-evolving data matrices is to analyze them ...
Low-Rank Tensor Completion via Novel Sparsity-Inducing Regularizers
To alleviate the bias generated by the <inline-formula><tex-math notation="LaTeX">$\ell_{1}$</tex-math></inline-formula>-norm in the low-rank tensor completion problem, nonconvex surrogates/regularizers have been suggested to replace the tensor nuclear ...
Optimization with Sparsity-Inducing Penalties

Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. They were first dedicated to linear variable selection but numerous extensions have now emerged such as structured sparsity or kernel selection. It ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2012

1616 pages

ISBN:9781450314626

DOI:10.1145/2339530

General Chair:
Qiang Yang
Hong Kong University of Science and Technology
,
Program Chairs:
Deepak Agarwal
LinkedIn
,
Jian Pei
Simon Fraser University

Copyright � 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '12

Sponsor:

KDD '12: The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 12 - 16, 2012

Beijing, China

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
551
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li WZhu HLi SWang HDai HWang CJin Q(2021)Evolutionary community discovery in dynamic social networks via resistance distanceExpert Systems with Applications10.1016/j.eswa.2020.114536171(114536)Online publication date: Jun-2021
https://doi.org/10.1016/j.eswa.2020.114536
Sun WLi L(2019)Dynamic Tensor ClusteringJournal of the American Statistical Association10.1080/01621459.2018.1527701114:528(1894-1907)Online publication date: 18-Mar-2019
https://doi.org/10.1080/01621459.2018.1527701
Ma XDong D(2017)Evolutionary Nonnegative Matrix Factorization Algorithms for Community Detection in Dynamic NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.265775229:5(1045-1058)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1109/TKDE.2017.2657752
Li FZhang S(2017)Co-clustering with Manifold and Double Sparse RepresentationIntelligent Data Engineering and Automated Learning – IDEAL 201710.1007/978-3-319-68935-7_31(279-286)Online publication date: 6-Oct-2017
https://doi.org/10.1007/978-3-319-68935-7_31
Zhu YYang HHe JCao LZhang CJoachims TWebb GMargineantu DWilliams G(2015)Co-Clustering based Dual Prediction for Cargo Pricing OptimizationProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining10.1145/2783258.2783337(1583-1592)Online publication date: 10-Aug-2015
https://dl.acm.org/doi/10.1145/2783258.2783337
Rongjian Li Wenlu Zhang Yao Zhao Zhenfeng Zhu Shuiwang Ji (2015)Sparsity Learning Formulations for Mining Time-Varying DataIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2014.237341127:5(1411-1423)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1109/TKDE.2014.2373411
Fang QNg WFeng JLi Y(2014)Mining order-preserving submatrices from probabilistic matricesACM Transactions on Database Systems10.1145/253371239:1(1-43)Online publication date: 6-Jan-2014
https://dl.acm.org/doi/10.1145/2533712
Bi JSun JXu TLu JMa YQiu L(2014)A sparse integrative cluster analysis for understanding soybean phenotypes2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM.2014.6999290(1-7)Online publication date: Nov-2014
https://doi.org/10.1109/BIBM.2014.6999290
Ji SZhang WLi R(2013)A Probabilistic Latent Semantic Analysis Model for Coclustering the Mouse Brain AtlasIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2013.13510:6(1460-1468)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.1109/TCBB.2013.135
Wang YChang XLi RXu Z(2013)Sparse K-Means with the l_q(0leq q< 1) Constraint for High-Dimensional Data Clustering2013 IEEE 13th International Conference on Data Mining10.1109/ICDM.2013.64(797-806)Online publication date: Dec-2013
https://doi.org/10.1109/ICDM.2013.64

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents