research-article

Open access

Leveraging Label-Specific Discriminant Mapping Features for Multi-Label Learning

Authors:

James C. GeeAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 13, Issue 2

Article No.: 24, Pages 1 - 23

https://doi.org/10.1145/3319911

Published: 26 April 2019 Publication History

All formats PDF

Abstract

As an important machine learning task, multi-label learning deals with the problem where each sample instance (feature vector) is associated with multiple labels simultaneously. Most existing approaches focus on manipulating the label space, such as exploiting correlations between labels and reducing label space dimension, with identical feature space in the process of classification. One potential drawback of this traditional strategy is that each label might have its own specific characteristics and using identical features for all label cannot lead to optimized performance. In this article, we propose an effective algorithm named LSDM, i.e., leveraging label-specific discriminant mapping features for multi-label learning, to overcome the drawback. LSDM sets diverse ratio parameter values to conduct cluster analysis on the positive and negative instances of identical label. It reconstructs label-specific feature space which includes distance information and spatial topology information. Our experimental results show that combining these two parts of information in the new feature representation can better exploit the clustering results in the learning process. Due to the problem of diverse combinations for identical label, we employ simplified linear discriminant analysis to efficiently excavate optimal one for each label and perform classification by querying the corresponding results. Comparison with the state-of-the-art algorithms on a total of 20 benchmark datasets clearly manifests the competitiveness of LSDM.

References

[1]

Zafer Barut�uoglu, Robert E. Schapire, and Olga G. Troyanskaya. 2006. Hierarchical multi-label prediction of gene function. Bioinformatics 22, 7 (2006), 830--836.

Digital Library

[2]

Matthew R. Boutell, Jiebo Luo, Xipeng Shen, and Christopher M. Brown. 2004. Learning multi-label scene classification. Pattern Recognition 37, 9 (2004), 1757--1771.

[3]

Ricardo Silveira Cabral, Fernando De la Torre, Jo�o Paulo Costeira, and Alexandre Bernardino. 2011. Matrix completion for multi-label image classification. In Proceedings of the Neural Information Processing Systems, Vol. 201. 2.

Digital Library

[4]

Ricardo Silveira Cabral, Fernando De la Torre, Jo�o Paulo Costeira, and Alexandre Bernardino. 2015. Matrix completion for weakly-supervised multi-label image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1 (2015), 121--135.

[5]

Nicol� Cesa-Bianchi, Matteo Re, and Giorgio Valentini. 2012. Synergy of multi-label hierarchical ensembles, data fusion, and cost-sensitive methods for gene functional inference. Machine Learning 88, 1--2 (2012), 209--241.

Digital Library

[6]

Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 3 (2011), 27.

Digital Library

[7]

Li Li, Houfeng Wang, Xu Sun, Baobao Chang, Shi Zhao, and Lei Sha. 2015. Multi-label text categorization with joint learning predictions-as-features method. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP'15). 835--839.

[8]

Weiwei Cheng and Eyke H�llermeier. 2009. Combining instance-based learning and logistic regression for multilabel classification. Machine Learning 76, 2--3 (2009), 211--225.

Digital Library

[9]

Amanda Clare and Ross D. King. 2001. Knowledge discovery in multi-label phenotype data. In Proceedings of the Principles of Data Mining and Knowledge Discovery. Springer, 42--53.

Digital Library

[10]

Janez Demsar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006), 1--30.

Digital Library

[11]

Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of American Statistical Association 56, 293 (1961), 52--64.

[12]

Johannes F�rnkranz, Eyke H�llermeier, Eneldo Loza Menc�a, and Klaus Brinker. 2008. Multilabel classification via calibrated label ranking. Machine Learning 73, 2 (2008), 133--153.

Digital Library

[13]

Yuhong Guo and Suicheng Gu. 2011. Multi-label classification using conditional dependency networks. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Catalonia, Spain. 1300--1305.

Digital Library

[14]

James A. Hanley and Barbara J. McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 1 (1982), 29--36.

[15]

Peng Hou, Xin Geng, and Min-Ling Zhang. 2016. Multi-label manifold learning. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix, Arizona, USA. 1680--1686.

Digital Library

[16]

Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2015. Learning label specific features for multi-label classification. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 181--190.

Digital Library

[17]

Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2016. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering 28, 12 (2016), 3309--3323.

Digital Library

[18]

Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2018. Joint feature selection and classification for multilabel learning. IEEE Transactions on Cybernetics 48, 3 (2018), 876--889.

[19]

Anil K. Jain, M. Narasimha Murty, and Patrick J. Flynn. 1999. Data clustering: A review. ACM Computing Surveys 31, 3 (1999), 264--323.

Digital Library

[20]

Shuiwang Ji, Lei Tang, Shipeng Yu, and Jieping Ye. 2010. A shared-subspace learning framework for multi-label classification. ACM Transactions on Knowledge Discovery from Data 4, 2 (2010), 8.

Digital Library

[21]

Abhishek Kumar, Shankar Vembu, Aditya Krishna Menon, and Charles Elkan. 2012. Learning and inference in probabilistic classifier chains with beam search. In Proceedings of the Machine Learning and Knowledge Discovery in Databases. Springer, 665--680.

[22]

Yu-Kun Li, Min-Ling Zhang, and Xin Geng. 2015. Leveraging implicit relative labeling-importance information for effective multi-label learning. In Proceedings of the IEEE International Conference on Data Mining. Atlantic City, NJ, USA. 251--260.

Digital Library

[23]

Eneldo Loza Menc�a and Johannes F�rnkranz. 2008. Pairwise learning of multilabel classifications with perceptrons. In Proceedings of the International Joint Conference on Neural Networks, Part of the IEEE World Congress on Computational Intelligence. Hong Kong, China. 2899--2906.

[24]

Gjorgji Madjarov, Dejan Gjorgjevikj, and Tomche Delev. 2011. Efficient two stage voting architecture for pairwise multi-label classification. In Proceedings of the Advances in Artificial Intelligence. Springer, 164--173.

[25]

Elena Monta��s, Robin Senge, Jos� Barranquero, Jos� Ram�n Quevedo, Juan Jos� del Coz, and Eyke H�llermeier. 2014. Dependent binary relevance models for multi-label classification. Pattern Recognition 47, 3 (2014), 1494--1508.

Digital Library

[26]

Lishan Qiao, Limei Zhang, Zhonggui Sun, and Xueyan Liu. 2017. Selecting label-dependent features for multi-label classification. Neurocomputing 259 (2017), 112--118.

[27]

Jesse Read, Bernhard Pfahringer, and Geoff Holmes. 2008. Multi-label classification using ensembles of pruned sets. In Proceedings of the Eighth IEEE International Conference on Data Mining. IEEE, 995--1000.

Digital Library

[28]

Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2011. Classifier chains for multi-label classification. Machine Learning 85, 3 (2011), 333--359.

Digital Library

[29]

Sam T. Roweis and Lawrence K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323--2326.

[30]

Timothy N. Rubin, America Chambers, Padhraic Smyth, and Mark Steyvers. 2012. Statistical topic models for multi-label document classification. Machine Learning 88, 1--2 (2012), 157--208.

Digital Library

[31]

Robert E. Schapire and Yoram Singer. 2000. BoosTexter: A boosting-based system for text categorization. Machine Learning 39, 2/3 (2000), 135--168.

Digital Library

[32]

Lu Sun, Mineichi Kudo, and Keigo Kimura. 2016. Multi-label classification with meta-label-specific features. In Proceedings of the 23rd International Conference on Pattern Recognition. IEEE, 1612--1617.

[33]

Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. 2010. Mining multi-label data. In Proceedings of the Data Mining and Knowledge Discovery Handbook. Springer, 667--685.

[34]

Grigorios Tsoumakas, Ioannis Katakis, and Ioannis P. Vlahavas. 2011. Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering 23, 7 (2011), 1079--1089.

Digital Library

[35]

G. Tsoumakas, M. L. Zhang, and Z. H. Zhou. 2009. Tutorial on learning from multi-label data. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases.

[36]

Alexis Vallet and Hiroyasu Sakamoto. 2015. A multi-label convolutional neural network for automatic image annotation. Journal of Information Processing 23, 6 (2015), 767--775.

[37]

Xiao Wang, Jun Zhang, and Guo-Zheng Li. 2015. Multi-location gram-positive and gram-negative bacterial protein subcellular localization using gene ontology and multi-label classifier ensemble. BMC Bioinformatics 16, Suppl 12 (2015), S1.

[38]

Wei Weng, Yaojin Lin, Shunxiang Wu, Yuwen Li, and Yun Kang. 2018. Multi-label learning based on label-specific features and local pairwise label correlation. Neurocomputing 273 (2018), 385--394.

Digital Library

[39]

Suping Xu, Xibei Yang, Hualong Yu, Dong-Jun Yu, Jingyu Yang, and Eric C. C. Tsang. 2016. Multi-label learning with label-specific feature reduction. Knowledge-Based Systems 104 (2016), 52--61.

Digital Library

[40]

Ju-Jie Zhang, Min Fang, and Xiao Li. 2015. Multi-label learning with discriminative features for each label. Neurocomputing 154 (2015), 305--316.

Digital Library

[41]

Min-Ling Zhang. 2011. LIFT: Multi-label learning with label-specific features. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Catalonia, Spain. 1609--1614.

Digital Library

[42]

Min-Ling Zhang and Lei Wu. 2015. Lift: Multi-label learning with label-specific features. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1 (2015), 107--120.

[43]

Min-Ling Zhang and Kun Zhang. 2010. Multi-label learning by exploiting label dependency. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, DC, USA, 999--1008.

Digital Library

[44]

Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition 40, 7 (2007), 2038--2048.

Digital Library

[45]

Min-Ling Zhang and Zhi-Hua Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2014), 1819--1837.

Cited By

Xu TXu YYang SLi BZhang W(2024)Learning Accurate Label-Specific Features From Partially Multilabeled DataIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.324192135:8(10436-10450)Online publication date: Aug-2024
https://doi.org/10.1109/TNNLS.2023.3241921
Zhang ZWang JGeng X(2024)Label distribution learning by utilizing common and label-specific feature fusion spaceInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02351-9Online publication date: 4-Sep-2024
https://doi.org/10.1007/s13042-024-02351-9
Zhao DTan YSun DGao QLu YZhu D(2024)Multi-label learning of missing labels using label-specific features: an embedded packaging methodApplied Intelligence10.1007/s10489-023-05203-154:1(791-814)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05203-1
Show More Cited By

Index Terms

Leveraging Label-Specific Discriminant Mapping Features for Multi-Label Learning
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Learning latent representations

Recommendations

MULFE: Multi-Label Learning via Label-Specific Feature Space Ensemble
In multi-label learning, label correlations commonly exist in the data. Such correlation not only provides useful information, but also imposes significant challenges for multi-label learning. Recently, label-specific feature embedding has been proposed ...
Multi-label learning by exploiting label dependency
KDD '10: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining

In multi-label learning, each training example is associated with a set of labels and the task is to predict the proper label set for the unseen example. Due to the tremendous (exponential) number of possible label sets, the task of learning from multi-...
Multi-label learning based on label-specific features and local pairwise label correlation

Multi-label learning has drawn great attention in recent years. One of its tasks aims to build classification models for the problem where each instance associates with a set of labels. In order to exploit discriminative features for classification, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 13, Issue 2

April 2019

342 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3319626

Editors:
Charu Aggarwal
IBM T. J. Watson Research, USA
,
Xindong Wu
University of Louisiana at Lafayette, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2019

Accepted: 01 February 2019

Revised: 01 February 2019

Received: 01 March 2018

Published in�TKDD�Volume 13, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

UGC under project Hong Kong PolyU
International Exchange Program for Graduate Students
National Key R8D Program of China
GRF
Tongji University
Central Research Grant
Hong Kong PolyU
Natural Science Foundation of China
Fundamental Research Funds for the Central public welfare research institutes

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
970
Total Downloads

Downloads (Last 12 months)155
Downloads (Last 6 weeks)21

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu TXu YYang SLi BZhang W(2024)Learning Accurate Label-Specific Features From Partially Multilabeled DataIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.324192135:8(10436-10450)Online publication date: Aug-2024
https://doi.org/10.1109/TNNLS.2023.3241921
Zhang ZWang JGeng X(2024)Label distribution learning by utilizing common and label-specific feature fusion spaceInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02351-9Online publication date: 4-Sep-2024
https://doi.org/10.1007/s13042-024-02351-9
Zhao DTan YSun DGao QLu YZhu D(2024)Multi-label learning of missing labels using label-specific features: an embedded packaging methodApplied Intelligence10.1007/s10489-023-05203-154:1(791-814)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-023-05203-1
Li GZhang HMin FLu Y(2023)Two-stage label distribution learning with label-independent prediction based on label-specific featuresKnowledge-Based Systems10.1016/j.knosys.2023.110426267(110426)Online publication date: May-2023
https://doi.org/10.1016/j.knosys.2023.110426
Zhang JWu HJiang MLiu JLi STang YLong J(2023)Group-preserving label-specific feature selection for multi-label learningExpert Systems with Applications10.1016/j.eswa.2022.118861213(118861)Online publication date: Mar-2023
https://doi.org/10.1016/j.eswa.2022.118861
Ai QLi FLi XZhao JWang WGao QZhao F(2023)An improved MLTSVM using label-specific features with missing labelsApplied Intelligence10.1007/s10489-022-03634-w53:7(8039-8060)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10489-022-03634-w
Weng WWei BKe WFan YWang JLi Y(2023)Learning label-specific features with global and local label correlation for multi-label classificationApplied Intelligence10.1007/s10489-022-03386-753:3(3017-3033)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1007/s10489-022-03386-7
Zhao TZhang YPedrycz W(2022)Robust Multi-Label Classification with Enhanced Global and Local Label CorrelationMathematics10.3390/math1011187110:11(1871)Online publication date: 30-May-2022
https://doi.org/10.3390/math10111871
Zhao TZhang YMiao D(2022)Intuitionistic Fuzzy-Based Three-Way Label Enhancement for Multi-Label ClassificationMathematics10.3390/math1011184710:11(1847)Online publication date: 27-May-2022
https://doi.org/10.3390/math10111847
Zhang ZLiu LLi JWu X(2022)Integrating Global and Local Feature Selection for Multi-Label LearningACM Transactions on Knowledge Discovery from Data10.1145/353219017:1(1-37)Online publication date: 10-May-2022
https://dl.acm.org/doi/10.1145/3532190
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents