research-article

Text Classification Method Based on BiGRU-Attention and CNN Hybrid Model

Authors:

Li LongAuthors Info & Claims

AIPR '21: Proceedings of the 2021 4th International Conference on Artificial Intelligence and Pattern Recognition

Pages 614 - 622

https://doi.org/10.1145/3488933.3488970

Published: 25 February 2022 Publication History

Abstract

Aiming at the problem that traditional Gated Recurrent Unit (GRU) and Convolution Neural Network (CNN) can not reflect the importance of each word in the text when extracting features, a text classification method based on BiGRU Attention and CNN is proposed. Firstly, CNN was used to extract the local information of the text, and then the full-text semantics was integrated. Secondly, BiGRU was used to extract the context features of the text, and attention mechanism was used after BiGRU to extract the attention score of the output information. Finally, the output of BiGRU attention was fused with the output of CNN to realize the effective extraction of text features and focused on the important content words. Experimental results on three public datasets showed that the proposed model was better than GRU, CNN and other models, which can effectively improve the effect of text classification.

References

[1]

LIU M Z, ZHOU F Y, CHEN K, Co-attention networks based on aspect and context for aspect-level sentiment analysis[J]. Knowledge-Based Systems, 2021, 217.

[2]

XU Z, ZHU Y, PENG Q, Using deep belief network to demote web spam[J]. Future Generation Computer Systems, 2021, 118.

[3]

CHEN L, LIU Y, CHANG Y D, Public opinion analysis of novel coronavirus from online data[J]. Journal of Safety Science and Resilience, 2020, 1(2).

[4]

SHAH K, PATEL H, SANGHVI D, A comparative analysis of logistic regression, random forest and knn models for the text classification[J]. Augmented Human Research, 2020, 5(1): 5-12.

[5]

ISMAÏL B, ABDELGHANI A, LOUIS R, Computer-Assisted Reading: Getting Help from Text Classification and Maximal Association Rules[J]. JAIT,2013,4(4).

[6]

YING Y, MURSITAMA T N, SHIDARTA, Effectiveness of the News Text Classification Test Using the Naïve Bayes’ Classification Text Mining Method[J]. Journal of Physics: Conference Series, 2021, 1764(1).

[7]

CHEN G N, DAI Z B, DUAN J T, Improved naive bayes with optimal correlation factor for text classification[J]. SN Applied Sciences, 2019,1(9): 1-10.

[8]

BEAKCHEOL J, INHWAN K, JONG K W. Word2vec convolutional neural networks for classification of news articles and tweets.[J]. Plos one, 2019, 14(8): e0220976

[9]

AURANGZEB K, BAHARUM B, LAM H L, A Review of Machine Learning Algorithms for Text-Documents Classification[J]. JAIT,2010,1(1).

[10]

DEWAN M. FARID, CHOWDHURY M R. Mining Complex Data Streams: Discretization, Attribute Selection and Classification[J]. JAIT,2013,4(3).

[11]

HE S, LU Y Y. A modularized architecture of multi branch convolutional neural network for image captioning[J]. Electronics, 2019, 8(12): 1417-1432.

[12]

FESSEHA A, XIONG S W, EMIRU E D, Text Classification Based on Convolutional Neural Networks and Word Embedding for Low-Resource Languages: Tigrinya[J]. Information, 2021, 12(2).

[13]

LUO L X. Network text sentiment analysis method combining LDA text representation and GRU-CNN[J]. Personal and Ubiquitous Computing, 2019, 23(3-4).

Digital Library

[14]

ZHANG Y S, ZHENG J, JIANG Y R, A text sentiment classification modeling method based on coordinated CNN-LSTM-Attention model[J]. Chinese Journal of Electronics, 2019, 28(01): 120-126.

[15]

BAHDANAU, DZMITRY, CHO, Neural machine translation by jointly learning to align and translate[J]. 2014.

[16]

Li H, Zheng Y, Ren P. Dual-channel attention model for text sentiment analysis[J]. International Journal of Performability Engineering, 2019, 15(3): 834-841.

[17]

ZHANG D J, HONG M B, ZOU L, Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification[J]. International Journal of Computational Intelligence Systems, 2019.

[18]

GU B W, SUNG Y S. Enhanced Reinforcement Learning Method Combining One-Hot Encoding-Based Vectors for CNN-Based Alternative High-Level Decisions[J]. Applied Sciences, 2021, 11(3).

[19]

MIKOLOV T, CHEN Kai, CORRADO G, Efficient estimation of word representations in vector space[EB/OL]. (2013-09-07). https://arxiv.org/abs/1301.3781.

[20]

BEHERA R K, JENA M L, RATH S K, Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data[J]. Information Processing & Management, 2021, 58(1).

[21]

HUANG X, CHEN B L, XIAO L, Label-Aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification[J]. Neural Processing Letters, 2021(prepublish).

[22]

GUO B, ZHANG C X, LIU J M, Improving text classification with weighted word embeddings via a multi-channel TextCNN model[J]. Neurocomputing, 2019, 363: 366-374.

Digital Library

[23]

ZHANG X, ZHAO J, LE C Y. Character-level convolutional networks for text classification[C] //Advances in Neural Information Processing Systems, 2015: 649-657.

[24]

LAN W F, XU W, WANG T. Text classification of Chinese news based on convolutional neural network[J]. Journal of South-Central University for Nationalities(Natural Science Edition). 2018, 37(1): 138-143.

[25]

MURAT A, ALI K. Improving the accuracy using pre-trained word embeddings on deep neural networks for Turkish text classification[J]. Physica A: Statistical Mechanics and its Applications, 2020, 541.

[26]

LI Q, WENG L H, DING X H. A Novel Neural Network-Based Method for Medical Text Classification[J]. Future Internet, 2019, 11(12).

[27]

LAI S, XU L, LIU K, Recurrent convolutional neural networks for text classification[J]. The association for the advance of Artificial Intelligence, AAAI, 2015, 333: 2267-2273.

[28]

YU J X, ZHANG X, Xu L L,et al. A hybrid CNN-GRU model for predicting soil moisture in maize root zone[J]. Agricultural Water Management, 2021, 245 (prepublish).

[29]

LUO L X. Network text sentiment analysis method combining LDA text representation and GRU-CNN[J]. Personal and Ubiquitous Computing, 2019, 23(3-4).

Digital Library

[30]

SMETANIN S, Komarov M. Deep transfer learning baselines for sentiment analysis in Russian[J]. Information Processing and Management, 2021, 58(3).

[31]

ASHRAF K, MUHAMMAD A. CAT-BiGRU: Convolution and Attention with Bi-Directional Gated Recurrent Unit for Self-Deprecating Sarcasm Detection[J]. Cognitive Computation, 2021(prepublish).

[32]

LI X, CUI M L, LI J P, BAI R B, A hybrid medical text classification framework: Integrating attentive rule construction and neural network[J]. Neurocomputing, 2021, 443.

Cited By

Ying ZLing MZhang Y(2024)Machine learning based software effort estimation using development-centric features for crowdsourcing platformIntelligent Data Analysis10.3233/IDA-22735828:2(451-465)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.3233/IDA-227358
Javeed DGao TKumar PJolfaei A(2024)An Explainable and Resilient Intrusion Detection System for Industry 5.0IEEE Transactions on Consumer Electronics10.1109/TCE.2023.328370470:1(1342-1350)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3283704
Sharma LDeepak ARanjan AKrishnasamy G(2023)A novel hybrid CNN and BiGRU-Attention based deep learning model for protein function predictionStatistical Applications in Genetics and Molecular Biology10.1515/sagmb-2022-005722:1Online publication date: 4-Sep-2023
https://doi.org/10.1515/sagmb-2022-0057
Show More Cited By

Index Terms

Text Classification Method Based on BiGRU-Attention and CNN Hybrid Model
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks
2. Information systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Chinese Text Classification Based on Hybrid Model of CNN and LSTM
DSIT 2020: Proceedings of the 3rd International Conference on Data Science and Information Technology

Text classification is one of the basic tasks of natural language processing. In recent years, deep learning has been widely used in text classification tasks. The representative one is the convolutional neural network. The convolutional neural network(...
Text classification of Chinese news based on multi-scale CNN and LSTM hybrid model
Abstract
Deep neural network has significant performance in text classification. Convolutional neural network (CNN) and recurrent neural network (RNN) are two main structures for natural language processing. They use different ways to understand natural ...
A Convolutional Neural Network with Word-level Attention for Text Classification
ICCMS '20: Proceedings of the 12th International Conference on Computer Modeling and Simulation

Text classification is a classic task in the NLP area which aims to predict the categories for given texts. Many neural network models are applied to this task with the development of the neural network technology. One of the typical neural network ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AIPR '21: Proceedings of the 2021 4th International Conference on Artificial Intelligence and Pattern Recognition

September 2021

715 pages

ISBN:9781450384087

DOI:10.1145/3488933

Copyright � 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AIPR 2021

AIPR 2021: 2021 4th International Conference on Artificial Intelligence and Pattern Recognition

September 24 - 26, 2021

Xiamen, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
300
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)12

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ying ZLing MZhang Y(2024)Machine learning based software effort estimation using development-centric features for crowdsourcing platformIntelligent Data Analysis10.3233/IDA-22735828:2(451-465)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.3233/IDA-227358
Javeed DGao TKumar PJolfaei A(2024)An Explainable and Resilient Intrusion Detection System for Industry 5.0IEEE Transactions on Consumer Electronics10.1109/TCE.2023.328370470:1(1342-1350)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3283704
Sharma LDeepak ARanjan AKrishnasamy G(2023)A novel hybrid CNN and BiGRU-Attention based deep learning model for protein function predictionStatistical Applications in Genetics and Molecular Biology10.1515/sagmb-2022-005722:1Online publication date: 4-Sep-2023
https://doi.org/10.1515/sagmb-2022-0057
Liu JLei XZhang YPan Y(2023)The prediction of molecular toxicity based on BiGRU and GraphSAGEComputers in Biology and Medicine10.1016/j.compbiomed.2022.106524153:COnline publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.106524

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents