research-article

Public Access

Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions

Authors:

Michael Bernstein,

Cristian Danescu-Niculescu-Mizil,

Jure LeskovecAuthors Info & Claims

CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

Pages 1217 - 1230

https://doi.org/10.1145/2998181.2998213

Published: 25 February 2017 Publication History

Abstract

In online communities, antisocial behavior such as trolling disrupts constructive discussion. While prior work suggests that trolling behavior is confined to a vocal and antisocial minority, we demonstrate that ordinary people can engage in such behavior as well. We propose two primary trigger mechanisms: the individual's mood, and the surrounding context of a discussion (e.g., exposure to prior trolling behavior). Through an experiment simulating an online discussion, we find that both negative mood and seeing troll posts by others significantly increases the probability of a user trolling, and together double this probability. To support and extend these results, we study how these same mechanisms play out in the wild via a data-driven, longitudinal analysis of a large online news discussion community. This analysis exposes temporal mood effects, and explores long range patterns of repeated exposure to trolling. A predictive model of trolling behavior reveals that mood and discussion context together can explain trolling behavior better than an individual's history of trolling. These results combine to suggest that ordinary people can, under the right circumstances, behave like trolls.

References

[1]

Yavuz Akbulut, Yusuf Levent Sahin, and Bahadir Eristi. 2010. Cyberbullying Victimization among Turkish Online Social Utility Members. Educ. Technol. Soc. (2010).

[2]

Jeff Atwood. 2013. Why is there a topic reply limit for new users? http://bit.ly/1XDLk8a. (2013).

[3]

Lars Backstrom, Jon Kleinberg, Lillian Lee, and Cristian Danescu-Niculescu-Mizil. 2013. Characterizing and curating conversation threads: expansion, focus, volume, re-entry. In Proc. WSDM.

Digital Library

[4]

Paul Baker. 2001. Moral panic and alternative identity construction in Usenet. J. Comput. Mediat. Commun. (2001).

[5]

Abhijit V Banerjee. 1992. A simple model of herd behavior. Q. J. Econ. (1992).

[6]

Sigal G Barsade. 2002. The ripple effect: Emotional contagion and its influence on group behavior. Adm. Sci. Q. (2002).

[7]

Roy F Baumeister, Ellen Bratslavsky, Catrin Finkenauer, and Kathleen D Vohs. 2001. Bad is stronger than good. Rev. Gen. Psychol. (2001).

[8]

Leonard Berkowitz. 1993. Aggression: Its causes, consequences, and control.

[9]

Amy Binns. 2012. DON'T FEED THE TROLLS! Managing troublemakers in magazines' online communities. Journalism Practice (2012).

[10]

Galen V Bodenhausen, Geoffrey P Kramer, and Karin Susser. 1994. Happiness and stereotypic thinking in social judgment. J. Pers. Soc. Psychol. (1994).

[11]

Erin E Buckels, Paul D Trapnell, and Delroy L Paulhus. 2014. Trolls just want to have fun. Pers. Individ. Dif. (2014).

[12]

Irina Ceaparu, Jonathan Lazar, Katie Bessiere, John Robinson, and Ben Shneiderman. 2004. Determining causes and severity of end-user frustration. Int. J. Hum. Comput. Interact. (2004).

[13]

Damon Centola. 2010. The spread of behavior in an online social network experiment. Science (2010).

[14]

Stevie Chancellor, Zhiyuan Jerry Lin, and Munmun De Choudhury. 2016. This Post Will Just Get Taken Down: Characterizing Removed Pro-Eating Disorder Social Media Content. In Proc. SIGCHI.

Digital Library

[15]

Daphne Chang, Erin L Krupka, Eytan Adar, and Alessandro Acquisti. 2016. Engineering Information Disclosure: Norm Shaping Designs. In Proc. SIGCHI.

Digital Library

[16]

Zhansheng Chen, Kipling D Williams, Julie Fitness, and Nicola C Newton. 2008. When Hurt Will Not Heal: Exploring the Capacity to Relive Social and Physical Pain. Psychol. Sci. (2008).

[17]

Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How community feedback shapes user behavior. In Proc. ICWSM.

[18]

Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial Behavior in Online Discussion Communities. In Proc. ICWSM.

[19]

Daegon Cho and Alessandro Acquisti. 2013. The more social cues, the less trolling? An empirical study of online commenting behavior. In Proc. WEIS.

[20]

Robert B Cialdini and Noah J Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. (2004).

[21]

Robert B Cialdini, Raymond R Reno, and Carl A Kallgren. 1990. A focus theory of normative conduct: recycling the concept of norms to reduce littering in public places. J. Pers. Soc. Psychol. (1990).

[22]

CNN. n.d. Community Guidelines. http://cnn.it/1WAhxh5. (n.d.).

[23]

John P Davis, Shelly Farnham, and Carlos Jensen. 2002. Decreasing online 'bad' behavior. In Proc. SIGCHI Extended Abstracts.

Digital Library

[24]

Nicholas Diakopoulos and Mor Naaman. 2011. Towards quality discourse in online news comments. In Proc. CSCW.

Digital Library

[25]

Discourse. n.d. This is a Civilized Place for Public Discussion. http://bit.ly/1TM8K5x. (n.d.).

[26]

Judith S Donath. 1999. Identity and deception in the virtual community. Communities in Cyberspace (1999).

[27]

Maeve Duggan. 2014. Online Harassment. Pew Research Center (2014).

[28]

Klint Finley. 2015. A brief history of the end of the comments. http://bit.ly/1Wz2OUg. (2015).

[29]

Joseph P Forgas and Gordon H Bower. 1987. Mood effects on person-perception judgments. J. Pers. Soc. Psychol. (1987).

[30]

Becky Gardiner, Mahana Mansfield, Ian Anderson, Josh Holder, Daan Louter, and Monica Ulmanu. 2016. The dark side of Guardian comments. Guardian (2016).

[31]

Francesca Gino, Shahar Ayal, and Dan Ariely. 2009. Contagion and differentiation in unethical behavior: the effect of one bad apple on the barrel. Psychol. Sci (2009).

[32]

Scott A Golder and Michael W Macy. 2011. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science (2011).

[33]

Roberto Gonz�lez-Ib��nez and Chirag Shah. 2012. Investigating positive and negative affects in collaborative information seeking: A pilot study report. In Proc. ASIST.

[34]

Jeff Greenberg, Linda Simon, Tom Pyszczynski, Sheldon Solomon, and Dan Chatel. 1992. Terror management and tolerance: Does mortality salience always intensify negative reactions to others who threaten one's worldview? J. Pers. Soc. Psychol. (1992).

[35]

The Guardian. n.d. Community Standards and Participation Guidelines. http://bit.ly/1YtIxfW. (n.d.).

[36]

Noriko Hara, Curtis Jay Bonk, and Charoula Angeli. 2000. Content analysis of online discussion in an applied educational psychology course. Instr. Sci. (2000).

[37]

Claire Hardaker. 2010. Trolling in asynchronous computer-mediated communication: From user discussions to academic definitions. J. Politeness Res. (2010).

[38]

Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing? trolling? in a feminist forum. The Information Society (2002).

[39]

Chiao-Fang Hsu, Elham Khabiri, and James Caverlee. 2009. Ranking comments on the social web. In Proc. CSE.

Digital Library

[40]

Clayton J Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proc. ICWSM.

[41]

John W Jones and G Anne Bogat. 1978. Air pollution and human aggression. Psychol. Rep. (1978).

[42]

Laura June. 2015. I'm Voting for Hillary Because of My Daughter. http://thecut.io/1rNEeBJ. (2015).

[43]

Joseph M Kayany. 1998. Contexts of uninhibited online behavior: flaming in social newsgroups on Usenet. J. Am. Soc. Inf. Sci. (1998).

Digital Library

[44]

Dacher Keltner, Phoebe C Ellsworth, and Kari Edwards. 1993. Beyond simple pessimism: effects of sadness and anger on social perception. J. Pers. Soc. Psychol. (1993).

[45]

Philip C Kendall, W Robert Nay, and John Jeffers. 1975. Timeout duration and contrast effects: A systematic evaluation of a successive treatments design. Behavior Therapy (1975).

[46]

Peter G Kilner and Christopher M Hoadley. 2005. Anonymity options and professional participation in an online community of practice. In Proc. CSCL.

Digital Library

[47]

Ben Kirman, Conor Lineham, and Shaun Lawson. 2012. Exploring mischief and mayhem in social computing or: how we learned to stop worrying and love the trolls. In Proc. SIGCHI Extended Abstracts.

Digital Library

[48]

Silvia Knobloch-Westerwick and Scott Alter. 2006. Mood adjustment to social situations through mass media use: How men ruminate and women dissipate angry moods. Hum. Commun. Res. (2006).

[49]

Agata Kołakowska. 2013. A review of emotion recognition methods based on keystroke dynamics and mouse movements. In Proc. HSI.

[50]

Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. PNAS (2014).

[51]

Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012a. Supporting reflective public thought with considerit. In Proc. CSCW.

Digital Library

[52]

Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012b. Is this what you meant?: promoting listening on the web with reflect. In Proc. SIGCHI.

Digital Library

[53]

Cliff Lampe and Paul Resnick. 2004. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proc. SIGCHI.

Digital Library

[54]

Hangwoo Lee. 2005. Behavioral strategies for dealing with flaming in an online forum. Sociol Q (2005).

[55]

So-Hyun Lee and Hee-Woong Kim. 2015. Why people post benevolent and malicious comments online. Commun. ACM (2015).

Digital Library

[56]

Karen Pezza Leith and Roy F Baumeister. 1996. Why do bad moods increase self-defeating behavior? Emotion, risk tasking, and self-regulation. In J. Pers. Soc. Psychol.

[57]

Jerome M Levine and Gardner Murphy. 1943. The learning and forgetting of controversial material. J. Abnorm. Soc. Psychol. (1943).

[58]

Tanya Beran Qing Li. 2005. Cyber-harassment: A study of a new method for an old behavior. In J. Educ. Comput. Res.

[59]

Nina Mazar, On Amir, and Dan Ariely. 2008. The dishonesty of honest people: A theory of self-concept maintenance. J. Mark. Res. (2008).

[60]

Brian James McInnis, Elizabeth Lindley Murnane, Dmitry Epstein, Dan Cosley, and Gilly Leshed. 2016. One and Done: Factors affecting one-time contributors to ad-hoc online communities. In Proc. CSCW.

Digital Library

[61]

DM McNair. 1971. Manual profile of mood states.

[62]

Stanley Milgram, Leonard Bickman, and Lawrence Berkowitz. 1969. Note on the drawing power of crowds of different size. J. Pers. Soc. Psychol. (1969).

[63]

Stanley Milgram and Ernest Van den Haag. 1978. Obedience to authority. (1978).

[64]

Lev Muchnik, Sinan Aral, and Sean J Taylor. 2013. Social influence bias: A randomized experiment. Science (2013).

[65]

J. Nathan Matias. 2016. Posting Rules in Online Discussions Prevents Problems & Increases Participation. http://bit.ly/2foNQ3K. (2016).

[66]

Vlad Niculae and Cristian Danescu-Niculescu-Mizil. 2016. Conversational markers of constructive discussions. In Proc. NAACL.

[67]

Deokgun Park, Simranjit Sachar, Nicholas Diakopoulos, and Niklas Elmqvist. 2016. Supporting Comment Moderators in Identifying High Quality Online News Comments. In Proc. SIGCHI.

Digital Library

[68]

James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001.

[69]

Adrian Raine. 2002. Annotation: The role of prefrontal deficits, low autonomic arousal, and early health factors in the development of antisocial and aggressive behavior in children. J. Child Psychol. Psychiatry (2002).

[70]

Juliana Raskauskas and Ann D Stoltz. 2007. Involvement in traditional and electronic bullying among adolescents. Dev. Psychol. (2007).

[71]

Emmett Rensin. 2014. Confessions of a former internet troll. Vox (2014).

[72]

Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika (1983).

[73]

Lee Ross. 1977. The intuitive psychologist and his shortcomings: Distortions in the attribution process. Adv. Exp. Soc. Psychol. (1977).

[74]

James Rotton and James Frey. 1985. Air pollution, weather, and violent crimes: concomitant time-series analysis of archival data. J. Pers. Soc. Psychol. (1985).

[75]

Paul Rozin and Edward B Royzman. 2001. Negativity bias, negativity dominance, and contagion. Pers. Soc. Psychol. Rev. (2001).

[76]

Matthew J Salganik, Peter Sheridan Dodds, and Duncan J Watts. 2006. Experimental study of inequality and unpredictability in an artificial cultural market. Science (2006).

[77]

Mattathias Schwartz. 2008. The trolls among us. NY Times Magazine (2008).

[78]

Norbert Schwarz and Herbert Bless. 1991. Happy and mindless, but sad and smart? The impact of affective states on analytic reasoning. Emotion and Social Judgments (1991).

[79]

Norbert Schwarz and Gerald L Clore. 1983. Mood, misattribution, and judgments of well-being: informative and directive functions of affective states. J. Pers. Soc. Psychol. (1983).

[80]

Pnina Shachaf and Noriko Hara. 2010. Beyond vandalism: Wikipedia trolls. J. Inf. Sci. (2010).

Digital Library

[81]

Muzafer Sherif, Oliver J Harvey, B Jack White, William R Hood, Carolyn W Sherif, and others. 1961. Intergroup conflict and cooperation: The Robbers Cave experiment.

[82]

Robert Slonje, Peter K Smith, and Ann Fris� N. 2013. The nature of cyberbullying, and strategies for prevention. Comput. Hum. Behav. (2013).

Digital Library

[83]

Abhay Sukumaran, Stephanie Vezich, Melanie McHugh, and Clifford Nass. 2011. Normative influences on thoughtful online participation. In Proc. SIGCHI.

Digital Library

[84]

John Suler. 2004. The online disinhibition effect. Cyberpsychol. Behav. (2004).

[85]

Peter C Terry and Andrew M Lane. 2000. Normative values for the Profile of Mood States for use with athletic samples. J. Appl. Sport Psychol. (2000).

[86]

Kris Varjas, Jasmaine Talley, Joel Meyers, Leandra Parris, and Hayley Cutts. 2010. High school students perceptions of motivations for cyberbullying: An exploratory study. West. J. Emerg. Med. (2010).

[87]

Lu Wang and Claire Cardie. 2014. A Piece of My Mind: A Sentiment Analysis Approach for Online Dispute Detection. In Proc. ACL.

[88]

Yang Wang, Gregory Norcie, Saranga Komanduri, Alessandro Acquisti, Pedro Giovanni Leon, and Lorrie Faith Cranor. 2011. I regretted the minute I pressed share: A qualitative study of regrets on Facebook. In Proc. SUPS.

Digital Library

[89]

Ladd Wheeler. 1966. Toward a theory of behavioral contagion. Psychol. Rev. (1966).

[90]

David Wiener. 1998. Negligent publication of statements posted on electronic bulletin boards: Is there any liability left after Zeran? Santa Clara L. Rev. (1998).

[91]

Robb Willer, Ko Kuwabara, and Michael W Macy. 2009. The False Enforcement of Unpopular Norms. Am. J. Sociol. (2009).

[92]

James Q Wilson and George L Kelling. 1982. Broken Windows. Atlantic Monthly (1982).

[93]

Carrie L Wyland and Joseph P Forgas. 2007. On bad mood and white bears: The effects of mood state on ability to suppress unwanted thoughts. Cognition and Emotion (2007).

[94]

Justine Zhang, Ravi Kumar, Sujith Ravi, and Cristian Danescu-Niculescu-Mizil. 2016. Conversational flow in Oxford-style debates. In Proc. NAACL.

[95]

Philip G Zimbardo. 1969. The human choice: Individuation, reason, and order versus deindividuation, impulse, and chaos. Nebr. Symp. Motiv. (1969).

Cited By

Шевкунова Т(2024)Нивелирования токсичности и управления коммуникацией субъектов с ярко выраженными ограничивающими убеждениямиModern Humanities Success10.58224/2618-7175-2024-8-80-85(80-85)Online publication date: 29-Aug-2024
https://doi.org/10.58224/2618-7175-2024-8-80-85
Navarro-Conticello J(2024)Un otro horroroso: imaginarios sociales sobre el inmigrante venezolano en las audiencias de la prensa online chilenaEstudios Fronterizos10.21670/ref.241415025Online publication date: 30-Aug-2024
https://doi.org/10.21670/ref.2414150
Jin SBouvier G(2024)Cancel Culture and Trigger-Ready Fragmented Interest Groups: The Case of Depp Versus Amber HeardTelevision & New Media10.1177/15274764241277465Online publication date: 4-Sep-2024
https://doi.org/10.1177/15274764241277465
Show More Cited By

Index Terms

Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions
1. Applied computing
  1. Law, social and behavioral sciences
2. Information systems
  1. Information systems applications

Recommendations

The Bad Boys and Girls of Cyberspace: How Gender and Context Impact Perception of and Reaction to Trolling

A significant feature of virtual interactions involve online deviance and trolling; these include behaviors that range from mild mischief, to offensive language, to hacking and trolling, and to expression of complex social problem, such as by ...
From Royals to Vegans: Characterizing Question Trolling on a Community Question Answering Website
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

The phenomenon of trolling has emerged as a widespread form of abuse on news sites, online social networks, and other types of social media. In this paper, we study a particular type of trolling, performed by asking a provocative question on a community ...
Identity and User Behavior in Online Communities
GROUP '20: Companion Proceedings of the 2020 ACM International Conference on Supporting Group Work

In online communities, people share and discuss information at all levels of topic sensitivity. Identity policies within these communities range from real names to anonymity. The amount of user engagement, the quality of the information, disinformation ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

February 2017

2556 pages

ISBN:9781450343350

DOI:10.1145/2998181

General Chairs:
Charlotte P. Lee
University of Washington
,
Steve Poltrock
Retired
,
Program Chairs:
Louise Barkhuus
Stockholm University and Cornell Tech
,
Marcos Borges
Universidade Federal do Rio de Janeiro
,
Wendy Kellogg
IBM T.J. Watson Research Center

Copyright � 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

CSCW '17

Sponsor:

SIGCHI

CSCW '17: Computer Supported Cooperative Work and Social Computing

February 25 - March 1, 2017

Oregon, Portland, USA

Acceptance Rates

CSCW '17 Paper Acceptance Rate 183 of 530 submissions, 35%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

292
Total Citations
View Citations
9,318
Total Downloads

Downloads (Last 12 months)1,966
Downloads (Last 6 weeks)192

Reflects downloads up to 21 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Шевкунова Т(2024)Нивелирования токсичности и управления коммуникацией субъектов с ярко выраженными ограничивающими убеждениямиModern Humanities Success10.58224/2618-7175-2024-8-80-85(80-85)Online publication date: 29-Aug-2024
https://doi.org/10.58224/2618-7175-2024-8-80-85
Navarro-Conticello J(2024)Un otro horroroso: imaginarios sociales sobre el inmigrante venezolano en las audiencias de la prensa online chilenaEstudios Fronterizos10.21670/ref.241415025Online publication date: 30-Aug-2024
https://doi.org/10.21670/ref.2414150
Jin SBouvier G(2024)Cancel Culture and Trigger-Ready Fragmented Interest Groups: The Case of Depp Versus Amber HeardTelevision & New Media10.1177/15274764241277465Online publication date: 4-Sep-2024
https://doi.org/10.1177/15274764241277465
Pasquier VBarros MDaudigeos T(2024)Trolling the Leviathan: How the use of social media by democratic organizations engenders monstersOrganization10.1177/13505084241238278Online publication date: 7-Apr-2024
https://doi.org/10.1177/13505084241238278
Mungall LPruysers SBlais J(2024)Online and Unkind: Examining the Personality Correlates of Online Political IncivilitySocial Science Computer Review10.1177/08944393241249725Online publication date: 30-Apr-2024
https://doi.org/10.1177/08944393241249725
Munn L(2024)Misinformation’s missing humanMedia, Culture & Society10.1177/0163443724124916446:6(1287-1298)Online publication date: 28-Apr-2024
https://doi.org/10.1177/01634437241249164
Walther J(2024)The Effects of Social Approval Signals on the Production of Online Hate: A Theoretical ExplicationCommunication Research10.1177/00936502241278944Online publication date: 14-Sep-2024
https://doi.org/10.1177/00936502241278944
Baumer ETaylor ABrubaker JMcGee M(2024)Algorithmic SubjectivitiesACM Transactions on Computer-Human Interaction10.1145/366034431:3(1-34)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3660344
Kim HLee JJang JKim J(2024)ReSPect: Enabling Active and Scalable Responses to Networked Online HarassmentProceedings of the ACM on Human-Computer Interaction10.1145/36373948:CSCW1(1-30)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637394
Scholz FKolb TNeidhardt JBoratto LGena CMarras MGermanakos PPopescus E(2024)Classifying User Roles in Online News Forums: A Model for User Interaction and Behavior AnalysisAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3665187(240-249)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3631700.3665187
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents