Academic Scientific JournalsOpinionated texts in social media: A proposal for evaluative judgement methodology
DOI:
https://doi.org/10.26881/bp.2024.1.07Keywords:
opinion, social media, evaluative language, incivilityAbstract
The paper reports on a pilot study devoted to the analysis of opinions expressed by Facebook discussants related to the immigration crisis in Europe. Opinions were conceptualised as the subjective expression of evaluation. The data were investigated in line with an original, tripartite typology consisting of 46 criteria of evaluative language coming from distinct scholarly traditions: media studies, linguistics and Critical Discourse Analysis. Generally speaking, the proposed templates proved effective when mapped onto social media data; however, as there is a partial overlap of some of the categories used in the pilot study, the number of categories needs to be reduced in the subsequent investigation. The secondary objective of this paper is to present computational tools employed in the study, which, as the study shows, can enrich research methodology in communication studies, and these comprise the “R” package used for data visualisation and lexical and sentence-based embeddings based on LLM.
Downloads
References
Alba-Juez, Laura, Geoff Thompson (2014). “The many faces and phases of evaluation”. In: Geoff Thompson, Laura Alba-Juez (eds.). Evaluation in Context. Amsterdam: John Benjamins Publishing Company, 3–23.
Åkerlund, Mathilda (2022). “Dog whistling far-right code words: The case of ‘culture enricher’ on the Swedish web”. Information, Communication and Society 25/12: 1808–1825. https://doi.org/10.1080/1369118X.2021.1889639.
Anstead Nick, Ben O’Loughlin (2015). “Social media analysis and public opinion: The 2010 UK General Election”. Journal of ComputerMediated Communication 20: 119–239.
Bączkowska, Anna (2022a). Explicit and implicit offensiveness in dialogical film discourse in Bridget Jones films. International Review of Pragmatics 14: 198–225. https://doi.org/10.1163/ 18773109- 01402003.
Bączkowska, Anna (2022b). “‘Hope you have a shit birthday you fat cunt’ – cognitive strategies, rhetorical figures and linguistic means used in insulting tweets”. Ateneum Philological Forum 10/1: 9–25.
Bączkowska, Anna (2023). “Implicit offensiveness from linguistic and computational perspectives: A study of irony and sarcasm”. Lodz Papers in Pragmatics 19/2: 353–383. https://doi.org/10.1515/ lpp-2023-0018.
Bączkowska, Anna, Barbara Lewandowska-Tomaszczyk, Giedre Valunaite Oleškevičiene, Slavko Žitnik, Chaya Liebeskind (2022). “A taxonomy of implicit language”. Nexus Workshops Days in Jerusalem. Jerusalem, May 23–24, 2022.
Bączkowska, Anna, Katarzyna Kukowicz-Żarska, Aneta Dłutek, Iwona Benenowska (2024). “Ironic opinions on Polish Reddit: Description and computational methods of analysis”. Beyond Philology 21/1: 91–141.
Bednarek, Monika (2006). Evaluation in Media Discourse: Analysis of a Newspaper Corpus. London: Continuum.
Biber, Douglas (2006). Stance in spoken and written university registers. Journal of English for Academic Purposes 5: 97–116.
Biber, Douglas, Edward Finegan (1989). “Styles of stance in English: Lexical and grammatical marking of evidentiality and affect”. Text 9/1: 93–124.
Bierwisch, Manfred (1989). “The semantics of gradation”. In: Manfred Bierwisch, Ewald Lang (eds.). Dimensional Adjectives: Grammatical Structure and Conceptual Interpretation. Berlin: Springer, 71– 261.
Boromisza-Habashi, David (2013). Speaking Hatefully:Culture, Communication, and Political Action in Hungary. University Park: The Pennsylvania State University Press.
Bruce, Rebecca F., Janyce M. Wiebe, (1999). “Recognising subjectivity: A case study of manual tagging”. Natural Language Engineering 5/2: 187–205. https://doi.org/10.1017/S1351324 999002181.
Burnette, Catherine E., Charles R. Figley (2017). “Historical oppression, resilience, and transcendence: Can a holistic framework help explain violence experienced by indigenous people?”. Social Work 62/1: 37–44. https://doi.org/10.1093/sw/sww065.
Buzan, Bary, Ole Wæver, Jaap de Wilde (1998). Security: A New Framework for Analysis. Boulder – London: Rienner.
Bybee Joan, L., Suzanne Fleischman (1995). Modality in Grammar and Discourse. Amsterdam: John Benjamins.
Carlson, Matt (2018). “Facebook in the news”. Digital Journalism 6/1: 4–20. https://doi.org/10.1080/21670811.2017.1298044.
Castells, Manuel (2009). Communication Power. Oxford: Oxford University Press.
Castells, Manuel (2012). Networks of Outrage and Hope: Social Movements in the Internet Age. Cambridge: Polity Press.
Cepollaro, Bianca (2020). Slurs and Thick Terms: When Language Encodes Values. Lanham: Lexington Books.
Conrad, Susan, Douglas Biber (2000). “Adverbial marking of stance in speech and writing”. In Susan Hunston, Geoff Thompson (eds). Evaluation in Text: Authorial Stance and the Construction of Discourse. Oxford: Oxford University Press, 56–73.
Culpeper, Jonathan (2011). Impoliteness: Using Language to Cause Offence. Cambridge: Cambridge University Press.
Croom, Adam M. (2011). “Slurs”. Language Sciences 33: 343–358.
Dewaele, Jean-Marc (2016). “Thirty shades of offensiveness: L1 and LX English users’ understanding, perception and self-reported use of negative emotion-laden words”. Journal of Pragmatics 94: 112–127.
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2018). “Bert: Pre-training of deep bidirectional transformers for language understanding”. arXiv preprint arXiv:1810.04805.
Eberl, Jakob-Moritz, Christine E. Meltzer, Tobias Heidenreich, Beatrice Herrero, Nora Theorin, Fabienne Lind, Rosa Berganza, Hajo G. Boomgaarden, Christian Schemer, Jesper Strömbäck (2018). “The European media discourse on immigration and its effects: A literature review”. Annals of the International Communication Association 42/3: 207–223. https://doi.org/10.1080/23808985.2018.1497452.
Englebretson, Robert (2007). “Stancetaking in discourse: An introduction”. In: Robert Englebretson (ed). Stancetaking in Discourse: Subjectivity, Evaluation, Interaction. Amsterdam: John Benjamins, 1–26.
Goodwin, Matthew, Caitlin Milazzo (2017). “Taking back control? Investigating the role of immigration in the 2016 Vote for Brexit”. The British Journal of Politics and International Relations 19/3: 450–464. https://doi.org/10.1177/1369148117710799.
Guillén-Nieto, Victoria (2023). “The wording of hate speech prohibition: “You can’t see the wood for the trees””. In: Victoria GuillénNieto, Antonio Doval Pais, Dieter Stein (eds). From Fear to Hate: Legal-Linguistic Perspectives on Migration. Berlin: De Gruyter, 173–200.
Guzek, Damian (2023). “When partisan groups get access to the digital society: Re-voicing religion in Poland”. Information, Communication and Society 26/6: 1177–1192. https://doi.org/10.1080/1369118X.2021.1994628.
Halliday, Michael A. K. (1994). An Introduction to Functional Grammar. 2nd edition. London: Edward Arnold.
Herring, Susan (2010). “Computer-mediated conversation: Introduction and overview”. Language@Internet 7: article 2.
Hess, Agnieszka (2013). Społeczni uczestnicy dyskursu politycznego w Polsce. Kraków: Jagiellonian University Press.
Hogan, Jackie, Kristin Haltinner (2015). “Floods, invaders, and parasites: Immigration threat narratives and right-wing populism in the USA, UK and Australia”. Journal of Intercultural Studies 36/5: 520–543. https://doi.org/10.1080/07256868.2015.1072 907.
Hunston, Susan (1994). “Evaluation and organisation in academic discourse”. In: Malcolm Coulthard (ed.). Advances in Written Text Analysis. London: Routledge, 191–218.
Hunston, Susan (2004). “Counting the uncountable: Problems of identifying evaluation in a text and in a corpus”. Corpora and Discourse 9: 157–188.
Hunston, Susan, Geoff Thompson (1999). Evaluation in Text: Authorial Stance and the Construction of Discourse. Oxford: Oxford University Press.
Jaszczyk-Grzyb, Magdalena, Anna Szczepaniak-Kozak, Sylwia Adamczak-Krysztofowicz (2023). “A corpus-assisted critical discourse analysis of hate speech in German and Polish social media posts”. Moderna Språk 117/1: 44–71. https://doi.org/10.58221/mosp.v117i1.11518.
Joppke, Christian (2020). “Immigration in the populist crucible: Comparing Brexit and Trump”. Comparative Migration Studies 8/1: 1–18. https://doi.org/10.1186/s40878-020-00208-y.
Kondor, Katherine, Sabina Mihelj, Václav Štětka, Fanni Tóth (2022). “News consumption and immigration attitudes: A mixed methods approach”. Journal of Ethnic and Migration Studies 48/17: 4129–48. https://doi.org/10.1080/1369183X.2022.2054786.
Kennedy, Christopher (2013). “Two sources of subjectivity: Qualitative assessment and dimensional uncertainty”. Inquiry 56/2–3: 258–277.
Lewandowska-Tomaszczyk, Barbara, Slavko Žitnik, Anna Bączkowska, Chaya Liebeskind, Jelena Mitrović, Giedre Valunaite Oleskeviciene (2021). “LOD-connected offensive language ontology and tagset enrichment”. In: Sara Carvalho, Renato Rocha Souza (eds). Proceedings of the Workshops and Tutorials Held at LDK 2021 Co-located with the 3rd Language, Data and Knowledge Conference, 135–150. CEUR Workshop Proceedings. Zaragossa, Spain.
Lewandowska-Tomaszczyk, Barbara, Chaya Liebeskind, Slavko Žitnik, Anna Bączkowska, Giedre Valunaite Oleškevičiene, Marcin Trojszczak (2022). “An offensive language taxonomy and a webcorpus discourse analysis for automatic offensive language identification”. Presentation at 3rd International Conference: Approaches to Digital Discourse Analysis (ADDA 3). St Petersburg, Florida, May 13–15, 2022.
Lewandowska-Tomaszczyk, Barbara, Slavko Žitnik, Chaya Liebeskind, Giedre Valunaite Oleskevicienė, Anna Bączkowska, Paul A. Wilson, Marcin Trojszczak, Ivana Brač, Lobel Filipić, Ana Ostroški Anić, Olga Doncheva-Navratilova, Agnieszka Borowiak, Kristina Despot, Jelena Mitrović (2023a). “Annotation scheme and evaluation: The case of offensive language”. Rasprave: Časopis Instituta za Hrvatski Jezik i Jezikoslovlje 49/1: 155–175. https://doi.org/10.31724/rihjj.49.1.8.
Lewandowska-Tomaszczyk, Barbara, Anna Bączkowska, Chaya Liebeskind, Giedre Valunaite Oleskeviciene, Slavko Žitnik (2023b). “An integrated explicit and implicit offensive language taxonomy”. Lodz Papers in Pragmatics 19/1: 7–48. https://doi.org/10.1515/lpp-2023-0002.
Lipiński, Artur (2017). “Legitymizacja władzy w perspektywie dyskursywnej: ujęcie Theo van Leeuwena”. Studia Polityczne 1: 225–241.
Lipiński, Artur (2019). “Constructing ‘The Others’ as a populist communication strategy: The case of the ‘refugee crisis’ in discourse in the Polish press”. In: Agnieszka Stępińska (ed). Populist Discourse in the Polish Media. Poznań: WNPID,155–180.
Ljubešić, Nikola, Darja Fišer, Tomaž Erjavec Ajda Šulc (2021). “Offensive language dataset of Croatian, English and Slovenian comments FRENK 1.1”. Ljubljana: Slovenian language resource repository CLARIN.SI. Available at <http://hdl.handle.net/11356/1462>. Accessed 15.09.2023.
Lombart, Erika (2022). L’implicite sur les réseaux sociaux: Entre les lignes des forums de discussion. Paris: L’Harmattan.
López-Paredes, Marco, Branco Di Fátima (2023). “Memética: La reinvención de las narrativas en el mundo digital, protestas sociales y discursos de odio”. In: Oliver Carrero Márquez, Alicia Parras Parras (eds). Visiones contemporáneas: Narrativas, escenarios y ficciones. Madrid: Fragua, 25–37.
Lyons, John (1977). Semantics. Cambridge: Cambridge University Press.
Mafeza, Faustin (2016). “Preventing genocide by fighting against hate speech”. International Journal of Advanced Research 4/3: 117–132.
Martin, James R. (2000). “Beyond exchange: Appraisal systems in English”. In: Susan Hunston Geoff Thompson (eds). Evaluation in Text: Authorial Stance and the Construction of Discourse. Oxford –New York: Oxford University Press, 142–175.
Martin, James R., Peter R. White (2003). The Language of Evaluation: Appraisal in English. London: Palgrave Macmillan.
Marwick, Alice, Ross Miller (2014). “Online harassment, defamation, and hateful speech: A primer of the legal landscape”. Fordham Center on Law and Information Policy (Report June 10, 2014), 1–73.
McInnes, Leland, John Healy, Steve Astels (2017). “Hdbscan: Hierarchical density based clustering”. Journal of Open Source Software 2/11: 205.
McInnes, Leland, John Healy, James Melville (2018). “Umap: Uniform manifold approximation and projection for dimension reduction”. arXiv preprint arXiv:1802.03426.
Moran, Mayo (1994). “Talking about hate speech: A rhetorical analysis of American and Canadian approaches to the regulation of hate speech”. Wisconsin Law Review 6: 1425–1514.
Musolf, Andreas (2017). “Dehumanising metaphors in UK immigrant debates in press and online media”. In: Monika Kopytowska (ed.). Contemporary Discourses of Hate and Radicalism across Space and Genres. Amsterdam: John Benjamins, pp. 41–56.
Nockleby, John (2000). Hate speech. In: Leonard W. Levy, Kenneth L. Karst, Adam Winkler (eds.), Encyclopedia of the American Constitution. New York: Macmillan, 1277–1278.
Nowak-Teter, Ewa (2024). Opinia publiczna online. Lublin: Wydawnictwo UMCS.
Ochs, Elinor (ed.) (1989). The Pragmatic of Affect. Special Issue of Text 9/1.
Odrowąż-Sypniewska, Joanna (2022). ”Vagueness in Natural Language”. In: Piotr Stalmaszczyk (ed.). The Cambridge Handbook of the Philosophy of Language. Cambridge: Cambridge University Press, 300–316.
Partington, Alan. 2007. “Irony and reversal of evaluation”. Journal of Pragmatics 39: 1547–1569. doi.org/10.1016/j.pragma.2007.04.009.
Pavlik, John V., Shawn McIntosh (2018). Converging Media: A New Introduction to Mass Communication. 6th ed. New York: Oxford University Press.
Ponsonnet, Maia (2014). The Language of Emotions: The Case of Dalabon (Australia). Amsterdam: John Benjamins.
Reimers, Nils, Iryna Gurevych (2019). “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks”. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: Association for Computational Linguistics, 3982–3992.
Scheffler, Samuel (1987). “Morality through thick and thin: A critical notice of Ethics and the Limits of Philosophy”. Philosophical Review 96/3: 411–34. https://doi.org/10.2307/2185227.
Sellars, Andrew F. (2016). Defining Hate Speech. Boston University School of Law, Public Law Research Paper No. 16–48. Available at SSRN: https://ssrn.com/abstract=2882244. Accessed 20.10.2023.
Soria Ruiz, Andrés, Bianca Cepollaro, Isidora Stojanovic (2022). “The semantics and pragmatics of value judgments”. In: Piotr Stalmaszczyk. The Cambridge Handbook of the Philosophy of Language. Cambridge: Cambridge University Press, 434–449.
Stubbs, Michael (1986). “A matter of prolonged fieldwork: Notes towards a modal grammar of English”. Applied Linguistics 7: 1–25.
Szulz, Winfried (2004). “Reconstructing mediatization as an analytical concept”. European Journal of Communication 19/1: 87–101.
Taboada, Maite, Radoslava Trnavac (2013). “Introduction”. In: MaiteTaboada, Radoslava Trnavac (eds). Nonveridicality and Evaluation: Theoretical, Computational and Corpus Approaches. Leiden: Brill.
Thompson, Geoff and Hunston, Susan (2000). Evaluation: An introduction. In: Susan Hunston, Geoff Thompson (eds). Evaluation in text: Authorial Stance and the Construction of Discourse. Oxford: Oxford University Press, 1–27.
Ullmann, S. (1972). Semantics: An Introduction to the Science of Meaning. Oxford: Blackwell.
Van Dijck, José (2013). The Culture of Connectivity: A Critical History of Social Media. Oxford: Oxford University Press.
Van Dijk, Teun A. (1998). Ideology: A Multidisciplinary Approach. London: SAGE Publications.
Van Leeuwen, Theo (1996). “The representation of social actors”. In: Carmen Rosa Caldas-Coulthard, Malcolm Coulthard (eds). Texts and Practices: Readings in Critical Discourse Analysis. London: Routledge, 32–70.
Van Leeuwen, Theo (2008). Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford: Oxford University Press.
Van Linden, An (2012). Modal Adjectives: English Deontic and Evaluative Constructions in Synchrony and Diachrony. Berlin: De Gruyter Mouton.
Väyrynen, Pekka (2011). “Thick concepts and variability”. Philosophers’ Imprint 11: 1–17.
Väyrynen, Pekka (2013). The Lewd, the Rude, and the Nasty. Oxford University Press.
Väyrynen, Pekka (2021). “Normative explanation and justification”. Noûs 55/1: 3–22.
Waldron, Jeremy (2010). Dignity and defamation: The visibility of hate. Harvard Law Review 123/7: 1596–1657.
Waltman, Michael, John Haas (2011). The Communication of Hate. Frankfurt: Peter Lang
White, P.R.R. (2015). “Appraisal Theory”. In: Karen Tracy, Cornelia Ilie, Todd Sandel (eds). The International Encyclopedia of Language and Social Interaction. London: Wiley-Blackwell.
