The Ongoing Trouble With Counter-Narratives: Why Evaluation May Not Be Everything

12.08.2020

Photo: Illustration

The last ten years have seen a stark increase in the proliferation of counter-narratives, alternative narratives and strategic communication campaigns used as soft power tools to mitigate the risk of radicalization into extremism. These measures rest on the assumption that radicalization is partially contingent upon the consumption of extremist material either online or offline; that is, individuals need to be socialized into what radicalism expert Dr. Haroro J. Ingram calls a “competitive system of meaning” through exposure to ideologically-charged narratives.[1] In other words, in order for the radicalization process to progress, individuals need to adopt extremist views and ‘narratives’ through which they judge the world. While there is no question that propagandistic materials are often consumed by extremists,[2] some have expressed criticism of this assumption and challenge the notion that exposure to ideology via narratives is a necessary condition for radicalization — even calling it a “preposterous idea”.[3] Nevertheless, many political actors — and by extension the civil society organizations they fund — seem to base their measures for preventing and countering violent extremism (P/CVE) on precisely this assumption. This is especially pronounced in the EU, where the UK has arguably pushed a successful counter-narrative framework,[4] which has now become an important theoretical foundation for both EU and national-level policy in various member states.[5]

If one assumes that radicalization partially depends on exposure to and adoption of certain narratives, it follows logically that counter-measures must mitigate the risk of extremist narratives. More hard power measures, such as the removal of extremist content from online platforms, are complemented by the development and dissemination of alternative and counter-narratives. Counter-narratives are aimed at delegitimizing extremist narratives by, for instance, exposing their hypocrisy or showing that violence is not an adequate means to facilitate political change.[6] While counter-narratives are often reactive, alternative narratives proactively display positive content and show what we are ‘for’ rather than ‘against’.[7] Both types of narrative approaches have been used both online and offline by governments — for instance, the US and the UK, as well as a multitude of civil society organizations.

Cost vs. Benefit

Whenever a policy area receives as many funds as P/CVE currently does, immediately there are calls for evaluation, and counter-narratives are no exception in this regard. Recent years have seen discussions surrounding how much ‘value for money’ such campaigns achieve.[8] Also, the fact that very few P/CVE programs have been evaluated by independent researchers has drawn ample criticism.[9] After all, so the argument goes, it could very well be that millions of Euros are invested into approaches that yield few, if any, provable positive results. In the worst case scenario, counter-narratives can even backfire and have counterproductive effects, making evaluation even more crucial.[10]

However, the list of obstacles to effective evaluation of counter-narrative campaigns is long and seemingly intractable. The problems mentioned here are but a snapshot of the various issues surrounding evaluation of counter-narrative campaigns.

Obstacles to Evaluation

For starters, it is extremely difficult to prove a negative event — that is, a non-radicalization. Given that radicalization is believed to be the product of a ‘kaleidoscope’ of factors on multiple levels pertaining to both the individual and his/her social environment,[11] it comes as no surprise that a non-radicalization is also impossible to trace back to a single factor such as exposure to a counter-narrative. It is very unlikely that counter-narratives alone are responsible for mitigating the risk of radicalization.

Secondly, while other interventions aimed at non-outcomes such as anti-drug campaigns or programs to prevent teen pregnancy can be evaluated using thousands of participants, P/CVE campaigns are, by their very nature, aimed at a tiny minority. There are few at risk of radicalizing and even fewer who would then act upon such a radicalization. This also means that no reasonable comparative study can be carried out, because it is unlikely to find a control group similar enough to the ‘at risk’ group, yet themselves not ‘at risk’.[12]

Third, in conjunction with such conceptual issues, there are practical concerns regarding evaluations. Civil society actors may not have the financial means to hire independent evaluators and may be reluctant to do so even if they could, as a negative evaluation may lead to the reduction or withdrawal of funds. Therefore, it might be the case that internal evaluations are carried out but are simply not made public. There also remains the question of which type of ‘success’ such evaluations would measure. Is a campaign a success only when no one in the target group ever radicalizes again? If not, how many can radicalize in order for the campaign to be considered successful? Is it even necessary to prove a non-radicalization? And if there is an agreed-upon determinant of success, which methodology is appropriate for measuring it?[13]

Furthermore, the online sphere may pose specific problems for evaluation. For instance, counter-narratives are often disseminated via social media, but traditional engagement measures used in advertising such as likes, shares or comments may be a good indicator of reach, but not of actual impact. A campaign can generate thousands of likes and still not reach its intended audience.

In short, with the tools currently available, attempting to evaluate counter-narrative campaigns and proving their effectiveness with an adequate degree of certainty is mission impossible. While evaluation certainly needs to be discussed continuously, it is unwise to question the usefulness of counter-narratives solely based on the lack of proof that they directly counter radicalization processes.

Counter-Narratives Critical In Shaping Public Discourse

In fact, counter-narratives have value beyond the sphere of counter-radicalization practices. Much has been written on the increasing polarization of societal discourse, especially in the online sphere. While extremists do have their own ‘ecosystems’ online,[14] one does not need to dive deep into the extremist abyss of ‘chan-culture’ to find hate speech, discrimination, cyber-bullying and generally heated, uncompromising discourse, increasingly devoid of any middle ground. This can also be observed on mainstream social media websites and many newspapers and blogs are shutting down their comment sections to prevent discourses from being hijacked by trolls or flooded with hate speech.[15] Part of this is due to what psychologists have termed the online disinhibition effect;[16] that is, that people feel more comfortable expressing non-mainstream opinions in an anonymous online world. However, it is also partially by design. Alt-right expert and author Nagle,[17] for instance, argues that right-wing extremists deliberately seek to move the Overton window of what can and cannot be publicly said to shift public discourse towards accepting more far-right terminology. For instance, it has been shown that comment sections are viewed as cues for social norms and can therefore impact both attitudes and behaviors.[18]

Because of polarizing discourse patterns online, counter-narratives and especially alternative narratives have value beyond their role in counter-radicalization processes. They are valuable tools to ‘contest the space’, not only of extremists, but of hateful discourse more generally. Discourse and language matter and with an increasingly intertwined online and offline world, online discourse matters even more. The frames and narratives we are exposed to shape our perceptions of the world. If civil society actors do not counteract hateful discourses online, it is only a matter of time until the brutalization of online language has offline effects: at the ballot booth, in parliaments and in social relations more generally. This is dangerous because healthy democracies are dependent upon public perceptions and attitudes. Positive, alternative narratives are needed as a counterweight to extremist, but also non-extremist discourses, endangering democracy by shifting society’s perceptions. Even if no one is ever stopped from radicalizing because they were exposed to a counter-narrative, these campaigns are of value for social relations as a whole.

Conclusion

It has long been established that framing and discourses matter because they shape what we perceive as real. Therefore, while evaluations are an important route to judge the value of counter-narratives, they cannot and should not be the only one. Those responsible for funding, developing, and implementing counter-narratives and especially those tasked with judging their effectiveness, would do well to move beyond the direct effects of counter-narratives on radicalization processes. Instead, the indirect effects these narratives have by shaping societal discourses must be taken into account. This is admittedly difficult in a ‘value for money’ policy approach. However, it is necessary to reconcile measurements of effectiveness with the broader effects counter-narratives may have. After all, we need to ask about the potential implications of not contesting the space, not investing in positive discourses and not seeking to prevent the shift of the Overton window for the future of liberal democracies.

_________________________________________________________________

References:

[1] Ingram, H. “Deciphering the Siren Call of Militant Islamist Propaganda: Meaning, Credibility & Behavioural Change.” 2016, https://icct.nl/wp-content/uploads/2016/09/ICCT-Ingram-Deciphering-the-Siren-Call-of-Militant-Islamist-Propaganda-September2016.pdf

[2] Gill, P., Corner, E., Conway, M., Thornton, A., Bloom, M. and Horgan, J. “Terrorist Use of the Internet by Numbers: Quantifying behaviors, patterns, and processes.” Criminology & Public Policy, vol. 16, no. 1, 2017, pp. 99-117

[3] Awan, A., Miskimmon, A. and O’Loughlin, B. “The Battle for the Battle of the Narratives: Sidestepping the double fetish of digital and CVE” in Bjola, C. and Pamment, J. (eds) Countering Online Propaganda and Extremism: The dark side of digital diplomacy, 2019, pp. 156-171

[4] Kundnani, A. and Hayes, B. “The globalization of Countering Violent Extremism policies: Undermining human rights, instrumentalizing civil society.” 2018, https://www.tni.org/files/publication-downloads/the_globalisation_of_countering_violent_extremism_policies.pdf

[5] For instance, the European Radicalizaiton Awareness Network has a working group on “communications and Narratives” and publishes advise for counter-narrative practitioners in EU member states, e.g. Ritzmann (2019). Effective Narratives: Updating the GAMMMA+ Model. Retrieved from: https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/networks/radicalisation_awareness_network/about-ran/ran-c-and-n/docs/ran_cn_academy_creating_implementing_effective_campaigns_brussels_14-15112019_en.pdf

[6] Hedayah. “Developing Effective Counter-Narrative Frameworks for Countering Violent Extremism.” 2014, http://www.icct.nl/download/file/Developing%20Effective%20CN%20Frameworks_Hedayah_ICCT_Report_FINAL.pdf

[7] Radicalization Awareness Network (2015). Counter Narratives and Alternative Narratives. Retrieved from: https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/networks/radicalisation_awareness_network/ran-papers/docs/issue_paper_cn_oct2015_en.pdf

[8] Lindekilde, L. “Value for Money? Problems of Impact Assessment of Counter-Radicalisation Policies on End Target Groups: The Case of Denmark.” European Journal on Criminal Policy and Research, vol. 18, 2012, pp. 385-402

[9] Hemmingsen, A-S. and Castro Moller, K. “The Trouble with Counter-Narratives.” 2017, https://www.diis.dk/en/research/why-counter-narratives-are-not-the-best-responses-to-terrorist-propaganda

[10] Bélanger, J., Nisa, C., Schumpe, B., Gurmu, T., Williams, M. and Eka Putra, I. “Do counter-narratives reduce support for ISIS? Yes, but not for their target audience.” Frontiers in Psychology, vol. 11, 2020, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7325943/

[11] Ranstorp, M. “Root Causes of Violent Extremism.” 2016, https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/networks/radicalisation_awareness_network/ran-papers/docs/issue_paper_root-causes_jan2016_en.pdf

[12] Carthy, S., Doody, C., O‘Hora, D. and Sarma, K. “PROTOCOL: Counter‐narratives for the prevention of violent radicalisation: A systematic review of targeted interventions.” 2018, https://onlinelibrary.wiley.com/doi/full/10.1002/CL2.202

[13] Feddes, A. and Galucci, M. “A Literature Review on Methodology used in Evaluating Effects of Preventive and De-radicalisation Interventions.” Journal for Deradicalization, vol. 5, 2015, pp. 1-27

[14] Guhl, J., Ebner, J. and Rau, J. “The Online Ecosystem of the German Far-Right.” 2020, https://www.bosch-stiftung.de/sites/default/files/publications/pdf/2020-02/ISD-The%20Online%20Ecosystem%20of%20the%20German%20Far-Right.pdf

[15] Green, M. “No comment! Why more news sites are dumping their comment sections,” KQED. 24 Jan. 2018, https://www.kqed.org/lowdown/29720/no-comment-why-a-growing-number-of-news-sites-are-dumping-their-comment-sections

[16] Suler, J. “The Online Disinhibition Effect.” Research Gate, 6 Jul. 2004, https://www.researchgate.net/publication/8451443_The_Online_Disinhibition_Effect

[17] Nagle, A. Kill All Normies: Online Culture Wars from 4chan and Tumblr to Trump and the Alt-Right. John Hunt Publishing, 2017.

[18] Stoud, N., Scacco, J., Muddiman, A. and Curry, A. “Changing deliberative norms on news organizations’ Facebook sites.” Journal of Computer-Mediated Communications. vol. 20, no. 2, 2014, pp. 108-203

Author: Linda Schlegel

Source: Link

RCCPCVE specialized website aims to publish a diversity of perspectives. The views expressed in this article represent the author alone.