The Industrialization of Terrorist Propaganda: Neural Language Models and the Threat of Fake Content Generation

Director Jason Blazakis, Deputy Director Kris McGuffie (MIIS ‘19) and online extremism researcher Alex Newhouse (MIIS ‘18) co-authored this report on the potential use online by terrorists and extremists of new neural language models.

Terrorists and extremists rely on the spread of information and ideologies to achieve their goals. Extreme online communities undertake coordinated harassment and radicalization campaigns using various linguistic and social contagion methods. In both cases, information is a weapon wielded to gather more followers, solidify reputations, and influence the narratives in the media. Extremists and terrorists are already acquainted with automated tools for message dissemination, including all of the tools used in mainstream public relations. As a result of their organizational and ideological needs, extremists are at the vanguard of information warfare. When new information technology arises, the most radical communities are among the earliest adopters.

Sophisticated language models will likely become one of the main tools in extremists’ information arsenals. In this paper, Blazakis, McGuffie and Newhouse explore whether a cutting-edge language model may be abused by terrorist- and extremist-affiliated information operations to not only disrupt  online discourse, but also create and amplify false narratives of their own.

Click here to read The Industrialization of Terrorist Propaganda: Neural Language Models and the Threat of Fake Content Generation.