1. Introduction

The concept of algorithmic governance has emerged over the last decade, but takes up an idea that has been present for much longer: that digital technologies structure the social in particular ways. Engaging with the concept of algorithmic governance is complex, as many research fields are interested in the phenomenon, using different terms and having different foci. To inquire what constitutes algorithmic governance makes an important contribution to contemporary social theory by interrogating the role of algorithms and their ordering effect. We define algorithms as computer-based epistemic procedures which are particularly complex – although what is complex depends on the context. Algorithms shape procedures with their inherent mathematical logics and statistical practices. With that, the discourse around algorithmic governance often overlaps and intersects with debates about datafication (cf. Mejias & Couldry, 2019 as part of this special section) and artificial intelligence (AI). Yet, algorithms sometimes also operate on ‘small data’ and use calculus-based procedures that do not learn and that are not adaptive.

While governance is a contested term, we define its core as coordination between actors based on rules. Other than regulation, governance is not necessarily intentional and goal-directed (Black, 2001), it also includes unintentional coordination (Hofmann, Katzenbach, & Gollatz, 2016). Yet, governance excludes all forms of social ordering which are purely occasional and don’t rely on some sort of rule; governance implies a minimum degree of stability which is necessary for actors to develop expectations, which are a precondition for coordination (Hobbes, 1909). We choose the term algorithmic governance instead of algorithmic regulation because governance allows to account for the multiplicity of social ordering with regard to actors, mechanisms, structures, degrees of institutionalisation and distribution of authority. It deliberately embraces social ordering that is analytically and structurally decentralised and not state-centred. Thus, algorithmic governance better reflects the ambition of this article to widely scrutinise the ways in which algorithms create social order. In that sense, we focus on governance by algorithms instead of the governance of algorithms (Musiani, 2013; Just & Latzer, 2017). In sum, algorithmic governance is a form of social ordering that relies on coordination between actors, is based on rules and incorporates particularly complex computer-based epistemic procedures.

The relevance of dealing with algorithmic governance becomes evident with regard to competing narratives of what changes in governance when it makes use of algorithms: one narrative is for example that governance becomes more powerful, intrusive and pervasive. A different narrative stresses that governance becomes more inclusive, responsive, and allows for more social diversity, as we will highlight in the following chapters.

If considered broadly, the roots of this concept can be traced back to the history and sociology of science, technology and society. Technology has always both reflected and reorganised the social (Bijker & Law, 1992; Latour, 2005). From Socrates’ concerns with writing and literacy (Ong, 1982) via cybernetic’s radically interdisciplinary connection between technical, biological and social systems and their control (Wiener, 1948) via Jacques Ellul’s bureaucratic dystopia of a 'technological society' (1964) to Langdon Winner’s widely cited, yet contested 'politics of artefacts' (1980) – the idea that technology and artefacts somehow govern society and social interactions is a recurring theme. The more direct predecessor of algorithmic governance is Lawrence Lessig’s famous catchphrase 'code is law'. Here, software code or, more generally, technical architectures are seen as one of four factors regulating social behaviour (next to law, market and social norms). Scholars have also conceptualised the institutional character of software and algorithms (Katzenbach, 2017, 2012; Napoli, 2013; Orwat et al., 2010). While Rouvroy and Berns used the term 'gouvernance algorithmique' in 2009, the first to conceptualise the term 'algorithmic governance' were Müller-Birn, Dobusch and Herbsleb (2013), presenting it as a coordination mechanism opposed to 'social governance'. 1 The concept of ‘algorithmic regulation' was introduced by US publisher Tim O’Reilly (2013), highlighting the efficiency of automatically governed spaces – but overlooking the depoliticisation of highly contested issues that comes with delegating them to technological solutions (Morozov, 2014). In contrast to the implicit technological determinism of these accounts, the interdisciplinary field of critical software studies has complicated – in the best sense – the intricate mutual dependencies of software and algorithms on the one hand, and social interactions and structures on the other (MacKenzie, 2006; Fuller, 2008; Berry, 2011; Kitchin & Dodge, 2011). This article sets out to provide a primer on the concept of algorithmic governance, including an overview on dominant perspectives and areas of interest (section 2), a presentation of recurrent controversies in this space (section 3), an analytical delineation of different types of algorithmic governance (section 4), and a short discussion of predictive policing and automated content moderation as illustrative case studies (section 5). We seek to steer clear of the deterministic impetus of the trajectory towards ever more automation, while taking seriously the turn to increasingly manage social spaces and interaction with algorithmic systems.

2. Algorithmic governance: perspectives and objects of inquiry

The notion of algorithmic governance is addressed and discussed in different contexts and disciplines. They share similar understandings about the importance of algorithms for social ordering, but choose different objects of inquiry. The choice of relevant sets of literature presented here has a focus on research in science and technology studies (STS), sociology, political science, communication and media studies, but includes research from relevant other disciplines interested in algorithmic governance, such as computer science, legal studies, economics, and philosophy.

Various closely related and overlapping research areas are interested in how algorithms contribute to re-organising and shifting social interactions and structures. In contrast to public debate, however, these scholars reject the notion of algorithms as independent, external forces that single-handedly rule our world. They complicate this techno-determinist picture by asserting the high relevance of algorithms (Gillespie, 2014), yet highlighting the economic, cultural, and political contexts that both shape the design of algorithms as well as accommodate their operation. Thus, empirical studies in this field typically focus on the social interactions under study and interrogate the role of algorithms and their ordering effect in these specific contexts (Kitchin, 2016; Seaver, 2017; Ziewitz, 2016). They share an interest in how data sets, mathematical models and calculative procedures pave the way for a new quality of social quantification and classification. The notions of 'algorithmic regulation' (Yeung, 2018) and ‘algorithmic governance’ (Just & Latzer, 2016; König, 2019) emanate from the field of regulation and governance research, mostly composed of scholars from legal science, political science, economy, and sociology. The relevant studies have had the effect of organising and stimulating research about algorithmic governance with a shared understanding of regulation - as “intentional attempts to manage risk or alter behavior in order to achieve some pre-specified goal” (Yeung, 2018). This focus on goal-directed, intentional interventions sets the stage for inquiries that are explicitly interested in algorithms as a form of government purposefully employed to regulate social contexts and alter the behaviour of individuals, for example in the treatment of citizens or the management of workers. Other approaches also study non-intentional forms of social ordering through and with algorithms.

A slightly different approach puts the technical systems in the centre, not the social structures and relations. Relevant studies, particularly in computer science aim to build and optimise algorithmic systems to solve specific social problems: detect contested content, deviant behaviour, and preferences or opinions – in short: they are building the very instruments that are often employed in algorithmic governance. The common goal in this approach usually is to effectively detect patterns in data, e.g., translating social context into computable processes (i.e., optimising detection). This research stream is seeking efficient, robust, fair and accountable ways to classify subjects and objects both into general categories (such as species) as well as into specific dimensions such as psychometric types, emotional states, credit worthiness, or political preferences (Schmidt and Wiegand, 2017; Binns et al., 2017). Producers and providers of algorithmically-fuelled services not only optimise the detection of patterns in existing data sets, but they often – in turn – also aim to optimise their systems to most effectively nudge user behaviour in a way that seeks to maximise organisational benefits (optimising behaviour). By systematically testing different versions of user screens or other features (A/B-testing) and applying user and behavioural analytics, companies continually work to direct user interactions more effectively towards more engagement and less friction (Guerses et al., 2018). It is, however, important to note that there is no clear line between the research that develops and optimises algorithmic governance and the research analysing its societal implications; they overlap and there are many studies that strive towards both aims. An example in case are studies about algorithmic bias, fairness and accountability that both conceptualise and test metrics (e.g., Waseem & Hovy, 2016). Another important area of research that is applied and critical are studies about ‘automation bias’, ‘machine bias’ or ‘over-reliance’ that study under which conditions human agents can take a truly autonomous decision (Lee & See, 2004; Parasuraman & Manzey, 2010).

One important domain of inquiry especially relevant to STS, communication and media studies is digital communication and social media. Scholars have been interested for more than a decade in how search engines and social media platforms organise and structure information that is available online and how this affects subjectivation (Couldry & Langer, 2005). Platforms prioritise certain types of content (typically based on metrics of 'engagement') – thus constituting a new dominant mode to ascribe relevance in society, complementing traditional journalistic routines. Platforms also deploy algorithms to regulate content by blocking or filtering speech, videos and photos that are deemed inacceptable or unlawful (Gillespie, 2018; Gorwa, 2019). With increasing scale and growing political pressure, platforms readily turn to technical solutions to address difficult platform governance puzzles such as hate speech, misinformation and copyright (Gorwa, Binns, & Katzenbach, 2019). Other areas under study that make use of automated content detection are plagiarism checks in teaching and academic writing (Introna, 2016) and sentiment analysis for commercial and political marketing (Tactical Tech, 2019).

Public sector service provisions, citizen management and surveillance constitute another key area of interest for algorithmic governance scholars. Especially political scientists and legal scholars investigate automated procedures for state service delivery and administrative decision-making. The ambition here is that algorithms potentially increase the efficiency and efficacy of state services, for example by rationalising bureaucratic decision-making, by targeting information and interventions to precise profiles or by choosing the best available policy options (OECD, 2015). Yet, their promises are heavily contested. Scholars have shown that the deployment of algorithmic systems in the public sector produced many non-intended and non-disclosed consequences (Veale & Brass, 2019; Dencik, Hintz, Redden, & Warne, 2018). Applying algorithmic tools in government often relies on new forms of population surveillance and classification by state and corporate actors (Neyland & Möllers, 2017; Lupton, 2016, Bennett, 2017). The grounds for many projects of digital service provision and algorithm-based policy choice are systems of rating, scoring and predicting citizen behaviour, preference and opinion. These are used for the allocation of social benefits, to combat tax evasion and fraud, to inform jurisdiction, policing and terrorism prevention, border control, and migration management.

Rating and scoring are not only applied to citizens, but also to consumers, as valuation and quantification studies have pointed out with regard to credit scores (Avery, Brevoort, & Canner, 2012; Brevoort, Grimm, & Kambara, 2015; Fourcade & Healy, 2016, 2017). These studies point out how algorithm-based valuation practices shape markets and create stratification mechanisms that can superimpose social class and reconfigure power relations – often to the detriment of the poor and ‘underscored’ (Fourcade & Healy, 2017; Zarsky, 2014).

Governance through algorithms is also an important matter of concern for scholars studying the digital transformation of work, such as the sociology of labour and labour economics. The objects of study here are automated governance on labour platforms and management of labour within companies, for example through performance management and rating systems (Lee, Poltrock, Barkhuus, Borges, & Kellogg, 2017; Rosenblat, 2018). This research field is characterised by empirical case studies that enquire the implications of algorithmic management and workplace surveillance for workers’ income, autonomy, well-being, rights and social security; and for social inequality and welfare states (Wood, Graham, Lehdonvirta, & Hjorth, 2019). Related objects of inquiry are algorithmic systems of augmented reality, of speech recognition and assistance systems for task execution, training and quality control (Gerber & Krzywdzinski, 2019). Important economic sectors under study are logistics, industrial production, delivery and services. Other relevant areas of research focus on the algorithmic management of transportation and traffic, energy, waste and water, for example in ‘smart city’ projects.

Some scholars approach algorithmic governance on a meta-level as a form of decentralised coordination and participation. They stress its power to process a high number of inputs and thus to tackle a high degree of complexity. As a consequence, they see algorithmic governance as a mode of coordination that offers new opportunities for participation, social inclusiveness, diversity and democratic responsiveness (König, 2019; Schrape, 2019). There is abundant research about the possibilities that software can offer to improve political participation through online participation tools (Boulianne, 2015; Boulianne & Theocharis, 2018), such as electronic elections and petitions, social media communication and legislative crowdsourcing. In addition, countless algorithmic tools are being developed with the explicit aim to ‘hear more voices’ and to improve the relationship between users and platforms or citizens and political elites. However, algorithmic governance through participatory tools often remains hierarchical, with unequal power distribution (Kelty, 2017).

3. Controversies and concerns

Across these different perspectives and sectors, there are recurring controversies and concerns that are regularly raised whenever the phenomenon of algorithmic governance is discussed. Looking at these controversies more closely, we can often detect a dialectic movement between positive and negative connotations.

Datafication and surveillance

The literature about algorithmic governance shows an ample consensus that big data, algorithms and artificial intelligence change societies’ perspectives on populations and individuals. This is due to the ‘data deluge’, an increase and variety in data collected by digital devices, online trackers and the surveillance of spaces (Beer, 2019). ‘Datafication’ (cf. Mejias & Couldry, 2019 as part of this special section) also benefits from increasingly powerful infrastructures which enable more and faster data analysis, and societal norms that benefit quantification, classification and surveillance (Rieder & Simon, 2016). Research about algorithmic governance has nevertheless always been concerned with the many risks of datafication and surveillance. To surveil entire populations and to create detailed profiles about individuals on the basis of their ‘data doubles’ creates ample opportunities for social sorting, discrimination, state oppression and the manipulation of consumers and citizens (Lyon, 2014; Gandy, 2010). Unfettered surveillance poses danger to many civil and human rights, such as freedom of speech, freedom of assembly, and privacy, to name just a few.

Agency and autonomy

The ubiquity of algorithms as governance tools has created concerns about the effects on human agency and autonomy (Hildebrandt, 2016) – a central concept of the Enlightenment and a key characteristic of the modern individual. While earlier approaches conceived of algorithms as either augmenting or reducing human agency, it has become clear that the interaction between human and machine agents is complex and needs more differentiation. While typologies and debates typically construct a binary distinction between humans-in-the-loop vs. humans-out-of-the-loop, this dichotomy does not hold for in-depth analyses of the manifold realities of human-computer-interaction (Gray & Suri, 2018). In addition, human agency cannot only be assessed with regard to machines, but also with regard to constraints posed by organisations and social norms (Caplan & boyd, 2018).

Transparency and opacity

The assumed opacity of algorithms and algorithmic governance is a strong and lasting theme in the debate, routinely coupled with a call for more transparency (Kitchin, 2016; Pasquale, 2015). However, more recent arguments point out that access to computer code should not become a fetish: absolute transparency is often not possible nor desirable and not the solution to most of the problems related to algorithmic governance, such as fairness, manipulation, civility, etc. (Ananny & Crawford, 2017; Mittelstadt, Allo, Taddeo, Wachter, & Floridi, 2016). In addition, the implementation of social norms into code not only creates opacity, but also unveils norms and processes that were previously hidden. A case in point are controversies around scoring systems about unemployment risk, as deployed in Austria and Poland (AlgorithmWatch and Bertelsmann Stiftung, 2019), and credit worthiness (AlgorithmWatch, 2019). The public interest in algorithmic governance has motivated civil society actors and scholars to inquire the composition and rationality of algorithmic scoring and to question the underlying social values. Given this development, the current turn to algorithmic governance might indeed even be conducive to more transparency as software code, once disclosed, requires the articulation of underlying assumptions into explicit models.

De-politicisation and re-politicisation

In a similar logic, there is a vivid public debate about the de-politicising and re-politicising effects of algorithms. Algorithms have often been criticised as de-politicising due to their ‘aura of objectivity and truth’ (boyd & Crawford, 2012) and their promise to solve problems of social complexity by the sheer size of data and increased computing power (Kitchin, 2013; Morozov, 2013). However, and as a consequence, many studies have disputed the idea that algorithms can be objective and neutral. Social inequality, unfairness and discrimination translate into biased data sets and data-related practices. This new public suspicion about the societal implications of algorithms has motivated critics to similarly question the rationalities of political campaigning, social inequality in public service delivery, and the implications of corporate surveillance on civil rights. In that way, algorithmic governance has contributed to a re-politicisation of governance and decision-making in some areas. Yet, this might be a short-lived gain since the installment of algorithmic governance as societal infrastructures will most certainly lead to their deep integration into our routines over time, eventually being taken for granted like almost all infrastructures once they are in place (Plantin et al., 2018; Gorwa, Binns, & Katzenbach, 2019)

Bias and fairness

Another key concern is that of algorithmic bias. Automated decision-making by algorithmic systems routinely favours people and collectives that are already privileged while discriminating against marginalised people (Noble, 2018). While this truly constitutes a major concern to tackle in the increasing automatisation of the social, this is not a new phenomenon – and the algorithm is not (the only one) to blame. Biased data sets and decision rules also create discrimination. This rather foregrounds the general observation that any technological and bureaucratic procedure materialises classifications like gender, social class, geographic space, race. These do not originate in these systems, they merely reflect prevalent biases and prejudices, inequalities and power structures – and once in operation they routinely amplify the inscribed inequalities. The current politicisation of these issues can be considered an opportunity to think about how to bring more fairness into societies with automated systems in place (Barocas & Selbst, 2016; boyd & Barocas, 2017; Hacker, 2018).

4. From out-of-control to autonomy-friendly: evaluating types of algorithmic governance

While algorithmic systems expand into various social sectors, additional research fields will develop, merge and create sub-fields. At the same time, controversies will shift and shape future developments. This makes it hard or even impossible to synthesise the diversity of perspectives on algorithmic governance and its numerous areas of interest into one systematic typology. In any case, typologies are always contingent on the priorities and motives of the authors and their perception of the phenomenon. Yet, there is growing demand from policymakers around the world and the broader public to evaluate deployments of algorithmic governance systems and guide future development. For good reasons: algorithmic governance like other sociotechnical systems is contingent on social, political, and economic forces and can take different shapes.

For these reasons, we present a typification that addresses the design and functionality of algorithmic systems and evaluates these against key normative criteria. Notwithstanding the dynamic character of the field, we choose the degree of automation and transparency as they stand out with regard to their normative implications for accountability and democracy, and thus will most likely remain key elements in future evaluations of different types of algorithmic governance. 2

Transparency matters as it constitutes a cornerstone of democracy and self-determination (Passig, 2017), yet it is particularly challenged in the face of the inherent complexity of algorithmic systems. Therefore, transparency is not only one of the major academic issues when it comes to algorithmic regulation, but also an important general matter of public controversy (Hansen & Flyverbom, 2015). Only (a certain degree of) transparency opens up decision-making systems and their inscribed social norms to scrutiny, deliberation and change. Transparency is therefore an important element of democratic legitimacy. It is, however, important to note that the assessment of a given case of algorithmic governance will differ between software developers, the public and supervisory bodies. As already mentioned (cf. section 3), algorithmic governance systems push informational boundaries in comparison to previous governance constellations: they demand formalisation, thus social norms and organisational interests need to be explicated and translated into code – thus potentially increasing the share of socially observable information. Yet, in practice, algorithmic governance often comes with an actual decrease in socially intelligible and accessible information due to cognitive boundaries (intelligibility of machine learning) and systemic barriers (non-access to algorithms due to trade secrecy, security concerns and privacy protection) (Ananny & Crawford, 2017; Pasquale, 2015; Wachter, Mittelstadt, & Floridi, 2017).

The degree of automation matters greatly because the legitimacy of governance regimes relies on the responsibility and accountability of a human decision-maker in her role as a professional (a judge, a doctor, a journalist) and ethical subject. To focus on the degree of automation marks also the choice to problematise the complex interaction within socio-technical systems: algorithmic systems can leave more or less autonomy to human decision-makers. Here, we reduce the gradual scale of involvement to the binary distinction between fully automated systems where decisions are not checked by a human operator, and recommender systems where human operators execute or approve the decisions ('human-in-the-loop') (Christin, 2017; Kroes & Verbeek, 2014; Yeung, 2018).

The combination of both dimensions yields four, in the Weberian sense, ideal-types of algorithmic governance systems with different characteristics: 'autonomy-friendly systems' provide high transparency and leave decisions to humans; 'trust-based systems' operate with low transparency and human-decision-makers; 'licensed systems' combine high transparency with automated execution; and finally 'out-of-control systems' demonstrate low transparency and execute decisions in a fully-automated way.

5. Algorithmic governance in operation: predictive policing and automated content moderation

The four ideal-types can be found in the full range of sectors and domains that employ algorithmic governance systems today (for recent overviews see AlgorithmWatch and Bertelsmann Stiftung, 2019; Arora, 2019; Dencik, Hintz, Redden, & Warne, 2018; Tactical Tech, 2019). In order to illustrate algorithmic governance both in the public and private sector, and in platforms, we shortly present two prominent and contested cases: automated risk assessment for policing (‘predictive policing’) is among the most disseminated forms of public algorithmic governance in industrial countries; and automated content moderation on social media platforms belongs to the various ways in which private platforms use algorithmic governance on a global scale. The cases show that algorithmic governance is not one thing, but it takes different forms in different jurisdictions and contexts, and it is shaped by interests, power, and resistance. Algorithmic governance is multiple, contingent and contested.

Predictive policing

Police authorities employ algorithmic governance by combining and analysing various data sources in order to assess crime risk and prevent crime (i.e., burglary, car theft, violent assault, etc.). This risk analysis addresses either individuals or geographic areas; some systems focus on perpetrators, others on potential victims. The results are predictions of risk that are mobilised to guide policing. Algorithmic governance can be directed towards the behaviour of citizens or of police officers. Typical actions are to assign increased police presence to geographic areas, to surveil potential perpetrators or to warn potential victims.

The degrees of transparency need to be assessed from two perspectives: with regard to the public and with regard to the organisation that uses predictive policing. In many cases, data collection, data analysis and governance measures lie in the responsibility of both police agencies and private companies, often in complex constellations (Egbert, 2019). Some projects rely on strictly crime-related data, other projects make use of additional data, such as data about weather, traffic, networks, consumption and online behaviour. In most cases, the software and its basic rationalities are not public. The same is true for the results of the analysis and their interpretation. 3 There is no predictive policing system that makes data and code available to the public, thus most applications in that space are trust-based systems. In some cases, such as in the German state of North Rhine-Westphalia, the software has been developed by the police. It is not public, but an autonomy-friendly system from the police’s perspective. This relatively high degree of opacity is justified by the police with the argument that transparency would allow criminals to ‘game the system’ and render algorithmic governance ineffective. Opacity, however, hinders evaluations of the social effects of algorithmic governance in policing. Major public concerns are whether predictive policing reinforces illegitimate forms of discrimination, threatens social values and whether it is effective and efficient (Ulbricht, 2018).

With regard to the degree of automation, it is noteworthy that in most cases of algorithmic governance for policing the software is still designed as a recommender system: human operators receive a computer generated information or recommendation. It is their responsibility to make the final decision of whether to act and how. However, police officers have complained about the lack of discretion in deciding of where to patrol (Ratcliffe, Taylor, & Fisher, 2019). Another concern is that police officers might not have the capacity to take an autonomous decision and to overrule the algorithmically generated recommendation (Brayne, 2017), effectively turning predictive policing into out-of-control or licensed systems of algorithmic governance. The massive number of research and pilot projects in this space indicate that in the near future, the degree of automation in predictive policing and border control governance will increase considerably.

Automated content moderation on social media platforms

Another highly relevant and contested field of algorithmic governance in operation is the (partly) automated moderation and regulation of content on social media platforms. Two developments are driving the turn to AI and algorithms in this field (Gollatz, Beer, & Katzenbach, 2018): (a) The amount of communication and content circulating on these platforms is so massive that it is hard to imagine that human moderators could cope manually with all posts and other material, screening them for compliance with public law and platform rules. As platforms thrive to find solutions that scale with their global outreach, they have strong economic interests to find technical solutions. This is (b) reinforced by the growing political pressure on platforms to tackle issues of hate speech, misinformation and copyright violation on their sites – with regulation partly moving towards immediate platform liability for illegal content (Helberger, Pierson, & Poell, 2019). Thus, platforms develop, test and increasingly put into operation automated systems that aim to identify hate speech, match uploaded content with copyrighted works and tag disinformation campaigns (Gillespie 2018; Duarte, Llanso, & Loup, 2018).

With regard to transparency, platforms such as Facebook, YouTube and Twitter have long remained highly secretive about this process, the decision criteria, and the specific technologies and data in use. The increasing politicisation of content moderation, though, has pressured the companies to increase transparency in this space – with limited gains. Today, Facebook, for example, discloses the design of the general moderation process as well as the underlying decision criteria, but remains secretive about specifics of the process and detailed data on removals. 4 YouTube’s system for blocking or monetising copyrighted content called ContentID provides a publicly accessible database of registered works. The high-level criteria for blocking content are communicated, yet critics argue that the system massively over-blocks legitimate content and that YouTube remains too secretive and unresponsive about the appeals process, including the exact criteria for delineating legitimate and illegitimate usage of copyrighted content (Erickson & Kretschmer, 2019; Klonick, 2018). The Global Internet Forum to Counter Terrorism (GIFCT), a joint effort by Facebook, Google, Twitter and Microsoft to combat the spread of terrorist content online, hosts a shared, but secretive database of known terrorist images, video, audio, and text.

With regard to automation, most systems in content moderation do not operate fully automated but most often flag contested content for human review – despite industry claims around efficiency of AI systems. For example, Facebook has technical hate speech classifiers in operation that evaluate apparently every uploaded post and flag items considered illegitimate for further human review. 5 In contrast, ContentID is generally operating fully automated, meaning that decisions are executed without routine human intervention: uploads that match registered content are either blocked, monetised by the rightsholder or tolerated according to the assumed right-holders’ provisions. In the case of GIFCT, early press releases emphasised that “matching content will not be automatically removed” (Facebook Newsroom, 2016). However, the response of platforms to major incidents like the shooting in Christchurch, New Zealand, and to propaganda of major terrorist organisations such as ISIS and Al-Qaeda now seems to indicate that certain matches of the GIFCT are executed and thus blocked automatically, without human moderators in the loop (out-of-control-systems) (Gorwa, Binns, & Katzenbach, 2019).

As these examples show, the binary classification of transparency and automation of a given system is not always easily drawn. Yet, until recently, most of these implementations of algorithmic governance could rightfully be considered out-of-control-systems. The recent political and discursive pressure has certainly pushed the companies towards more transparency, although in our evaluation this still does not qualify them as autonomy-friendly- or licensed”-systems as they still lack meaningful transparency.

6. Conclusion

The concept of algorithmic governance encapsulates a wide range of sociotechnical practices that order and regulate the social in specific ways ranging from predictive policing to the management of labour and content moderation. It is one benefit of the concept that it brings together these diverse sets of phenomena, discourses, and research fields, and thus contributes to the identification of key controversies and challenges of the emerging digital society. Bias and fairness, transparency and human agency are important issues that are to be addressed whenever algorithmic systems are deeply integrated into organisational processes, irrespective of the sector or specific application. Algorithmic governance has many faces: it is seen as ordering, regulation and behaviour modification, as a form of management, of optimisation and of participation. Depending on the research area it is characterised by inscrutability, the inscription of values and interests, by efficiency and effectiveness, by power asymmetry, by social inclusiveness, new exclusions, competition, responsiveness, participation, co-creation and overload. For most observers, governance becomes more powerful, intrusive and pervasive with algorithimisation and datafication. A different narrative stresses that governance becomes more inclusive, responsive, and allows for more social diversity.

And indeed, algorithmic governance is multiple. It does not follow a purely functional, teleological path thriving for ever-more optimisation. It is rather contingent on its social, economic and political context. The illustrative case studies on predictive policing and content moderation show that algorithmic governance can take very different forms, and it changes constantly – sometimes optimised for business interests, sometimes pressured by regulation and public controversies. The proposed ideal-types of algorithmic governance for means of evaluation constitute one way of assessing these systems against normative standards. We chose transparency and the degree of automation as key criteria, resulting in a spectrum of implementation ranging from out-of-control-systems to autonomy-friendly-systems – other criteria for evaluation could be the types of input data or of decision models. In any case, these structured and integrated ways of thinking about algorithmic governance might help us in the future to assess on more solid grounds which forms of algorithmic governance are legitimate and appropriate for which purpose and under which conditions – and where we might not want any form of algorithmic governance at all.

Introduction

Globally operating platform businesses, from Facebook to Uber, and from Amazon to Coursera, are becoming increasingly central to public and private life, transforming key economic sectors and spheres of life, including journalism, transportation, entertainment, education, finance, and health care. This transformation can be understood as a process of ‘platformisation’, which this article sets out to contextualise, define, and operationalise.

To contextualise the concept, we start with the notion of ‘platform’ from which ‘platformisation’ has been derived. In the first section we discuss the history of the platform concept which has seen different uses among business and scholarly communities. We highlight these differences not only to offer conceptual clarity, but also to move towards scholarly consensus. Subsequently, reflecting on initial efforts to specify the contours of platformisation, we argue that it is productive to develop a broad perspective to understand the socio-cultural and political economic consequences of this process. To that end, the second section defines platformisation by combining insights from four distinct research perspectives that each map onto different scholarly traditions: 1) software studies, 2) business studies, 3) critical political economy, and 4) cultural studies. The final section of the article demonstrates how platformisation can be operationalised in research. Building on the four scholarly traditions, we argue that the institutional dimensions of platformisation—data infrastructures, markets, and governance—need to be studied in correspondence with shifting cultural practices.

Developing this argument, it is important to keep in mind that platformisation deeply affects societies around the globe, but in the current moment it is primarily a process driven by US-based platform companies. There are regional and national exceptions, the most prominent being China, where a range of domestic platforms emerged—Baidu, Alibaba, and Tencent—marked by strong state support and oversight (De Kloet et al., 2019). Considering how US-based companies steer platformisation, we cannot do justice to the many global variations, which would require a much longer analysis. While this process everywhere involves changes in infrastructures, markets, and governance, there are crucial differences in how these changes take shape in particular countries and regions.

1. The platform concept: different streams of literature

To set the context, we start with the notion of ‘platform’ from which ‘platformisation’ has been derived. The usage of the platform concept, both in academia and in business, has seen a number of key shifts since the start of the new millennium. Predating the arrival of contemporary tech behemoths, such as Google and Facebook, the fields of (network) economics and business studies already popularised and theorised the term platform, most prominently in Japan, France, and the United States (Steinberg, 2019). In the early 2000s, US companies such as Microsoft, Intel, and Cisco provided management scholars with rich examples of how to attain “platform leadership” (Gawer & Cusumano, 2002). One of the most influential contributions to this scholarship conceptualised platforms (e.g., game consoles) as “two-sided markets” (Rochet & Tirole, 2003). Platform operators aggregate, on the one side buyers or end-users (e.g., players) and on the other side sellers or complementors, such as game publishers. Later contributions incorporated work from neighbouring fields including industrial organisation economics, strategic management, and information technology. This body of work has had significant impact on business discourse and strategies deployed by platform companies, much more so than critical media perspectives.

In media and communication studies, the emergence of the platform concept evolved alongside conversations about broader shifts in communication technology, the information economy, and the subsequent reorientation of users as active producers of culture (Benkler, 2006; Jenkins, 2006). Around 2005, the concept of “Web 2.0” entered the popular lexicon to serve as a shorthand for these shifts, signalling that the internet as a whole had become a platform for users and businesses to build on (O’Reilly, 2005). The Web 2.0 concept is best seen as a discursive exercise speaking to a business audience first and foremost, rather than an attempt to historicise any technological, economic, and socio-cultural shift in particular (Van Dijck & Nieborg, 2009). In hindsight, the concept was effective in paving the way for the further erosion of the open web or “generative Internet” towards an “appliancized network” of proprietary social network sites (Zittrain, 2008, p. 12). Services such as YouTube, Facebook, MySpace, and Twitter were increasingly hailed as social network platforms, constituting “a convergence of different systems, protocols, and networks” (Langlois et al., 2009).

Closely connected with the Web 2.0 discourse, early mentions of the ‘platform’ concept share a distinctive economic purpose; they served as a metaphor or imaginary, employed by business journalists and internet companies to draw end-users to platforms and simultaneously obfuscate their business models and technological infrastructures (Couldry, 2015; Gillespie, 2010). As Gillespie (2017) writes “Figuratively, a platform is flat, open, sturdy. In its connotations, a platform offers the opportunity to act, connect, or speak in ways that are powerful and effective [...] a platform lifts that person above everything else”. In this regard, the term platform should be seen as “productive” in its own right, prompting users to organise their activities around proprietary, for-profit platforms.

Parallel to the business discourses, a distinct computational perspective on platforms emerged in the late 2000s. In 2009, Montfort and Bogost launched a book series titled ‘platform studies’ with each volume dissecting a particular computational platform (e.g., the Atari VCS or the French Minitel). Collectively, these titles are attentive to the material dimension (hardware) of platforms and the software frameworks that support the development of third-party programmes, particularly games. A broader field of software studies research has been developed in parallel by scholars who foregrounded platforms as (re-)programmable software systems that revolve around the systematic collection and processing of user data (Helmond, 2015; Langlois & Elmer, 2013; Plantin et al., 2018). Research in this field is influenced by work that typically lies at the edge of traditional humanities programmes, such as computer and organisational science, information systems, and critical code studies.

While business studies and software studies research generated different understandings of platforms, these perspectives effectively complement each other: business interests and efforts to develop two-sided markets inform the development of platform infrastructures. Vice versa, platform architectures are modular in design so its technology can be selectively opened up to complementors to build and integrate their services to be used by end-users. To gain insight in platforms as both markets and computational infrastructures, it is vital to combine these approaches. Thus, we define platforms as (re-)programmable digital infrastructures that facilitate and shape personalised interactions among end-users and complementors, organised through the systematic collection, algorithmic processing, monetisation, and circulation of data. Our definition offers a nod to software studies by pointing to the programmable and data-driven nature of platform infrastructures, while acknowledging the insights of business studies perspective by including the main stakeholders or “sides” in platform markets: end-users and complementors.

2. (Re-)Defining platformisation

The next step is to explain how the scholarly community moved from a discussion of 'platforms' as ‘things’ to an analysis of 'platformisation' as a process. We identify a variety of scholarly traditions that have studied this process from different angles. Although the academic disciplines we introduce below are not always consistent, nor explicit in their terminology, we can nevertheless infer a particular understanding of platformisation from their research trajectories. To develop platformisation as a critical conceptual tool, it is important to explore and combine different approaches and understandings.

The first approach we would like to focus on is software studies, which has most explicitly foregrounded and defined platformisation. Starting from the computational dimension of platforms, this strand of research is especially focussed on the infrastructural boundaries of platforms, their histories and evolution. Helmond’s (2015) work has been foundational in this respect as she defines platformisation as the “penetration of platform extensions into the web, and the process in which third parties make their data platform-ready”. Key objects of study include Application Programming Interfaces (APIs), which allow for data flows with third parties (i.e., complementors), and Software Development Kits (SDKs), which enable third parties to integrate their software with platform infrastructures (Bodle, 2011; Helmond, Nieborg, & van der Vlist, 2019). Together, these computational infrastructures and informational resources afford institutional relationships that are at the root of a platform’s evolution and growth as platforms “provide a technological framework for others to build on” (Helmond, 2015).

The infrastructural dimension of platforms has been further explored from a software studies perspective by Plantin and his colleagues (2018), who observe a simultaneous “platformisation of infrastructures” and a “infrastructuralization of platforms”. They contend that digital technologies have made “possible lower cost, more dynamic, and more competitive alternatives to governmental or quasi-governmental monopoly infrastructures, in exchange for a transfer of wealth and responsibility to private enterprises” (Plantin et al., 2018, p. 306). In this transfer, major platform companies have emerged as the “modern-day equivalents of the railroad, telephone, and electric utility monopolies of the late 19th and the 20th centuries” (Plantin et al., 2018, p. 307). From this infrastructural perspective case studies have been developed, for example on Facebook’s history and evolution (Nieborg & Helmond, 2019). Here, the social media platform is understood as a “data infrastructure” that hosts a variety and constantly evolving set of “platform instances”, for example apps such as Facebook Messenger and Instagram. Each app then contributes to the platform’s expanding boundaries as it forges both computational and economic connections with complementors, such as content developers, businesses, content creators, and advertisers.

While software studies highlights the infrastructural dimension of platform evolution, business studies foregrounds the economic aspects of platformisation. The latter approach takes platform businesses as its key unit of analysis and theorises how platforms can gain a competitive advantage by operating multi-sided markets (McIntyre & Srinivasan, 2017). For platform companies, one of the advantages inherent to platform markets that can be leveraged are network “externalities” or effects(Rohlfs, 1974; Rochet & Tirole, 2003). These effects manifest themselves either directly, when end-users or complementors join one side of the market, or indirectly, when the other side of the market grows. As McIntryre and Srinivasan (2017, p. 143) explain, “direct network effects arise when the benefit of network participation to a user depends on the number of other network users with whom they can interact”. And indirect network effects occur when “different ‘sides’ of a network can mutually benefit from the size and characteristics of the other side”.

The managerial and economic blueprint for multi-sided markets theorised by business scholars invariably leads to the accumulation of capital and power among a small group of platform corporations (Haucap & Heimeshoff, 2014; Srnicek, 2016). As a counterweight to these business studies accounts, it is important to turn to a third approach: critical political economy. While most scholars in this tradition do not explicitly use the notion of platformisation, their work is vital as it signals how this process involves the extension and intensification of global platform power and governance. Critical political economists have drawn attention to issues of labour exploitation, surveillance, and imperialism (Fuchs, 2017). For example, Scholz (2016, p. 9) considers the issue of platform labour, maintaining that “wage theft is a feature, not a bug” of platforms. Considering the global footprint of platform companies, Jin (2013, p. 167) introduced the notion of “platform imperialism”, arguing that the rapid growth of companies such as Facebook and Google demonstrates that “American imperialism has been continued” through the exploitation of platforms.

Important to note is that the discussed research traditions all primarily conceive of platforms and platformisation in institutional terms, as data infrastructures, markets, and forms of governance. Notably absent is an analysis of how platforms transform cultural practices, and vice versa, how evolving practices transform platforms as particular socio-technical constructs. These transformations have been extensively studied by scholars working in the broader tradition of cultural studies, who mostly do not employ the notion of platformisation either, but whose work is important for understanding this process. As the cultural studies literature on platforms is very extensive—ranging from self-representation and sexual expression, to the transformation of labour relations and visual culture (Burgess, Marwick, & Poell, 2017), we cannot do justice to its full scope. We do want to stress the importance of considering platform-based user practices when analysing platformisation. A major challenge in such examinations is to trace how institutional changes and shifting cultural practices mutually articulate each other.

One body of work that is at the forefront of theorising the shifting relationships among users and platforms concerns work on labour. By introducing concepts such as “aspirational labor”, “relational labor”, and “hope labor”, cultural studies researchers have critically examined how specific practices and understandings of labour emerged within platform markets (Baym, 2015; Duffy, 2016; Kuehn & Corrigan, 2013). As Duffy (2016, p. 453) points out, newly emerging occupations, such as streamers, vloggers and bloggers, tend to reify “gendered social hierarchies”, that “leave women’s work unrecognized and/or under-compensated”. Considering platformisation from this perspective means analysing how social practices and imaginations are organised around platforms. This, in turn, shapes how platforms evolve as particular data infrastructures, markets, and governance frameworks.

Although these four approaches provide us with different foci and interpretations of platformisation, we would like to argue that they are not mutually exclusive (Nieborg & Poell, 2018). The observed institutional changes and shifts in cultural practices associated with platforms are in practice closely interrelated. Thus, a more fundamental and critical insight in what platformisation entails can only be achieved by studying these changes and shifts in relation to each other. Following research in software studies, business studies, and political economy, we therefore understand platformisation as the penetration of the infrastructures, economic processes, and governmental frameworks of platforms in different economic sectors and spheres of life. And in the tradition of cultural studies, we conceive of this process as the reorganisation of cultural practices and imaginations around platforms. The next section will discuss what platformisation entails in practice and how this rather abstract definition can be operationalised in research.

3. Operationalising platformisation: studying the impact of platforms

The different perspectives on platformisation, which we derived from the various research traditions, suggest that this process unfolds along three institutional dimensions: data infrastructures, markets, and governance. And we observed that, from a cultural studies perspective, platformisation leads to the (re-)organization of cultural practices around platforms, while these practices simultaneously shape a platform’s institutional dimensions. Ultimately, the collective activities of both end-users and complementors, and the response of platform operators to these activities, determine a platform’s continued growth or its demise. As pointed out by critical political economists, the power relations among platform operators, end-users, and complementors are extremely volatile and inherently asymmetrical as operators are fully in charge of a platform’s techno-economic development. Moreover, network effects, together with platform strategies that frustrate attempts by end-users or complementors to leave a platform, have resulted in highly concentrated platform markets (Barwise & Watkins, 2018). While the media and telecom industries have been dominated by internationally operating conglomerates for decades (Winseck, 2008), the rapid emergence of a handful of platform companies challenges the power of industry incumbents. Poignant examples of digital dominance by platform companies can be witnessed in the new markets for digital advertising, apps, e-commerce, and cloud computing. With these considerations in mind, we propose to study the three institutional dimensions of platformisation as interactive processes that involve a wide variety of actors, but which are also structured by fundamentally unequal power relations. We will use the example of app stores to illustrate how the three dimensions can be operationalised.

The first dimension is the development of data infrastructures, which has especially been explored by software studies scholars. As a process, the development of data infrastructures has been captured through the notion of datafication, referring to the ways in which digital platforms render into data, practices and processes that historically eluded quantification (Kitchin, 2014; Mayer-Schönberger & Cukier, 2013; Van Dijck, 2014; and Mejias & Couldry, 2019 on datafication, as part of this special section). This process does not just concern demographic or profiling data volunteered by users or solicited via (online) surveys, but especially also behavioural meta-data. Such behavioural data collection is afforded by still expanding platform infrastructures in the form of apps, plugins, active and passive sensors, and trackers (Gerlitz & Helmond, 2013; Nieborg & Helmond, 2019). As such, platform infrastructures are integrated with a growing number of devices, from smartphones and smartwatches to household appliances and self-driving cars. This myriad of platform extensions allows platform operators to transform virtually every instance of human interaction into data: rating, paying, searching, watching, talking, friending, dating, driving, walking, etc. This data is then algorithmically processed and, sometimes under strict conditions, haphazardly made available to a wide variety of external actors (Bucher, 2018; Langlois & Elmer, 2013). Important to note: this datafication process is simultaneously driven by complementors, who actively integrate platform data in products and services that are used in everyday practices and routines. Many news organisations and journalists, for example, use social media data in editorial decision-making and in content distribution strategies (Van Dijck, Poell, & De Waal, 2018). It is through such emerging cultural practices that data infrastructures become important in particular economic sectors and activities.

One example of a ubiquitous data infrastructure for software distribution are the app stores operated by Apple and Google. Instead of downloading software applications from distributed locations, as is common in desktop-based software environments, app stores are highly centralised, heavily controlled and curated software marketplaces. In the case of Apple’s iOS mobile operating system for the iPhone, iPad and Apple Watch, the App Store is the only sanctioned way for users to access third-party software, allowing Apple to track and control which apps are distributed by whom and thus, indirectly, also which data are collected, by whom, and for what purpose. This strict control over app distribution allows Apple to set technical standards and define data genres, categories, and subsequent actions. For instance, Apple’s HealthKit data framework provides “a central repository for health and fitness data” on iOS devices. Of course, this repository and its related data standards only become influential because many app developers (i.e., complementors) use this functionality and thereby subject themselves to Apple’s interpretation and standardisation of what counts as “health” data.

The second dimension of platformisation concerns markets, the reorganisation of economic relations around two-sided or multi-sided markets, which has especially been studied and theorised by business scholars. Traditional pre-digital market relations, with some notable exceptions, tend to be one-sided, with a company directly transacting with buyers. Conversely, platforms constitute two-sided, or increasingly, complex multi-sided markets that function as aggregators of transactions among end-users and a wide variety of third parties. A classic example of a two-sided market similar to the App Store is a dedicated game console, such as the PlayStation, which connects game publishers with players (Rochet & Tirole, 2003). A game platform that also lets advertisers target users, becomes a multi-sided market, connecting gamers, game publishers, and advertisers. Market arrangements like these affect the distribution of economic power and wealth, as they are subject to strong network effects. A game platform that attracts a lot of game publishers and game titles becomes more attractive for users, and vice versa, more users make a platform more attractive for game publishers and advertisers, with the latter generating additional income that can be used to subsidise content.

Thus, changes in market relations are not simply ‘institutional’, but for an important part driven by the practices of end-users, content producers, and other “sides” in the market, such as advertisers and data intermediaries. If many end-users suddenly embrace a new platform, as happened in the case of the smartphone, content producers and advertisers are likely to follow quickly. Yet, once end-users and complementors have been aggregated and integrated at scale, it becomes increasingly hard for other platforms to break into a particular market, or, for content and service providers to ignore platform monopolies. Whereas, for example, newspapers were for a long time successful non-digital two-sided markets attracting readers and advertisers (Argentesi & Filistrucchi, 2007), they are increasingly turned into platform complementors offering content to end-users through platforms, such as Facebook, Twitter, and Instagram, who then “monetise” this content by surrounding it with advertisements (Nieborg & Poell, 2018).

App stores also serve as examples of two-sided platform markets, connecting end-users with app developers. This market arrangement affects the distribution of power, as all app-based commercial transactions are subject to the economic imperatives set out by the Apple/Google duopoly. As app-related income is not the primary revenue generator for either platform company, both app stores have rigid pricing standards and a relatively low barrier to market entry for developers. Consequently, app supply is high, counted in the millions. New entrants in the app economy, therefore, have become highly dependent on advertising and on selective promotion by platform operators to gain visibility in what has become a hyper competitive market. This market dynamic is reinforced by the collective practices of end-users, who rather than downloading new apps on a weekly basis, tend to stick to using around 40 apps at any time (Comscore, 2017). Important to note is that this rearrangement of market relations is intrinsically connected with the previous dimension of datafication. Because of fierce competition, app developers are incentivised to systematically collect end-user data to track and optimise user engagement, retention, and monetisation (Nieborg, 2017).

Third, platforms not only steer economic transactions, but also platform-based user interactions. This leads us to the dimension of governance, which has especially been put on the research agenda by political economic and software studies scholars (Gillespie, 2018; Gorwa, 2019). Most visibly, platforms structure how end-users can interact with each other and with complementors through graphical user interfaces (GUIs), offering particular affordances while withholding others, for example in the form of buttons—like, follow, rate, order, pay—and related metrics (Bucher & Helmond, 2018). This form of platform governance materialises through algorithmic sorting, privileging particular data signals over others, thereby shaping what types of content and services become prominently visible and what remains largely out of sight (Bucher, 2018; Pasquale, 2015). Equally important, platforms control how complementors can track and target end-users through application programming interfaces (APIs), software development kits (SDKs), and data services (Langlois & Elmer, 2013; Nieborg & Poell, 2018). Finally, platforms govern through contracts and policies, in the form of terms of service (ToS), license agreements, and developer guidelines, all of which have to be agreed with when accessing or using a platform’s services (Van Dijck, 2013). On the basis of these terms and guidelines, platforms moderate what end-users and complementors can share and how they interact with each other (Gillespie, 2018).

As platforms tend to employ these different governing instruments—interfaces, algorithms, policies—without much regard for particular political-cultural traditions, there are often clashes with local rules, norms, and regulatory frameworks. At the same time, it should be observed that all these governing instruments have been developed and constantly adjusted in response to the evolving practices of end-users and complementors. The widespread circulation of disinformation and hate speech by end-users prompts platform operators to devise stricter policies and moderation practices, as well as algorithmic systems that can filter out this content. And, when large numbers of advertisers and content producers leave a platform, its operators will adjust the governing instruments to try to keep these complementors on board.

In our app store example, platform operators constantly tinker with their governing instruments to keep end-users and complementors tied to the platform. Google’s Play Store frequently changes its algorithmic sorting mechanisms, privileging particular data signals over others to arrive at a commercially optimal ranking of apps. While external actors affect the development of governance instruments, they lack insight in how platforms design and adjust these instruments. For developers and end-users, the Play Store is a typical black box, as apps rankings are based on opaque and largely invisible algorithms. Whereas such instances of algorithmic obfuscation received a lot of public and scholarly attention, we want to emphasise that these are elements of larger governance frameworks, which need to be scrutinised in their entirety. In the case of app stores, it is the combination of controlled access to data, algorithmic sorting, and often opaque moderation practices—especially Apple has a history of unexpected app-removals (Hestres, 2013)—that constitute its governance framework.

Conclusion

Taken together, the analysis of these three dimensions of platformisation enables a comprehensive understanding of how this process brings about a transformation of key societal sectors and how it presents particular challenges for stakeholders in these sectors. It is vital that we move beyond the particular foci of software studies, business studies, political economy, and cultural studies that have, so far, dominated the study of platforms and platformisation. We need to gain insight in how changes in infrastructures, market relations, and governance frameworks are intertwined, and how they take shape in relation with shifting cultural practices. Such an exploration is not just of academic interest. Platformisation can only be regulated democratically and effectively by public institutions if we understand the key mechanisms at work in this process.

Evidently, this short paper only provides the outline of such a research programme. Further developing this analytical framework, it is especially important to enhance our understanding of how the institutional changes are entangled with shifting cultural practices. Recent scholarship tends to focus on one or the other, which prohibits insight in the ever-evolving dynamics of platformisation. A systematic inquiry into the connections between the institutional and cultural dimensions of platformisation is particularly crucial because it will bring into view the correspondences and tensions between, on the one hand, global platform infrastructures, market arrangements, and governing frameworks, and, on the other hand, local and national practices and institutions. As political-cultural rules and norms widely diverge across the globe, the challenge is to integrate platforms in society without undermining vital traditions of citizenship and without increasing disparities in the distribution of wealth and power.

Introduction

In its contemporary meaning, the term ‘filter bubble’ was introduced and popularised by the US tech entrepreneur and activist Eli Pariser, most significantly in his 2011 book The Filter Bubble: What the Internet Is Hiding from You. Pariser opens the book with an anecdote:

in the spring of 2010, while the remains of the Deepwater Horizon oil rig were spewing crude oil into the Gulf of Mexico, I asked two friends to search for the term “BP.” They’re pretty similar – educated white left-leaning women who live in the Northeast. But the results they saw were quite different. One of my friends saw investment information about BP. The other saw news. For one, the first page of results contained links about the oil spill; for the other, there was nothing about it except for a promotional ad from BP. (Pariser, 2011a, p. 2)

Pariser goes on to speculate that such differences are due to the algorithmic personalisation of search results that Google and similar search engines promise, and that in effect each search engine user exists in a filter bubble – a “personalized universe of information” (Pariser, 2015, n.p.) – that differs from individual to individual. This idea can therefore be seen as a far more critical counterpoint to Nicholas Negroponte’s much earlier vision of the Daily Me (Negroponte, 1995), a personalised online newspaper that would cater to its reader’s specific interests rather than merely providing general-interest news and information. Such differences in the evaluation of otherwise similar concepts also demonstrate the considerably changed public perception of online services and their algorithmic shaping, from the early-Web enthusiasm of the mid-1990s to the growing technology scepticism of the 2010s.

It is important to note from the outset that, in writing for a general audience and promoting his concept through TED talks and similar venues (e.g., Pariser, 2011b), Pariser largely fails to provide a clear definition for the ‘filter bubble’ concept; it remains vague and founded in anecdotes. Subsequently, this has generated significant problems for scholarly research that has sought to empirically verify the widespread existence of filter bubbles in real-life contexts, beyond anecdotal observations. This definitional blur at the heart of the concept did not prevent it from gaining considerable currency in scientific as well as mainstream societal discourse, however: in his farewell speech, even outgoing US President Barack Obama warned that “for too many of us it’s become safer to retreat into our own bubbles” (Obama, 2017, n.p.) rather than engage with divergent perspectives. Politicians, journalists, activists, and other societal groups are now frequently accused of ‘living in a filter bubble’ that prevents them from seeing the concerns of others.

However, in such recent discussions the term ‘filter bubble’ is no longer primarily applied to search results, as it was in Pariser’s original conceptualisation; today, filter bubbles are more frequently envisaged as disruptions to information flows in online and especially social media. Pariser himself has made this transition from search engines to social media in his more recent writing, while continuing to point especially to the role of algorithms in creating such filter bubbles: he suggests, for instance, that “the Facebook news feed algorithm in particular will tend to amplify news that your political compadres favor" (Pariser, 2015, n.p.). This shift in the assumed locus of filter bubble mechanisms also points to the growing importance of social media as the primary sources for news and other information, of course – a change that longitudinal studies such as the Digital News Report (e.g., Newman et al., 2016, 10) have documented clearly.

In this new, social media-focussed incarnation, the ‘filter bubble’ concept has become more and more interwoven with the related but earlier concept of ‘echo chambers’, unfortunately. Indeed, a substantial number of mainstream media discussions, but also many scholarly articles, now use the two terms essentially interchangeably, in formulations like “filter bubbles (aka ‘echo chambers’)” (Orellana-Rodriguez & Keane, 2018). A clear distinction between the two terms is complicated by the fact that – like Pariser for ‘filter bubble’ – the originator of the ‘echo chamber’ concept, the legal scholar Cass Sunstein, never clearly defines the latter term either. As a result, just like ‘filter bubble’, the term ‘echo chamber’ has been applied to various online contexts, ranging from ‘the Internet’ in the abstract to specific social media spaces, since it first appeared in Sunstein’s 2001 book on the concept. This terminological confusion – about the exact definitions of either term in itself, and about their interrelationship with each other – has significantly hindered our ability to test them through rigorous research.

(Inter-)disciplinary perspectives on filter bubbles

Indeed, the most fundamental problem that emerges from the profound lack of robust definitions for these terms is the fact that empirical studies exploring the existence and impact of filter bubbles (and echo chambers) in any context have generally been forced to introduce their own definitions, which reduces their comparability: a study that claims to have found clear evidence for filter bubbles might have utilised a very different definition from another study that found the precise opposite. Attempts have been made in recent years to develop more systematic and empirically verifiable definitions for either term (e.g., O’Hara & Stevens, 2015; Zuiderveen Borgesius et al., 2016; Dubois & Blank, 2018), and to re-evaluate extant studies against such definitions (e.g., Bruns, 2019), but there is a need for considerably more progress in this effort.

Today, much of the scholarly focus in the investigation of these concepts is on social media; in part, this is because in spite of Pariser’s founding anecdote there is a severe lack of convincing evidence for the existence of significant filter bubbles in search results. In 2018 alone, three major studies showed substantial overlaps in the search results seen by users of different political and other interests: Nechushtai and Lewis reported that Google News searchers in the US were directed overwhelmingly to “longstanding national newspapers and television networks” (2018, p. 302); Haim et al. found “only minor effects of personalization on content diversity” for similar news searchers in Germany (2018, p. 339); and Krafft et al. build on their findings for both Google Search and Google News to explicitly “deny the algorithmically based development and solidification of isolating filter bubbles” (Krafft et al., 2018, p. 53; my translation). Notably, these results stem from studies that were conducted in different countries, at very different scales, and utilised a variety of methods. While further research should confirm these results for a greater number of national contexts and a broader range of search engines and news portals, at least for search, it seems that the filter bubble idea has deflated: far from the vision (or threat) of an individually unique Daily Me, personalisation in general and news-specific search results still appears to be exceptionally limited.

For social media, on the other hand, the debate about filter bubbles and echo chambers continues, with various studies both supporting and denying their existence. Here, the definitional confusion is most acutely felt, and also manifests very differently across the different disciplines that are involved in testing these concepts. For the purpose of the following discussion, and in order to introduce a meaningful distinction between the ‘echo chamber’ and ‘filter bubble’ concepts, we might employ the following minimal definitions (cf. Bruns, 2019, p. 29):

• echo chamber: emerges when a group of participants choose to preferentially connect with each other, to the exclusion of outsiders (e.g., by friending on Facebook, following on Twitter, etc.)
• filter bubble: emerges when a group of participants choose to preferentially communicate with each other, to the exclusion of outsiders (e.g., by comments on Facebook, @mentions on Twitter, etc.)

The effects of such connective and communicative structures would be further heightened if such echo chambers and filter bubbles overlapped each other closely: that is, when users who only follow each other also choose only to communicate with each other. This is not necessarily guaranteed: Twitter users may @mention others whom they do not follow, for instance, and Facebook users may encounter others with whom they are not friends in the comment pages of Facebook pages and groups.

Notably, especially in comparison with earlier concerns about the fragmentation of national mediaspheres as a result of the multiplication of cable channels or online information sources, or about social stratification due to diverging media literacies amongst different parts of the population, the echo chamber and filter bubble concepts clearly centre on the individual media user and their treatment by search engines and social media platforms. While such earlier concerns tackled aggregate, population-wide trends, therefore, these new phenomena are driven by individualised personalisation processes (and possible user interventions in such personalisation).

Network science

As these definitions already foreshadow, one key approach to the study of filter bubbles is the (usually computational) analysis and visualisation of the network structures between social media participants. At the risk of oversimplification, such network science approaches mainly tend to look for evidence of homophily: a preference for interconnections between participants with similar interests, views, and ideologies. This, in turn, may also lead to selective exposure, as members of such homophilous communities are expected to preferentially circulate content that matches their worldviews (e.g., Batorski & Grzywińska, 2018).

The definitions suggested here also enable an assessment of the degree of preferential attachment to like-minded others: put simply, they enable an evaluation of the balance between in-group and out-group connections and communication. Here, in light of the considerable disconnective and disruptive effects that their proponents ascribe to filter bubbles and echo chambers, it is clear that such an evaluation would need to show considerably more than only a mild tendency towards homophily in users’ connection and communication choices. In order to result in a notable divergence in the information diets experienced by members and non-members of an echo chamber or filter bubble, they would actively have to both seek out engagement with like-minded others (in other words, pursue selective exposure), and stay away from those who might introduce them to alternative views (that is, practice selective avoidance).

On most social media platforms, basic homophilous tendencies are indeed very easy to find: in the context of the current political climate in the United States, for instance, it is unsurprising that Twitter hashtags such as #MAGA or #resist attract highly homophilous and diametrically opposed participant groups, for instance (Chong, 2018); similarly, it is to be expected that Facebook pages and groups with an explicit anti-vaccination agenda will mainly attract participants that share this agenda (Smith & Graham, 2017). From this perspective, social media platforms and their respective affordances (such as Twitter hashtags, or Facebook pages and groups) can be understood as engines of homophily– and indeed, since dial-in bulletin board systems and the early Usenet first became popular, it has been this very opportunity to connect with like-minded others that has attracted users to computer-mediated communication platforms (e.g., Baym, 2000).

Notably, there are a great many contexts where such homophily can be understood as beneficial: for instance as it allows communities of interest to connect online in spite of considerable geographical distance, as it enables groups of participants with special interests to share relevant information with each other, or as it enables the members of vulnerable minorities in society to provide mutual support to one another (cf. Helberger, 2019). Is homophily alone a sufficient criterion for communicative dysfunction, then – should we now reclassify any such communities of interest in online environments as filter bubbles? After all, while politically hyperpartisan hashtags or conspiracy theory pages may attract a fairly homogenous community of participants, these particular online spaces do not exist in isolation, but are embedded into a much more complex and varied social media platform, which in turn forms only one component of a rich and diverse media ecology (cf. Dubois & Blank, 2018). As a result, network science studies that look beyond such inherently ideological communities find significantly less homophily and polarisation in non-political contexts, and also detect considerable cross-connection between the groups that populate those ideological spaces.

For instance, a network analysis commissioned by newspaper Süddeutsche Zeitung before the 2017 German federal election found that, for all their political differences, the followers of the major parties’ Facebook pages still shared many non-political interests, and would encounter each other on the pages relating to those interests (ranging from news through entertainment to sports) – only the Facebook followers of the extreme-right AfD party diverged significantly from this pattern by concentrating predominantly on anti-migrant topics. The report concluded that “apparently closed filter bubbles do not exist in large parts of the political spectrum in Germany. … Users from different party-political milieux often encounter the same posts” (Rietzschel, 2017, n.p.; my translation) – and while the AfD’s departure from this societal mainstream is concerning in its own right, it shows that (if at all) filter bubbles exist only at the very extremes of the ideological spectrum.

For Twitter, comprehensive maps of follower connections in the Australian (Bruns et al., 2017) and Norwegian Twitterspheres (Bruns & Enli, 2018) have similarly shown the existence of interest-driven clusters of dense interconnection around topics from politics to sports, but point to few significant disconnects across the overall network. Where clusters have deliberately detached from the wider network, this is due to their significant topical divergence (porn) or enhanced need to maintain a strictly professional network (education); otherwise, the network structures of both Twitterspheres facilitate a largely unencumbered flow of information across the entire user base and would therefore be unable to support the existence of filter bubbles.

Social science

Such studies point to the fact that the multifaceted, multi-interest nature of mainstream social media platforms actively militates against the formation of echo chambers and filter bubbles. Many users are not participating in Facebook, Twitter, and other leading platforms only because of a narrow political agenda or interest, but use these platforms to pursue a multiplicity of divergent and sometimes contradictory interests, connecting in the process with various groups and communities that may overlap to a greater or lesser degree. Such processes of accidental or deliberate overlap are now well established under the term ‘context collapse’ (Marwick & boyd, 2011) – and as much as we might see Facebook groups and Twitter hashtags in themselves as engines of homophily, we must also regard these platforms in their entirety as engines of context collapse.

Such context collapse is especially prone to occur at the point where a user’s different communities of interest, and the networks of contacts that they represent, are most likely to intersect: in the personal public (Schmidt, 2014) that surrounds the user’s social media profile. In part because ethical and practical considerations about access to data on private and semi-private interactions at the profile level prevent large-scale network science studies, such tendencies towards context collapse have been observed predominantly through the use of social science methods from ethnographic observation through interviews to surveys – and this research often presents substantial challenges to the filter bubble and echo chamber concepts.

A representative Pew Center survey of US social media users before the 2016 presidential election reported, for example, that only “23% of Facebook users and 17% of Twitter users say [that] most of the people in their networks hold political beliefs similar to theirs” (Duggan & Smith, 2016, p. 9). The same survey also noted that more than one quarter of respondents had “blocked or unfriended” contacts because of unwanted political content (2016, p. 4). Such attempts to disconnect might be seen as efforts to build a personal filter bubble by this minority of users; that they are necessary, however, and that overall users are “worn out by the amount of political content they encounter” (2016, p. 2), clearly shows that to date any manual or algorithmic attempts to reduce the heterogeneity of personal networks on Facebook and Twitter have failed.

In passing, this undermines Pariser’s suggestion that Facebook algorithm “will tend to amplify news that your political compadres favor" (2015, n.p.), as we had encountered it previously: if our Facebook networks are inherently heterogeneous – for political as well as for other interests – then how would the algorithm be able to detect which of these connections are our ‘compadres’, and privilege their ideologies? Yes, such a selection might be possible for users who are on Facebook purely for politics, but it is patently obvious that the vast majority of Facebook participants merely endure rather than actively enjoy political discussions (Duggan & Smith, 2016). Indeed, arguably it is a fundamental flaw of both the ‘filter bubble’ and ‘echo chamber’ concepts that they are championed by authors like Pariser and Sunstein who genuinely are deeply engaged in political debates, but who fail to recognise that their experience of online and social media therefore diverges considerably from social media users who are not ‘political junkies’ (Coleman, 2003).

For such ordinary, apolitical social media users the encounter with political news and debate is therefore substantially more likely to be unplanned and serendipitous, through “casual political talk” in non-political contexts (Wojcieszak & Mutz, 2009, p. 50) and the occasional sharing of news items by others in their personal networks. Because they are so unplanned, however, such serendipitous encounters are unlikely to drive filter bubble tendencies: indeed, research using the survey data gathered for the Digital News Report has shown that “those who are incidentally exposed to news on social media use more different sources of online news than non-users” (Fletcher & Nielsen, 2018, p. 2459). This means that, contrary to concerns about the fragmentation of society as a result of filter bubbles, for many users social media have positively increased the diversity of their information diet, and prevent them from becoming locked into ideological monocultures.

Notably, a long-term study of political polarisation in the United States suggests that “the groups least likely to use the Internet experienced larger changes in polarization between 1996 and 2016 than the groups most likely to use the Internet” (Boxell et al., 2017, p. 10612). This would mean that, pace Sunstein and Pariser and in contrast to current moral panics about the impact of social media on political discourse, online and social media have the potential to actively mitigate polarisation tendencies. It should be acknowledged here, however, that such large-scale, longitudinal, survey-based observations may be valid at an aggregate level, but show considerable variation for individual communities and individuals: clearly, as examples like the anti-vaccination activists or AfD supporters show, for some groups it does remain possible to use online and social media to seek out strong homophily and engage preferentially in the development of distinct and divergent ideological positions.

Media and communication studies

Disciplines within the general field of media and communication studies are a particularly rich source of explanations for such divergent tendencies at the individual and group level. Researchers here are perhaps especially likely to consider the sometimes contradictory results of individual studies in the broader context of the overall media ecology, both taking into account the multiplicity of communicative practices and spaces within specific social media platforms and especially also recognising the importance of interaction and information flows across multiple social media platforms and other media channels within the contemporary hybrid media environment (Chadwick et al., 2016).

From this disciplinary perspective, it seems entirely possible that sufficiently motivated (that is, hyperpartisan and polarised) participants on a given platform may engage in communal spaces that, from a network science approach that traces their horizontal linkages, appear as highly homophilous and detached from other communication spaces on the platform – but that these same participants will nonetheless remain embedded in the larger media environment observed by social science research, even if such vertical linkages across platforms and channels remain invisible to computational data capture. In other words, the localised homophily that is likely to exist in specific contexts and spaces does not fundamentally undermine the general heterogeneity of the hybrid media ecology, and cannot usually prevent its participants from encountering – willingly or unwillingly – a broad range of information and perspectives.

To fully detach from this diversity would require considerably more drastic steps: as O'Hara and Stevens describe it, “a networked individual would have to enter the echo chamber and somehow lose his or her diverse connections, replacing them with more and stronger connections within the echo chamber”, in a way similar to the processes by which “people are adopted into cults, brainwashed, and alienated from their contacts, but … this is not a common scenario” (2015, p. 416). Instead, in fact, many of the hyperpartisans with the strongest adherence to extremist views – from anti-vaccination and anti-climate science activists to the right-wing extremists supporting Donald Trump or the AfD – are also highly engaged with the mainstream media, at least in order to monitor what their enemies are thinking: readers of extremist white supremacy sites are significantly more likely to visit the New York Times or similar quality news outlets than ordinary news users, for example (Gentzkow & Shapiro 2011, p. 1823). In other words, their ideology may diverge in extreme ways from the societal mainstream, but they do not exist in a filter bubble by any definition.

Social and political relevance and impact

As early as 2004, David Weinberger remarked that “the problem with an extraterrestrial-conspiracy mailing list isn’t that it’s an echo chamber; it’s that it thinks there’s a conspiracy by extraterrestrials” (2004, n.p.). Concepts like ‘echo chamber’ and ‘filter bubble’ – which, notably, were introduced not by scholars in media, communication, internet, or related fields of study, but by authors working well outside their area of expertise (Pariser is an activist and tech entrepreneur, Sunstein is a legal scholar) – have served to obscure considerably more pressing societal problems, and to misdirect scholarly, journalistic, and regulatory attention to the technological rather than social and societal factors underpinning these problems. While the empirical evidence does not support the existence of echo chambers and filter bubbles as actual, observable phenomena in public communication, therefore, the persistent use of these concepts in mainstream media and political debates around the world has created its own discursive reality that continues to impact materially on societal institutions, media and communication platforms, and ordinary users themselves. As scholars, we cannot therefore simply close the case on filter bubbles and echo chambers and move on to more realistic concerns, but are forced to continue to push back against the simplistic models of connection and communication that these concepts continue to represent.

Even before the advent of contemporary social media platforms like Facebook (launched 2004) or Twitter (launched 2006), Weinberger saw early glimpses of these developments. He referred to the ‘echo chamber’ concept when he noted that the “meme is not only ill-formed, but it also plays into the hands of those who are ready to misconstrue the Net in order to control it” (2004, n.p.), yet this applies just as much to the subsequent ‘filter bubble’ idea: much of the contemporary public debate about echo chambers and filter bubbles has straightforwardly assumed that these phenomena exist in reality and have a significant deleterious effect on society; that they are caused by the new communication technologies of search engines and now especially also of social media; that a particular root cause of the problem lies in the personalisation and recommendation algorithms deployed by these platforms; and that this technological problem must therefore also have a technological solution (Meineck 2018, n.p.). This technologically determinist approach to echo chambers and filter bubbles is despite the fact that – as we have seen in this article, as well as in a series of more detailed critical reviews of empirical research on these phenomena (e.g., Bruns, 2019; Dubois & Blank, 2018; Zuiderveen Borgesius et al., 2016) – there is a pronounced absence of evidence that genuine filter bubbles or echo chambers are real, outside of highly specific and unusual contexts.

This disconnect between the public understanding of and the scientific evidence on these concepts has all the hallmarks of a moral panic, similar to those that have accompanied the transition to almost any other major new communication technology introduced in human history. These moral panics point to the persistence of a simplistic and naïve understanding of media effects both amongst the general public and amongst media and political actors. In this simple view, we are defenceless especially as new and emerging media technologies ‘do things to us’ (change our attention spans, make us more angry, enclose us in filter bubbles); by contrast, the predominant conclusion of media effects research over the last decades has been that new media adoption is always a negotiated process of social construction during which these media are adapted and changed at least as much as media users adjust their own practices. That this message is still not getting through to the general public shows the seductive nature of moral panic narratives, but must also count as a failing for media and communication research.

Further, such moral panics often also serve as part of the rear-guard defence of the old elites that stand to lose the most from any change to the status quo, so it is worth asking cui bono: who benefits from the ‘filter bubble’ meme? Mainstream media have operationalised it to suggest that only their orderly and professional gatekeeping procedures, and not the collective and self-organising gatewatching processes on social media, have the ability to sufficiently inform citizens, for example; establishment politicians have used it to assert that only their well-designed party structures, and not the populist and/or grassroots models of their opponents, can provide strong and stable leadership for their countries. In either case, the claim that audiences and voters diverging from these well-trodden paths do so because they are caught in technologically created filter bubbles and can no longer be reached by reasoned, sensible argument also has the considerable added benefit that the journalists and politicians making that claim are never required to confront their own failings, and to examine other possible reasons for their own declining popularity. It is interesting to note in this context that the relative prevalence of the filter bubble idea in different countries’ political debates may also indicate the depth of broader concerns about their political and democratic processes: it is no accident that the filter bubble and echo chamber concepts originate from the US.

Such moral panics distract us from more important matters; as Sebastian Meineck has put it, it is only “when the tale of the filter bubble bursts [that] the debate about the transformation of the public sphere can get started” (2018, n.p.; my translation). This debate will need to examine whether societies around the world, from Australia to Brazil, from Germany to the United States, are becoming increasingly polarised, or whether such polarisation is simply becoming more visible; that this polarisation is being exploited by a new breed of political actors that employ radical grassroots approaches, offer highly populist solutions centred on strongman leaders, or combine both approaches; and that these transformations severely disrupt and sometimes paralyse existing political systems and undermine fundamental societal consensus. But it will also need to recognise that such transformations are not fuelled simply by surface factors such as the communication technologies and platforms preferred by these new political actors and their established opponents, respectively – rather, they are an expression of far more fundamental social, economic, societal, and political challenges. This does not mean that search and social media platforms are free of fault, of course – indeed, at present there is an acute need to compel them (through regulatory or other means) to do more to remove extremist accounts, prevent the circulation of disinformation, and open themselves to independent scholarly scrutiny. On the specific question of filter bubbles, however, they appear largely free of blame.

One of the few benefits of the ‘filter bubble’ concept – which Meineck, with some justification, describes as “the dumbest metaphor of the Internet” (2018, n.p.; my translation) – is that it has spawned a considerable wave of research that shows the diversity of most citizens’ media uses, and indeed points to the fact that online and social media users consume a particularly diverse news diet (Fletcher & Nielsen, 2018; Anspach, 2017). If societal and ideological polarisation persists and worsens in this environment, then this cannot be caused by filter bubbles or echo chambers, or by the algorithmic shaping of users’ information feeds that has been posited so often as the cause for such phenomena; rather, such polarisation persists in spite of the absence of filter bubbles, and perhaps even because of it: the thorough and direct interconnection across society and societies that online and social media have enabled has only made it easier to observe and express the differences between different social, economic, ethnic, religious, and ideological groups.

Indeed, recent studies have shown that partisan and hyperpartisan users often employ an inherently and staunchly oppositional reading stance as they engage with mainstream media content: they consult such media not to be informed, but to incorporate this new information into their existing picture of the ideological opponent. Worse still, direct attempts by mainstream sources to confront and correct the highly biased worldviews of the partisan fringes – for instance through fact-checking initiatives – only “confirm one’s status as a critical outsider” (Krämer, 2017, p. 1302; cf. Spohr, 2017, p. 151). To put it simply, when conspiracy theorists are told, by those whom they suspect of having orchestrated a conspiracy, that their conspiracy theories are unfounded, this only confirms the existence of the conspiracy. Fact checks may still be valuable to prevent mainstream users from drifting off to the fringes – but for those already on the fringe they only serve to deepen their disconnect from rational public debate.

Conclusion

If there is a filter at all, then, it is not the algorithmic filter postulated by the ‘filter bubble’ concept, which prevents us altogether from seeing ‘different’ content that runs counter to our own worldviews – rather, the more critical filter exists (more weakly formed perhaps in the societal mainstream, more strongly developed on the extreme fringes) in our heads, and variously leads us to adopt dominant, negotiated, and oppositional stances (cf. Hall, 1980) towards the information we encounter from a multitude of sources in our daily engagement with a hybrid, multifaceted, multi-platform media environment. The critical question then becomes why and how different groups in society come to develop such highly divergent personal readings of the same information, and how the ossification of these diverse ideological perspectives into partisan group identities can be prevented or undone – in order to mitigate the very real threat of fundamental societal polarisation, and even of a complete breakdown of the societal consensus. For societies where ideological boundaries align with clear economic, ethnic, or religious divisions, and where bipolar two-party systems prevent the emergence of centrist consensus alternatives to the polarised status quo, this challenge is especially acute.

Phenomena such as homophily (as well as heterophily) can be readily observed in contemporary communicative spaces, as can the algorithmic shaping and personalisation of newsfeeds and information streams (as well users’ efforts to control such shaping). New media and communication technologies have always undergone a process of individual adaptation and social construction; while not neutral, the technologies and their providers are neither inherently good nor evil in this, but can be employed by their users to serve socially and societally beneficial as well as disruptive ends. As scholars, one of our primary tasks is to understand what motivates these individual and collective choices. The ‘filter bubble’ and ‘echo chamber’ concepts, however, with their strong technologically determinist elements, have very little to contribute to the solution of such fundamental challenges; indeed, with evidence for their absence in observable reality continuing to mount, perhaps it is time to allow them to fade into obscurity. Yet while politicians, journalists, technologists, and other stakeholders continue to use these terms as if they describe actual real-life phenomena, and while there is a possibility that they might build on this crucial misunderstanding of the causes of current societal challenges in their development of political, regulatory, legislative, technological, educational, or social initiatives that seek to address them, it remains incumbent on scholars to confront these ill-conceived memes head-on. “The myth of the filter bubble”, above all, “is one thing: a big misunderstanding” (Meineck, 2018, n.p.; my translation). But while that misunderstanding continues to circulate in public debate, so must scholars push back against it: by pointing to the extant studies that debunk it, and by conducting further research that uncovers the actual dynamics of polarisation.

Acknowledgments/Funding

This research is supported by the Australian Research Council Future Fellowship project Understanding Intermedia Information Flows in the Australian Online Public Sphere, Discovery project Journalism beyond the Crisis: Emerging Forms, Practices and Uses, and LIEF project TrISMA: Tracking Infrastructure for Social Media in Australia.

Introduction

The term “datafication” implies that something is made into data. What that something is, and what the processing comprises, are matters that need to be put into context. The term “data”, however, is relatively clear, at least in its contemporary usage. Data is the “material produced by abstracting the world into categories, measures and other representational forms [...] that constitute the building blocks from which information and knowledge are created” (Kitchin, 2014, p. 1). While, in principle, any thing or process (from a sun or rain pattern, to a beating heart, to a lesson delivered in a class) can be made into data, our focus in this short essay will be on processes of datafication that create digital data out of human life. Since most writers on data also care about what happens to human life, the term “datafication” has quickly acquired an additional meaning: the wider transformation of human life so that its elements can be a continual source of data. The beneficiaries of this are very often corporations, but also states and sometimes civil society organisations and communities.

The term “datafication” was introduced in a 2013 review of “big data” processes across business and the social sciences (Mayer-Schönberger and Cukier, 2013, chapter 5): “to datafy a phenomenon is to put it in quantified form so that it can be tabulated and analyzed” (2013, p. 78). Datafication, the authors argued, involves much more than converting symbolic material into digital form, for it is datafication, not digitization, that “made [digital] text indexable and thus searchable” (2013, p. 84). Through this process, large domains of human life became susceptible to being processed via forms of analysis that could be automated on a large-scale. The dynamic that drives datafication as a social process then becomes apparent: the drive to “render [...] human behavior… into an analyzable form” in a process that in the review mentioned above was already called “the datafication of everything” (2013, p. 93-94).

It was not long before critical perspectives on datafication began to appear. As our initial definition of “data” makes clear, data do not naturally exist, but only emerge through a process of abstraction: something is taken from things and processes, something which was not already there in discrete form before. Lisa Gitelman (2013) sums up this point in the title of a well-known edited collection: Raw Data is an Oxymoron. Indeed, implicit in the very notion of data (or what is given as fact, from the Latin data) are the notions of selection and transformation: “data are [...] elements that can be abstracted from [...] phenomena” (Kitchin, 2014, p. 2). Kitchin even argues that “data” should be replaced with another Latin term, capta—what is captured—to refer to how, practically, data is harvested from life. José van Dijck, surveying various terms that emerged around data processes, also offers a critical interpretation of datafication as “a means to access [...] and monitor people’s behavior” (Van Dijck, 2014, p. 1478). She proposes that practices of datafication are becoming “an accepted new paradigm for understanding [...] social behavior” (2014, p. 1478, added emphasis). Such understanding involves a vision of “processes of datafication as a new way of interpreting the world”. Pushing the argument further, Shoshana Zuboff argues that what we are living through is a new stage of “surveillance capitalism” (2019) in which human experience becomes the raw material that produces the behavioural data used to influence and even predict our actions.

We can approach the study of digital data as a complex matrix of actors and structures, which different disciplines can help us analyze at multiple levels. In terms of actors, we have corporations, states, and various civic (activists, journalists, etc.) and even non-state (terrorists, hackers) actors, all of which can produce, collect and analyse data for different purposes. Here the focus can range from the big corporate players responsible for the bulk of datafication in our lives—Facebook, Apple, Microsoft, Google, and Amazon in the West, and their Chinese counterparts Baidu, Alibaba, Tencent and Xiaomi)—to smaller players across what can be called the “social quantification sector” (Couldry and Mejias, 2018), including hardware, software, platforms, data analytics, data brokerage firms, and even spammers (depending on which country we examine, this sector has more or less close relations with how government at various levels seeks to extract data for monitoring its citizens; China is one country where those relations are particularly close, cf. Chen and Qiu, 2019). Datafication can obviously benefit some of these actors, but it can also be used to discriminate against others on the basis of race, class, etc. (cf. Gandy, 1993; Peña Gangadharan, 2012). In terms of structures, data can flow within various architectures which can include platforms, services, apps, databases, and hardware devices. To make sense of this complexity, various research disciplines can help us zoom in or out on different intersections of players and infrastructures. For instance, software or platform studies can address issues of technological configuration and affordances, while a critical political economy approach can address issues of commodification and exploitation. Most of these approaches attempt to explain in some way how big data is “made” in terms of its relationship to time, context, and power (Boellstorff, 2013).

Next, we consider the specific elements that make up datafication, and the perspectives from which different disciplines have approached datafication’s consequences, with specific emphasis on datafication by corporations for economic profit.

Elements of datafication

The production of data cannot be separated from two essential elements: the external infrastructure via which it is collected, processed and stored, and the processes of value generation, which include monetisation but also means of state control, cultural production, civic empowerment, etc. This infrastructure and those processes are multi-layered and global, including mechanisms for dissemination, access, storage, analysis and surveillance that are owned or controlled mostly by corporations and states.

Put another way, datafication combines two processes: the transformation of human life into data through processes of quantification, and the generation of different kinds of value from data. Despite its clunkiness, the term datafication is necessary because it signals a historically new method of quantifying elements of life that until now were not quantified to this extent.

The process of quantifying life itself requires various components and conditions. First, as we already identified, it involves mechanisms of data collection. This can take many forms, but very often involves an app or platform that collects wide-ranging data about users, aggregates and analyses the data, and generates micro-targeted marketing data and predictive insights about behaviours. Some platforms such as Facebook have acquired the power to incorporate links to their mechanisms of data gathering within other platforms, turning Facebook itself in all its manifestations into a ‘data infrastructure’ (Nieborg and Helmond, 2019). The process is then monetised by using such data to sell products or services to the users, or by selling the data to parties wishing to influence or persuade users towards various goals. But that infrastructure also involves prior conditions: the condition of encouraging people to use the app or platform, that is, organising their habits so that life actions previously performed elsewhere (such as communicating with friends, sharing cultural products, hailing a taxi, etc.) become actions performed via the app. Even more importantly, the process of quantification involves abstraction via the process of turning the flow of social life and social meaning into streams of numbers that can be counted. This form of abstraction involves many subtle transformations, both cognitive and evaluative, as management theorists Cristina Alaimo and Jannis Kallinikos describe (2017). The transformations of social life that are inherent to datafication are so many, and so consequential for our orientation to the social domain, that Alaimo and Kallinikos write of a “computed sociality” (2017, p. 177; see also Van Dijck, 2013, p. 5, on “platformed sociality”).

Even though these processes are relatively new, the basic idea of datafication—that the flow of human life could be converted into discrete data—has a long history.

Datafication: from past to present

Datafication is implicated in more than just social media apps and content sharing platforms. The first domain of datafication was business, not social life. Even today, the amount of data generated by commerce exceeds the amount of data generated by the datafication of human life (Chairman’s Letter in IBM, 2018). Key areas of business, such as logistics—the management of the flow of goods and information—have matured into complex practices thanks to datafication. The monitoring of continuously connected data flows to organize all aspects of production and distribution across space and time within global commodity chains could not be achieved without datafication (Cowen, 2015).

But there are many other ways in which aspects of the social world came to be counted or quantified during modernity, as a way of making it more ‘legible’ for governing (Poovey, 1998, chapters 2 and 7; Scott, 1990). One of particular importance is social network analysis, where applications of network science to social domains have contributed to the evolution of datafication. Social graphs and network visualisations have allowed corporations to extract information from the flow of life for descriptive and predictive use, aided by the incorporation of “smart” devices into these social circles (the so-called Internet of Things), which record not just interactions between people, but between people and things, or between things themselves.

Issues of power permeate these apparently neutral forms of datafication. The reason derives from the underlying way in which data is produced so that it can be counted. In a network, nodes only recognise other nodes, and if something is not represented as a node it does not exist. Likewise, a process or entity can only be represented in a network if it can be described in terms of the relations that the network can count or process. Something that cannot be codified as a potential network member cannot be accounted for. This process of nodocentrism (Mejias, 2013) is similarly implicit in the social modelling that renders social flux into data-driven computer processes (Rieder, 2012). When such schemes are applied, the result is the transformation of the very ways in which the social world is accounted for, as various sociologists have noted (Fourcade and Healey, 2013; Espeland and Sauder, 2007). The question of who is doing this codifying of life into datafied realities acquires extreme importance at this point.

Yet the effects of power that are intrinsic to datafication are often made invisible. Paradoxically, much-used metaphors that equate datafication to other extractive processes help to further obscure, not uncover, these power relations. Consider the saying that “data is the new oil”, something that can be naturally extracted or mined since it exists in the “ground” of social life. As legal scholar Lauren Scholz notes, this metaphor “sidesteps evaluation of any misappropriation or exploitation that might arise from data use” (Scholz, 2018, p. 2). This understanding of datafication as somehow a natural process is surprisingly common, as evident in this sentence from an information booklet distributed by the UK’s Royal Society: “Machine learning is a brand of artificial intelligence that allows computer systems to learn directly from examples, data and experience” (2019, n.p.). The idea of direct learning from data is regarded by many critical data scientists as mythical; it is part of a discourse which critical disciplines have attempted to debunk, as we will see in the next section.

Controversies over datafication

Important controversies over social justice have emerged about how datafication is applied by corporations or states in particular sectors (from credit ratings to social services) to discriminate against individuals particularly from disadvantaged classes and ethnic populations (e.g., Gandy, 1993; Eubanks, 2017; Benjamin, 2019). More broadly, disciplines like political economy, legal studies, and decolonial theory approach the social quantification sector’s work from different angles, each drawing on critical data studies.

Political economy

Marxist critiques of data production have mostly analysed the power dynamics inherent to datafication by focusing on a traditional interpretation of labour relations, looking at the "labour" that users perform by interacting with digital media and generating data (Fuchs and Mosco, 2017). Outside the Marxist tradition, similar critiques of digital labour and data production have emerged (cf. Scholz, 2016), while management scholar Shoshana Zuboff has advanced the thesis that the large-scale collection of personal data by corporations represents an aberrant form of capitalism (Zuboff, 2015, 2019). Common to these approaches is the fact that, as a social process, datafication is linked to the generation of profit—whether through data’s sale as a commodity or data’s incorporation as a factor of production (Sadowski, 2019, alternatively formulates data itself as ‘capital’).

However, recent critical work on datafication looks beyond the idea of labour. One approach is to consider the economic form constituted by the platforms across which so much data is generated and collected. Platforms represent much more than a commercial label for computing interfaces, as Tarleton Gillespie first noted (2010). They are a fundamental new kind of multi-sided market focused on datafication, a market that brings together platform users who generate data, data buyers (advertisers and data brokers), and platform service providers who benefit from the release, sale, and internal use of data (Rieder and Sire, 2014; Cohen, 2018).

Another approach interprets datafication via a rereading of Marx to argue that the most fundamental characteristic of datafication is not labour, but the abstracting force of the commodity, that is, the very possibility of transforming life processes into “things” with value through abstraction (Couldry and Mejias, 2018, 2019; Sadowski, 2019). This interpretation frames datafication as a social process configured around new relations (“data relations”) designed to optimise the generation of data from social life (compare to Zuboff, 2015, 2019).

Legal studies

Legal theory offers an alternative critique of datafication, arguing that datafication threatens the basic rights of the self. This is already suggested in the first sentence of the General Data Protection Regulation (GDPR): “the protection of natural persons in relation to the processing of personal data is a fundamental right” (Recital 2). The risks from the collection of personal data for individual autonomy have been predicted for at least two decades (cf. Schwartz, 1999; Cohen 2000). Legal theorist Julie Cohen in particular has argued for the importance of holding onto the concept of privacy in some form as a defense versus the chilling effects of continuous data collection and processing (Cohen, 2013). The processes of datafication are so wide-ranging, however, that others have raised questions about the usefulness of the term ‘privacy’ itself (Barocas and Nissenbaum, 2014). In a world where datafication seems continuous and multi-layered, there is clearly a need for a more contextual approach to the norm of privacy (Nissenbaum, 2013).

Lately, questions have emerged about the implications of datafication—and artificial intelligence based on processing data—for the concept of autonomy (Hildebrandt, 2015). The datafication enabled by things like self-tracking devices, psychometric algorithms, and workplace tracking systems arguably interferes with the minimal integrity of the self as a self (Couldry and Mejias, 2019), which can be understood as the very basis of autonomy. Similar concerns have been expressed in terms of attempts by marketers and others to influence behaviour through data analytics (cf. Rouvroy, 2015, on “data behaviorism”). This line of critique argues that we are, through datafication, becoming dependent on (external, privatised) data measurements to tell us who we are, what we are feeling, and what we should be doing, which challenges our basic conception of human agency and knowledge.

Nonetheless, datafication creates practical openings for proposals for regulation. One such opening revolves around the question of who owns the data. There are competing interests set up by datafication, which means regulatory nuances have to be worked out. On one side, there are the interests of the individual who generates data or owns a device that produces the data; on the other, there are the interests of the owners of the infrastructure through which data flows and is collected (the social quantification sector). The latter usually ask the former to forgo any ownership rights to their data as a condition for using their infrastructure, sometimes framing access to the infrastructure as a “free” service that offsets the surrendering of property rights. Regulators, mostly in the EU through efforts such as the GDPR, are starting to intervene in this relationship to uphold some minimal rights for the individual.

Legal critiques sometimes imply an even broader question: how is it that human life came to be datafied—treated as an open domain for data extraction—in the first place (Cohen, 2018)? This is better understood in a longer historical perspective, which decolonial critiques provide.

Decolonial theory

If datafication within capitalism is a process of abstracting and extracting life across various spaces to generate profit (with ancillary benefits for governments), then where does the wealth generated by this extraction go, and why? In order to examine the geography and politics of datafication (Thatcher et al., 2016), a connection to historical colonialism might be instructive.

Datafication can be understood as itself a colonial process, not just in the metaphorical sense of saying things like “data is the new oil”, but quite literally as a new mode of data colonialism (Couldry and Mejias, 2019) that appropriates human life so that data can be continuously extracted from it for the benefit of particular(Western, but also increasingly global capitalist)interests. Instead of territories, natural resources, and enslaved labour, data colonialism appropriates social resources. While the modes, intensities, scales and contexts of data colonialism are different from those of historic colonialism, the function remains the same: to dispossess.

Within this wider perspective, datafication can be analysed as a continuation of the coloniality of power (Quijano, 2007), a form of domination in both social and cognitive domains (de Sousa Santos, 2016). A war for the social resources of the world is currently being waged between the social quantification sectors of China and the United States, principally (Couldry and Mejias, 2019). This “land grab” employs a whole arsenal of quantification weapons, from artificial intelligence, facial recognition, and new e-commerce models, to cyberwarfare, chip manufacturing, and multinational agreements regulating intellectual property. It is important to recall that, historically, information and communication technologies enabled the administration and surveillance of colonised territories, as well as the propagation of narratives that legitimised extraction and dispossession. Datafication continues and extends these functions.

Conclusion

The analytical value of the term “datafication” lies in its ability to name the processes and the frameworks by which a new form of extractivism is unfolding in our times, via the appropriation of data about our lives. Corporations are the main actors in, and beneficiaries of, this process, with government in many countries having a strong stake in the process as well. Assuming that the problem is not with data per se (there are indeed consensual community projects for data collection), but with how and by whom it is systematically collected and used, a key question becomes how to halt the social quantification sector’s expansion across social space. How do we stand outside datafication, when it seeks to capture the entirety of social space and time?

The term datafication itself can suggest practical ways to do this. By naming a process (datafication), we also invoke its limits. Just like the colonial project involved the separation of the world into centres and peripheries, datafication as a form of rationality also creates peripheral (or paranodal, cf. Mejias, 2013) things that cannot be quantified, and so, in principle, cannot be datafied.

Various forms of resistance—from the ineffective but strategic opting out of individual platforms, to a larger awareness of ourselves as the objects of datafication—can contribute to creating challenges and alternatives to the growth of datafication. Whether such resistance becomes successful in halting certain aspects of datafication remains uncertain, but it is surely one of the major social questions of our time.

Defining concepts of the digital society

In our research on ‘artificial intelligence’, robots or autonomous systems in Berlin it is a recurring theme that preconceived images shape many of the expectations and fears associated with technologies. These images, however, do not necessarily reflect actual capabilities. Phenomena such as “machine learning” or “decision-making systems” are often misguidedly attributed with notions of intentionality, free will or consciousness. Still, these imaginations and figures of speech have actual political and social clout, shape research and technological development goals and inform discourses on regulation, innovation and potential futures.

Terminology shapes reality. What’s true for the phenomena that we address in our research is certainly also true for the terminology we use for our research. What at first sounded like a banal truism, for us gradually evolved into the idea for this project, establishing a new special section Defining concepts of the digital society. At a time, when branding new, occasionally innovative but often only catchy terms has become a familiar activity of researchers, companies and policymakers alike, we felt it was particularly necessary to reflect on which of these concepts was actually worthwhile, provided analytic value and actually described something new – besides the fluffy rhetoric that repeatedly becomes rampant in academic discourse.

Algorithmic governance, autonomous systems, transparency, smart technologies– these concepts are among the best candidates to serve this cause. They have become part of the vocabulary that is mobilised to make sense of the current rapid social and technological change. In this quest to understand the digital society, some ideas have proved to be more successful than others in stimulating public discourse, academic thinking, as well as economic and political activities. More recently, platformisation and datafication have become household-terms although relating to highly complicated and multi-facetted phenomena that could potentially also be described differently. Some concepts even strongly shape public and policy discourse albeit lacking solid empirical validation (the commonly referenced filter bubble is a case in point here).

There is high demand for concepts and explanations that condense the complexity of the world by transforming it into cogent and manageable ideas. Empirical research typically addresses single aspects of the current transformations. Adding small pieces to the puzzle, individual reports, research papers and essays tend to be rather unconnected, sometimes even resisting being combined with each other. While they certainly have a heuristic value, for example by validating or falsifying assumptions for well-defined, yet restricted contexts, they cannot provide overarching explanations and narratives. This is where more abstract concepts come into the picture. Operating on the level of middle range theories, they are able to integrate diverse phenomena under one notion by foregrounding certain shared characteristics. We need those overarching concepts to make sense of the current transformations.

A new special section defining concepts of the digital society

With this new special section Defining concepts of the digital society in Internet Policy Review, we seek to foster a platform that provides and validates exactly these overarching frameworks and theories. Based on the latest research, yet broad in scope, the contributions offer effective tools to analyse the digital society. Their authors offer concise articles that portray and critically discuss individual concepts with an interdisciplinary mindset. Each article contextualises their origin and academic traditions, analyses their contemporary usage in different research approaches and discusses their social, political, cultural, ethical or economic relevance and impact as well as their analytical value. With this, the authors are building bridges between the disciplines, between research and practice as well as between innovative explanations and their conceptual heritage.

We hope that this growing collection of reference papers will succeed in providing guidance for research and teaching as well as inform stakeholders in policy, business and civil society. For scholars, the articles seek to constitute an instructive reference that points to current research, historical and (interdisciplinary) backgrounds of the respective concepts, and relevant ongoing debates. For teachers and students alike, the articles offer an accessible overview that covers and contextualises broad themes while providing useful pointers to further research. Being relatively short and accessible in format, the articles thrive to become instructive and relevant beyond academia. Stakeholders in policy and business as well as journalists and civil society are increasingly interested in research evidence and academic perspectives on the entanglement of digitalisation and society. With its newly developed format the special section helps to navigate relevant research fields for these interdisciplinary questions. As an ongoing publication in this journal on internet regulation, we hope to not only meet the existing demand for overarching concepts and explanations but also being able to quickly adapt to the rapidly changing transformations.

The politics of concepts – and the limits of this special section

Terms and concepts are lenses on the complexity of reality that foreground some aspects while neglecting others. They bear normative assumptions, install specific ways of understanding new phenomena, and possibly even create regulatory implications. The more we use these terms, the more both the phenomena they refer to as well as their specific framing increasingly become self-evident and ordinary. At the same time, however, each of these concepts has its own ideational, theoretical and rhetorical histories rooted, for example, in social theory or political thought, but also on a very practical level in business decisions to invest in certain ideas or policy debates with their own discursive rules. As a consequence, these concepts are far from being natural, let alone a neutral designator of existing phenomena. Concepts always bear their own politics – and in mobilising them, we need to carefully and critically reflect these politics and the choices they represent.

Of course, this special section on concepts of the digital society is necessarily and inescapably part of the very politics it seeks to reflect. By choosing certain terms over others, giving voice to a selection of authors, their respective disciplines and viewpoints, the special section itself undoubtedly takes part in the hierarchisation of terms and ideas. One could easily point at the limitations that result from providing predominantly Western perspectives, an uneven mix of disciplinary positions, even the dominant representation of certain auctorial subjectivities in terms of gender, race or ethnicity. Ultimately, any form of conceptual work struggles with blind spots. While we certainly acknowledge that the project poses challenges, we are certain that it is a worthwhile and necessary endeavour.

The special section is a continuing project. This first collection of five concepts offers a critical assessment of prominent, yet hitherto often nebulous or vague ideas, terms or descriptions. It does by no means seek to provide a finite and unalterable list of definitions. Its very objective is to encourage dialogue and contestation. We explicitly invite contributions to promote dialogue between the concepts and also to take counter-positions. With mostly co-authored pieces representing differing academic disciplines, the special section is already striving for a heterogeneity of viewpoints in individual papers. The larger quest of the project is to offer a genuine multitude of positions, extending, opposing or updating the concepts, their premises or consequences.

With this special section we are seeking to find a middle ground between the conceptual challenges and the aim of providing short and focused concept papers on the one hand and what we regard as the unquestionable need for interdisciplinary, concise and scholarly rigorous contributions that help to understand digital societies. This is the prime objective of this project.

First articles in the special section and future concepts

This launch of the special section in Internet Policy Review represents only the first installment of an ongoing project that seeks to build both a repertoire of instructive concepts and a platform to contest and elaborate on already published ones. Further iterations with additional concepts and commentaries on existing papers will follow in regular intervals.

With this first collection, the special section particularly focuses on the important role of data, the practices of their production, dissemination and trade as well as the ensuing broader social, political and cultural ramifications. Ulises A. Mejias and Nick Couldry look at the concept of datafication which describes a cultural logic of quantification and monetisation of human life through digital information. They identify the major social consequences which are aligned at the intersection of power and knowledge: in political economy, datafication has implications for labour and the establishment of new markets. Not only in this regard is it closely connected to the tendency – and concept – of platformisation (see below). With the help of decolonial theory Mejias and Couldry put particular emphasis on the politics and geography of datafication in what they call data colonialism: the large-scale extraction of data equals the appropriation of social ressources with the general objective (mostly by Western companies) to “dispossess”. In the context of legal theory, Mejias and Couldry note that the processes of datafication are so wide-ranging that basic rights of the self, autonomy and privacy are increasingly called into question.

It is exactly this disposition of once authoritative ideas that has become quite fragile. In this context, Tobias Matzner and Carsten Ochs analyse the concept of privacy in relation to changing socio-technical conditions. They emphasise the need to understand and theorise privacy differently with the advent of digital technologies. These “shift the possibilities and boundaries of human perception and action” by creating visibilities and forms of interaction that are no longer defined by physical presence: personal information or pictures become potentially accessible for a worldwide audience, data “is easy and cheap to store” and becomes permanent in digital records. In addition to these technical contexts they argue that the scope of the “inherent individualism” of “conventional privacy theories” and data protection legislation does not meet the needs brought about by datafication: the forms of aggregated data used to identify behavioural patterns, they argue, is not the same as personal data.

One of the reasons why these forms of aggregated data operate at said intersection of knowledge and power is the practice of increasingly managing social spaces and interactions with algorithmic systems. Christian Katzenbach and Lena Ulbricht discuss algorithmic governance as a notion that builds on the longstanding theme that technology allows for a specific mode of governing society. Datafication, increasing computing power, more sophisticated algorithms, the economic and political interest in seemingly efficient and cost-reducing solutions, as well as the general trend towards digitalisation have all contributed to the new appeal and actual deployment of technological means to order the social. Eschewing the deterministic tendencies of the notion, yet taking seriously the increasing influence of algorithmic systems, the authors discuss a range of sectors from predictive policing to automated content moderation that increasingly rely on algorithmic governance. The concept brings previously unconnected objects of inquiry and research fields together and allows to identify overarching concerns such as surveillance, bias, agency, transparency and depoliticisation.

Many of these developments are primarily attributed to what we have converged on calling platforms: huge, often globally operating companies and services such as Facebook and Alibaba, Google and Uber that seek to transform and intermediate transactions across key economic sectors to position themselves as indispensable infrastructures of private and public life. Thomas Poell, David Nieborg and José van Dijck discuss platformisation as key development and narrative of the digital society. They argue that academic disciplines need to join forces in order to systematically investigate how changes in infrastructures, market relations and governance frameworks are intertwined, and how they take shape in relation to shifting cultural practices. We are only starting to understand how and why platforms have become the dominant mode of economic and social organisation and what the long-term effects might be.

One of the more prominent notions that seek to capture the effects of the reorganisation of social life by platforms and datafication is the metaphor of the filter bubble. Axel Bruns critically discusses this concept and carves out why it holds a special position in the set of concepts in this special section: while the idea of an algorithmically curated filter bubble seems plausible and enjoys considerable popularity in public and political discourse, empirical research shows little evidence that the phenomenon actually exists. Based on different readings of the concept and existing studies, Bruns argues that, rather than acutely capturing an empirical phenomenon, the persistent use of the notion has now created its own discursive reality that continues to have an impact on societal institutions, media and communication platforms as well as the users themselves. In consequence, the notion might even redirect scholarly attention away, warns Bruns, from far more critical questions such as why different groups in society “come to develop highly divergent personal readings of information” in the first place, and how the “ossification of these diverse ideological perspectives into partisan group identities” can be prevented or undone.

In 2020 the special section will continue, featuring concepts such as Digital commons, Transparency, Autonomous systems, Value in design and Smart technologies. Honouring the openness of the project, we appreciate suggestions for future concepts to be considered and any constructive feedback on the project itself. We sincerely hope the special section Defining concepts of the digital society will become a valuable forum and a helpful resource for many.

Acknowledgments

For an academic publication project such as this, most credit is routinely attributed to only a few named authors and editors. The success of a publication, however, always builds on a much broader group of people. This is particularly true for this special section. The long journey from the first idea to the publication of this collection of concepts was made possible by the help of many. We thank first and foremost Frédéric Dubois, the Internet Policy Review’s managing editor who has steered this rocky ride from beginning to end, with careful attention, from the overarching process to the details of wording. Uta Meier-Hahn helped to push the idea towards realisation by drafting a first exposé. The board of the Internet Policy Review and colleagues gave valuable guidance, especially Melanie Dulong De Rosnay, Jeanette Hofmann, David Megías Jiménez, Joris van Hoboken, Seda Guerses and Lilian Edwards provided instrumental feedback on the way. And Patrick Urs Riechert gave all the manuscripts the final polish. Thank you all!

Thomas Christian Bächle and Christian Katzenbach

Berlin, December 2019

​The public is increasingly concerned with the abilities of data collectors, like Facebook and Google, to understand and influence individual users. Most data collectors rely on online technologies to do so. We define online technologies as connected data-gathering software, like social media algorithms, or hardware, like smartwatches, that interact with users. For example, by sending users push-notifications or compiling content based on user preferences.

This public concern has resonated in academia. More and more researchers argue that online technologies manipulate human users and, therefore, undermine their autonomy. We call this the MAL view on online technology because it argues from Manipulation to Autonomy-Loss. MAL enjoys public visibility and will shape the academic discussion to come. 

This view of online technology, however, fails conceptually. MAL presupposes that manipulation equals autonomy loss, and that autonomy is the absence of manipulation. That is mistaken. In short, an individual can be manipulated while being fully personally autonomous. 

Internet policy researchers should be aware of this point to avoid looking in the wrong place in future research on manipulative and harmful online technology. Showing that manipulative online technology leads to autonomy-loss requires empirical testing, or so we will argue. 

1. Reconstructing the Manipulation to Autonomy-Loss (MAL) view

We will illustrate MAL in more detail by discussing a recent article by Daniel Susser, Beate Roessler, and Helen Nissenbaum in this journal (2019). Their article presents a well-informed and lucid account of the potentially harmful effects of online technology. Since they articulate their assumptions about the relationship between manipulation and autonomy lucidly, their article helps to illustrate what is mistaken about the MAL view of online technology. 

Susser et al.’s argument involves three crucial claims. (1) A claim about the influence of online technologies on users (call this Influence). (2) A claim about the manipulativeness of online technologies (call this Manipulation). (3) The MAL claim, according to which manipulation equals autonomy loss. Schematically, their argument goes as follows:

Influence: Online technologies influence human users. 

Manipulation: Online technologies are manipulative.

MAL: If an influence is manipulative, then it is autonomy undermining. 

Conclusion: So, online technologies are autonomy undermining.

In support of Influence, they note how data collectors compile our online traces “into enormously detailed profiles,” which can then be used by “advertisers and others engaging in behavioural targeting […] to detect when and how to intervene in order to most effectively influence us” (p. 6, page numbers refer to Susser et al.’s article). Moreover, they suggest that “digital surveillance enables detection of increasingly individual- or person-specific vulnerabilities,” including the exploitation of cognitive biases and other needs (ibid.). We take Influence to be well supported.

They defend Manipulation by defining online manipulation as follows. Manipulation is “the use of information technology to covertly influence another person’s decision-making, by targeting and exploiting decision-making vulnerabilities” (p. 6). They then argue that, given Influence, online technologies plausibly manipulate users. Importantly, they claim that to exploit individuals’ decision-making vulnerabilities is to fail to “encourage individuals to slow down, reflect on, and make more informed choices” (ibid.).

Finally, in support of MAL, they write that “manipulation violates its target’s autonomy” (p. 8). To unpack this claim, need to introduce their account of autonomy and then explain how manipulation jeopardises it. They define personal autonomy with two conditions. (1) One has the competencies (cognitive and affective) to consider one’s choices and to act upon them. (2) One reflectively endorses the ends (e.g,. goals) and grounds (e.g., reasons) of one’s actions (pp. 7-8). They then establish the connection between manipulation and autonomy as follows. (Online) Manipulation, they write, “undermines a target’s autonomy in two ways: first, it can lead them to act toward ends they have not chosen, and second, it can lead them to act for reasons not authentically their own” (p. 9). 

In conclusion, Susser et al. argue that online technologies frequently manipulate and, therefore, undermine users’ autonomy, which they consider morally wrong in most cases. Thus, they claim that potential autonomy loss explains why “online manipulation poses such a grave threat” (p. 9). 

2. Clarifying the MAL view of online technology

Susser et al. do not make the nature of the manipulation-autonomy connection explicit. What they write leaves open two options. A contingent reading of the claim (roughly, manipulating S often or mostly undermines S’s autonomy). A necessary reading (roughly, manipulating S always undermines S’s autonomy). 

The contingent reading is the weaker claim, because it allows for more exceptions, and thus the more charitable reading of their argument. However, they explicitly define manipulation as covertly influencing someone so that they fail to “slow down reflect on, and make more informed choices” (p. 6). So, it becomes hard to see how there could be genuine cases of manipulation (on their account) without autonomy loss (again, on their account of autonomy). 

Moreover, they consider it a sign of manipulation that “one did not understand one’s motivations” (p. 4) and that one was “directed, outside one’s conscious awareness, to act for reasons one can’t recognise, and toward ends one may wish to avoid” (p. 4).

There are thus clear signs that support interpreting Susser et al.’s endorsement of MAL as a necessary conceptual link between manipulation and autonomy-loss. 

3. Challenging the MAL view of online technology

The move from manipulation to autonomy-loss does not stand up to scrutiny. To see why it helps to look at the conditions for MAL to be true. MAL is a view of a conceptual link between manipulation and autonomy-loss. It says that whenever one finds something that is manipulative, one has found something that is autonomy-undermining. 

There are many such conceptual links. For example, whenever one encounters a bachelor, one encounters an unmarried man – the concept ‘bachelor’ implies the concepts ‘unmarried’ and ‘man.’ However, there is a danger of jumping to conclusions here. We should be wary of letting contingent empirical observations confuse our claims about conceptual necessity. For example, it is an empirical fact that, say, many or most bottles are plastic. Nevertheless, we cannot conclude that the concept ‘bottle’ implies the concept of ‘plastic.’ The relation is empirical, not conceptual. 

The lesson is this. In an argument about bachelors, we only need to show that someone is a bachelor to get the result that he is unmarried ‘for free,’ by courtesy of a conceptual link. However, in an argument about bottles, we do not get the claim that a given bottle is made from plastic ‘for free,’ because there is no conceptual link between ‘bottle’ and ‘plastic.’

The MAL view makes the same mistake. It suggests that there is a necessary conceptual link between manipulation and autonomy-loss. But that is mistaken. There are cases of manipulation that are not autonomy-undermining, on any plausible understanding of personal autonomy. 

Susser et al. understand personal autonomy in broadly externalist terms. According to externalist approaches, personal autonomy comes down to the extent to which the agent can appreciate and endorse her reasons for acting. The intuition behind externalist approaches is as follows. A person cannot wholly ‘own’ her actions, or act for reasons “authentically their own” (p. 9), insofar as she does not take some appropriate attitude like endorsement or understanding toward her reasons for acting. We will look at the two most influential externalist accounts in philosophy.

One prominent externalist conception of autonomy goes as follows. The ability to assess and chose an action is fleshed out as an agent’s ability to evaluate her motives based on whatever else she believes and desires, and to adjust her motives in response to these evaluations (Christman, 1991). For example, indoctrinated people are not autonomous. Their indoctrination prevents them from evaluating doctrine in light of their own (potentially) critical beliefs and emotions. Susser et al. credit this conception as the basis of their account.

There is an alternative externalist conception. On this view, the ability to assess and chose an action has been fleshed out as an agent’s ability to appropriately respond to a sufficiently wide range of reasons for and against behaving as she does (Fischer & Ravizza, 1998). For example, there are reasons for and against pursuing a challenging career (e.g., personal reward vs less family time). One acts autonomously when one can ‘feel the pull’ of both reasons for and against a particular act. 

Neither conception of externalist autonomy implies that manipulation is incompatible with autonomy. Consider the following example:

Breakthrough: Johannes cherishes autonomy above everything else, and he wants others to be autonomous, too. He creates a self-optimisation app called Breakthrough that helps users to free themselves of societal expectations and conventions and to determine for themselves the lives they want to live. Breakthrough reminds users of their goals. It points out how societal expectations may have contributed to their choice. It also creates opportunities for users to reflect on and potentially revise their motives and goals. It does so in light of the user’s motives and also in light of what the app’s advanced algorithm deems good reasons for doing something, e.g., eating healthier. The ultimate aim is for users to breakthrough. To abscond any habitual, unconsidered, socially-influenced action so that they take any action with full emotional and cognitive endorsement, in line with all their ends and grounds. Cordula is an avid user of the app and eventually breaks through. She would not have thought how much the app would change her life. Amongst other things, she stops seeing several long-term friends, to whom her relationship seemed merely conventional and not genuine, to focus prepping for a triathlon. She is ok with that, however, because she prefers being fully autonomous to her former life. 

Cordula is autonomous according to either externalist conception of autonomy. She responds well to reasons (e.g., reasons for eating healthier) and to reasoning (e.g., to eat healthier, given that she wants to be healthier and committed to that goal). Indeed, that is the very aim of the Breakthrough app and the very reason that Cordula is using it. Nevertheless, Cordula seems to be manipulated by the Breakthrough app. There is a sense in which she lives a life that is authentically hers, because of the way she reflects on and endorses her motives and reasons. However, there is also a nagging sense that she may have given up too much of her life to the Breakthrough app. The app seems to exert an overpowering and illegitimate influence on her behaviour. She seems manipulated by Breakthrough. Therefore, manipulation need not undermine externalist autonomy, contrary to Susser et al.’s argument. 

That observation generalises and thus relies less on potentially problematic intuitions about particular cases. If someone, like Cordula, reflectively endorses an action (like eating healthier), we can always ask how she arrived at her endorsement. We can then ask whether those grounds are authentically hers. And so on – into a regress. Manipulation can sneak in anywhere in that line. Externalist accounts must allow it on pain of raising the bar much too high for autonomous action (cf. Gorin, 2014, p. 89).

Let us recap. Susser et al. defend the MAL view, the view that online technology manipulates and, therefore, undermines autonomy. They defended this view on the assumption that manipulation equals autonomy-loss and they understood autonomy externalistically. We suspect that Susser et al. are not the only ones who embrace the MAL view on online technology. Other scholars also operate with a broadly externalist conception of autonomy and suggest that manipulative online technology undermine autonomy, though often less explicitly. For example, Frischmann and Selinger see autonomy in an externalist light as they link it to unhampered “self-reflection” and the ability to “determine one’s own intentions” (Frischmann & Selinger, 2018, 18, 153). They see that type of autonomy in jeopardy as “we’re being conditioned to obey” by online technologies (2018, pp. 4–6). 

However, as we have shown, both intuitive cases and general theoretical considerations suggest that manipulation does not necessarily undermine autonomy on an externalist understanding. Manipulation does not equal autonomy-loss. The MAL view on online manipulation fails. 

4. Implications for internet policy research

The failure of the MAL view of online technology has three crucial implications for internet policy researchers interested in online technology and autonomy-loss. 

First, one should do better conceptual work to understand manipulation in such a way that manipulation does not equal autonomy-loss (cf. Klenk, forthcoming). The consequences of classification are not merely terminological but practical. Manipulative technologies would, and should, be subject to different policies than non-manipulative technologies. 

Second, one could do additional conceptual work to identify conceptual links to go from the influence of online technology to autonomy-loss. The concept of manipulation will not be able to do this work. But there might be others, like coercion. 

Third, one should do empirical work on the experiences that lead people to feel their autonomy compromised in the context of online technology. MAL depends entirely on the conceptual link between manipulation and autonomy-loss. Since we cut that link, we need new ways to show that online technologies subvert autonomy, if they do. This goes to show that this is not this just a semantic worry about the meaning of the word manipulation or the concept of manipulation. At stake is the genuine problem of how online technologies affect autonomy. 

5. Conclusion

Online technology can manipulate us without compromising our autonomy. It is plausible that manipulation is compatible with autonomy, and that autonomy-loss can come by other means than manipulation. Hence, the MAL view of online technology, and Susser et al.’s argument that depends on it, fail. 

Several other scholars (e.g., Zuboff, 2019) make an equally problematic assumption about the link from Autonomy-loss to Manipulation (what we call the ALM view). If our argument in this paper is any indication, the ALM view is ripe for a reality check, too. 

Going forward, we will need more conceptual work on the concept of (online) manipulation, and more empirical work to test its links to autonomy(-loss).

References

Christman, J. (1991). Autonomy and personal history. Canadian Journal of Philosophy, 21(1), 1–24.

Fischer, J. M., & Ravizza, M. (1998). Responsibility and control: A theory of moral responsibility. Cambridge: Cambridge University Press. 

Frischmann, B. M., & Selinger, E. (2018). Re-engineering humanity. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781316544846

Gorin, M. (2014). Towards a theory of interpersonal manipulation. In C. Coons & M. Weber (Eds.), Manipulation: Theory and practice (pp. 73–97). Oxford: Oxford University Press.

Klenk, M. (forthcoming). Digital well-being and manipulation online. In C. Burr & L. Floridi (Eds.), Ethics of Digital Well-Being: A Multidisciplinary Approach. Retrieved from https://philpapers.org/rec/KLEDWA

Susser, D., Roessler, B., & Nissenbaum, H. (2019). Technology, autonomy, and manipulation. Internet Policy Review, 8(2), 1–22. https://doi.org/10.14763/2019.2.1410

Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. New York, NY: PublicAffairs. 

Introduction

In recent years, the spread of disinformation on online platforms and micro-targeted data-driven political advertising has become a serious concern in many countries around the world, in particular as regards the impact this practice may have on informed citizenship and democratic systems. In April 2019, for the first time in the country’s modern history, Switzerland’s supreme court has overturned a nationwide referendum on the grounds that the voters were not given complete information and that it "violated the freedom of the vote”. While in this case it was the government that had failed to provide correct information, the decision still comes as another warning of the conditions under which elections nowadays are being held and as a confirmation of the role that accurate information plays in this process. There is limited and sometimes even conflicting scholarly evidence as to whether today people are exposed to more diverse political information or trapped in echo chambers, and whether they are more vulnerable to political disinformation and propaganda than before (see, for example: Bruns, 2017, and Dubois & Blank, 2018). Yet, many claim so, and cases of misuse of technological affordances and personal data for political goals have been reported globally.

The decision of Switzerland’s supreme court has particularly resonated in Brexit Britain where the campaign ahead of the European Union (EU) membership referendum left too many people feeling “ill-informed” (Brett, 2016, p. 8). Even before the Brexit referendum took place, the House of Commons Treasury Select Committee complained about “the absence of ‘facts’ about the case for and against the UK’s membership on which the electorate can base their vote” (2016, p. 3). According to this, the voters in the United Kingdom were not receiving complete or even truthful information, and there are also concerns that they might have been manipulated by the use of bots (Howard & Kollanyi, 2016) and by the unlawful processing of personal data (ICO, 2018a, 2018b).

The same concerns were raised in the United States during and after the presidential elections in 2016. Several studies have shown evidence of the exposure of US citizens to social media disinformation in the period around elections (see: Guess et al., 2018, and Allcott & Gentzkow, 2017). In other parts of the world, such as in Brazil and in several Asian countries, the means and platforms for transmission of disinformation were somewhat different but the associated risks have been deemed even higher. The most prominent world media, fact checkers and researchers systematically reported about the scope and spread of disinformation on the Facebook-owned and widely used messaging application WhatsApp in the 2018 presidential elections in Brazil. Freedom House warned that elections in some Asian countries, such as India, Indonesia, and Thailand, were also afflicted by falsified content.

Clearly, online disinformation and unlawful political micro-targeting represent a threat to elections around the globe. The extent to which certain societies are more resilient or more vulnerable to the impact of these phenomena depends on different factors, including, among other things, the status of journalism and legacy media, levels of media literacy, the political context and legal safeguards (CMPF, forthcoming). Different political and regulatory traditions play a role in shaping the responses to online disinformation and data-driven political manipulation. Accordingly, these range from doing nothing to criminalising the spread of disinformation, as is the case with the Singapore’s law1 which came into effect in October 2019. While there seems to be more agreement that regulatory intervention is needed to protect democracy, the concerns over the negative impact of inadequate or overly restrictive regulation on freedom of expression remain. In his recent reports (2018, 2019), UN Special Rapporteur on Freedom of Expression David Kaye warned against regulation that entrusts platforms with even more powers to decide on content removals in very short time frames and without public oversight. Whether certain content is illegal or problematic on other grounds is not always a straightforward decision and often depends on the context in which it is presented. Therefore, as highlighted by the Transatlantic High Level Working Group on Content Moderation Online and Freedom of Expression (2019), to require platforms to make these content moderation decisions in an automated way, without built-in transparency, and without notice or timely recourse for appeal, contains risks for freedom of expression.

The European Commission (EC) has recognised the exposure of citizens to large scale online disinformation (2018a) and micro-targeting of voters based on the unlawful processing of personal data (2018b) as major challenges for European democracies. In a response to these challenges, and to ensure citizens’ access to a variety of credible information and sources, the EC has put in place several measures which aim to create an overarching “European approach”. This paper provides an analysis of this approach to identify the key principles upon which it builds, and to what extent, if at all, they differ from the principles of “traditional” political advertising and media campaign regulation during the electoral period. The analysis further looks at how these principles are elaborated and whether they reflect the complexity of the challenges identified. The focus is on the EU as it is “articulating a more interventionist approach” to the relations with the online platform companies (Flew et al., 2019, p. 45). Furthermore, due to the size of the European market, any relevant regulation can set the global standard, as is the case with the General Data Protection Regulation (GDPR) in the area of data protection and privacy (Flew et al., 2019).

The role of (social) media in elections

The paper starts from the notion that a healthy democracy is dependent on pluralism and that the role of (social) media in elections and the transparency of data-driven political advertising are among the crucial components of any assessment of the state of pluralism in a given country. In this view, pluralism “implies all measures that ensure citizens' access to a variety of information sources, opinion, voices etc. in order to form their opinion without the undue influence of one dominant opinion forming power” (EC, 2007, p. 5; Valcke et al., 2009, p. 2). Furthermore, it implies the relevance of citizens' access to truthful and accurate information.

The media have long been playing a crucial role in election periods: serving, on one side, as wide-reaching platforms for parties and candidates to deliver their messages, and, on the other, helping voters to make informed choices. They set the agenda by prioritising certain issues over others and by deciding on time and space to be given to candidates; they frame their reporting within a certain field of meaning and considering the characteristics of different types of media; and, if the law allows, they sell time and space for political advertising (Kelley, 1963). A democracy requires the protection of media freedom and editorial autonomy, but asks that the media be socially responsible. This responsibility implies respect of fundamental standards of journalism, such as impartiality and providing citizens with complete and accurate information. As highlighted on several occasions by the European Commission for Democracy through Law (so-called Venice Commission) of the Council of Europe (2013, paras. 48, 49): “The failure of the media to provide impartial information about the election campaign and the candidates is one of the most frequent shortcomings that arise during elections”.

Access to the media has been seen as “one of the main resources sought by parties in the campaign period” and to ensure a level playing field “legislation regarding access of parties and candidates to the public media should be non-discriminatory and provide for equal treatment” (Venice Commission, 2010, para. 148). The key principles of media regulation during the electoral period are therefore media impartiality and equality of opportunity for contenders. Public service media are required to abide by higher standards of impartiality compared to private outlets, and audiovisual media are more broadly bound by rules than the printed press and online media. The latter are justified by the perceived stronger effects of audiovisual media on voters (Schoenbach & Lauf, 2004) and by the fact that television channels benefit from the public and limited resource of the radio frequency spectrum (Venice Commission, 2009, paras. 24-28, 58).

In the Media Pluralism Monitor (MPM) 2, a research tool supported by the European Commission and designed to assess risks to media pluralism in EU member states, the role of media in the democratic electoral process is one out of 20 key indicators. It is seen as an aspect of political pluralism and the variables against which the risks are assessed have been elaborated in accordance with the above-mentioned principles. The indicator assesses the existence and implementation of a regulatory and self-regulatory framework for the fair representation of different political actors and viewpoints on public service media and private channels, especially during election campaigns. The indicator also takes into consideration the regulation of political advertising – whether the restrictions are imposed to allow equal opportunities for all political parties and candidates.

The MPM results (Brogi et al., 2018) showed that the rules to ensure the fair representation of political viewpoints in news and informative programmes on public service media channels and services are imposed by law in all EU countries. It is, however, less common for such regulation and/or self-regulatory measures to exist for private channels. A similar approach is observed in relation to political advertising rules, which are more often and more strictly defined for public service than for commercial media. Most countries in the EU have a law or another statutory measure that imposes restrictions on political advertising during election campaigns to allow equal opportunities for all candidates. Even though political advertising is “considered as a legitimate instrument for candidates and parties to promote themselves” (Holtz-Bacha & Just, 2017, p. 5), some countries do not allow it at all. In cases when there is a complete ban on political advertising, public service media provide free airtime on principles of equal or proportionate access. In cases when paid political advertising is allowed, it is often restricted only to the campaign period and regulation seeks to set limits on, for example, campaign resources and spending, the amount of airtime that can be purchased and the timeframe in which political advertising can be broadcast. In most countries there is a requirement for transparency – how much was spent for advertising in the campaign, presented through spending on different types of media. For traditional media, the regulatory framework requires that political advertising (as any other advertising) be properly identified and labelled as such.

Television remains the main source of news for citizens in the EU (Eurobarometer, 2018a, 2017). However, the continuous rise of online sources and platforms as resources for (political) news and views (Eurobarometer, 2018a), and as channels for more direct and personalised political communication, call for a deeper examination of the related practice and potential risks to be addressed. The ways people find and interact with (political) news and the ways political messages are being shaped and delivered to people has been changing significantly with the global rise, popularity and features offered by the online platforms. An increasing number of people, and especially young populations, are using them as doors to news (Newman et al., 2018, p. 15; Shearer, 2018). Politicians are increasingly using the same doors to reach potential voters, and the online platforms have become relevant, if not central, to different stages of the whole process. This means that platforms are now increasingly performing functions long attributed to media and much more through, for example, filtering and prioritising certain content offered to users, and selling the time and space for political advertising based on data-driven micro-targeting. At the same time, a majority of EU countries still do not have specific requirements that would ensure transparency and fair play in campaigning, including political advertising in the online environment. According to the available MPM data (Brogi et al., 2018; and preliminary data collected in 2019), only 11 countries (Belgium, Bulgaria, Denmark, Finland, France, Germany, Italy, Latvia, Lithuania, Portugal and Sweden) have legislation or guidelines to require transparency of online political advertisements. In all cases, it is the general law on political advertising during the electoral period that also applies to the online dimension.

Political advertising and political communication more broadly take on different forms in the environment of online platforms, which may hold both promises and risks for democracy (see, for example, Valeriani & Vaccari, 2016; and Zuiderveen Borgesius et al., 2018). There is still limited evidence on the reach of online disinformation in Europe, but a study conducted by Fletcher et al. (2018) suggests that even if the overall reach of publishers of false news is not high, they achieve significant levels of interaction on social media platforms. Disinformation online comes in many different forms, including false context, imposter, manipulated, fabricated or extreme partisan content (Wardle & Derakhshan, 2017), but always with an intention to deceive (Kumar & Shah, 2018). There are also different motivations for the spread of disinformation, including financial and political (Morgan, 2018), and different platforms’ affordances affect whether disinformation spreads better as organic content or as paid-for advertising. Vosoughi et al. (2018) have shown that Twitter disinformation organically travels faster and further than true information pieces due to technological possibilities, but also due to human nature that is more likely to spread something surprising and emotional, which disinformation often does. On Facebook, on the other hand, the success of spread of disinformation may be significantly attributed to advertising, claim Chiou and Tucker (2018). Accordingly, platforms have put in place different policies towards disinformation. Twitter has recently announced a ban on political advertising, while Facebook continues to run it and exempts politician’s speech and political advertising from third-party fact-checking programmes.

Further to different types of disinformation, and different affordances of platforms and their policies, there are “many different actors involved and we’re learning much more about the different tactics that are being used to manipulate the online public sphere, particularly around elections”, warns Susan Morgan (2018, p. 40). Young Mie Kim and others (2018) have investigated the groups that stood behind divisive issue campaigns on Facebook in the weeks before the 2016 US elections, and found that most of these campaigns were run by groups which did not file reports to the Federal Election Commission. These groups, clustered by authors as non-profits, astroturf/movement groups, and unidentifiable “suspicious” groups, have sponsored four times more ads than those that did file the reports to the Commission. In addition to the variety of groups playing a role in political advertising and political communication on social media today, a new set of tactics are emerging, including the use of automated accounts, so-called bots, and data-driven micro-targeting of voters (Morgan, 2018).

Bradshaw and Howard (2018) have found that governments and political parties in an increasing number of countries of different political regimes are investing significant resources in using social media to manipulate public opinion. Political bots, as they note, are used to promote or attack particular politicians, to promote certain topics, to fake a follower base, or to get opponents’ accounts and content removed by reporting it on a large scale. Micro-targeting, as another tactic, is commonly defined as a political advertising strategy that makes use of data analytics to build individual or small group voter models and to address them with tailored political messages (Bodó et al., 2017). These messages can be drafted with the intention to deceive certain groups and to influence their behaviour, which is particularly problematic in the election period when the decisions of high importance for democracy are made, the tensions are high and the time for correction or reaction is scarce.

The main fuel of contemporary political micro-targeting is data gathered from citizens’ online presentation and behaviour, including from their social media use. Social media has also been used as a channel for distribution of micro-targeted campaign messages. This political advertising tactic came into the spotlight with the Cambridge Analytica case reported by journalist Carole Cadwalladr in 2018. Her investigation, based on the information from whistleblower Christopher Wylie, revealed that the data analytics firm Cambridge Analytica, which worked with Donald Trump’s election team and the winning Brexit campaign, harvested the personal data of millions of peoples' Facebook profiles without their knowledge and consent, and used it for political advertising purposes (Cadwalladr, 2018). In the EU, the role of social media in elections came high on the agenda of political institutions after the Brexit referendum in 2016. The focus has been in particular on the issue of ‘fake news’ or disinformation. The reform of the EU’s data protection rules, which resulted in the GDPR, started in 2012. The Regulation was adopted on 14 April 2016, and its scheduled time of enforcement, 25 May 2018, collided with the outbreak of the Cambridge Analytica case.

Perspective and methodology

Although, European elections are primarily the responsibility of national governments, the EU has taken several steps to tackle the issue of online disinformation. In the Communication of 26 April 2018 the EC called these steps a “European approach” (EC, 2018a), with one of its key deliverables being the Code of Practice on Disinformation (2018), presented as a self-regulatory instrument that should encourage proactivity of online platforms in ensuring transparency of political advertising and restricting the automated spread of disinformation. The follow up Commission’s Communication from September 2018, focused on securing free and fair European elections (EC, 2018f), suggests that, in the context of elections, principles set out in the European approach for tackling online disinformation (EC, 2018a) should be seen as complementary to the GDPR (Regulation, 2016/679). The Commission also prepared specific guidance on the application of GDPR in the electoral context (EC, 2018d). It further suggested considering the Recommendation on election cooperation networks (EC, 2018e), and transparency of political parties, foundations and campaign organisations on financing and practices (Regulation, 2018, p. 673). This paper provides an analysis of the listed legal and policy instruments that form and complement the EU’s approach to tackling disinformation and suspicious tactics of political advertising on online platforms. The Commission’s initiatives in the area of combating disinformation contain also a cybersecurity aspect. However, this subject is technically and politically too complex to be included in this specific analysis.

The EC considers online platforms as covering a wide range of activities, but the European approach to tackling disinformation is concerned primarily with “online platforms that distribute content, particularly social media, video-sharing services and search engines” (EC, 2018a). This paper employs the same focus and hence the same narrow definition of online platforms. The main research questions are: which are the key principles upon which the European approach to tackling disinformation and political manipulation builds; and to what extent, if at all, do they differ from the principles of “traditional” political advertising and media campaign regulation in the electoral period? The analysis further seeks to understand how these principles are elaborated and whether they reflect the complexity of the challenges identified. For this purpose, the ‘European approach’ is understood in a broad sense (EC, 2018f). Looking through the lens of pluralism, this analysis uses a generic inductive approach, a qualitative research approach that allows findings to emerge from the data without having pre-defined coding categories (Liu, 2016). This methodological decision was made as this exploratory research sought not only to analyse the content of the above listed documents, but also the context in which they came into existence and how they relate to one another.

Two birds with one stone: the European approach in creating fair and plural campaigning online

The actions currently contained in the EU’s approach to tackling online disinformation and political manipulation derive from the regulation (GDPR), EC-initiated self-regulation of platforms (Code of Practice on Disinformation), and the non-binding Commission’s communications and recommendations to the member states. While some of the measures, such as data protection, have a long tradition and have only been evolving, some represent a new attempt to develop solutions to the problem of platforms (self-regulation). In general, the current European approach can be seen as primarily designed towards (i) preventing unlawful micro-targeting of voters by protecting personal data; and (ii) combating disinformation by increasing the transparency of political and issue-based advertising on online platforms.

Protecting personal data

The elections of May 2019 were the first European Parliament (EP) elections after major concerns about legality and legitimacy of the vote in US presidential election and the UK's Brexit referendum. The May 2019 elections were also the first elections for the EP held under the GDPR, which became directly applicable across the EU as of 25 May 2018. As a regulation, the GDPR is directly binding, but does provide flexibility for certain aspects of the regulation to be adjusted by individual member states. For example, to balance the right to data protection with the right to freedom of expression, article 85 of the GDPR provides for the exemption of, or derogation for, the processing of data for “journalistic purposes or the purpose of academic artistic or literary expression”, which should be clearly defined by each member state. While the GDPR provides the tools necessary to address instances of unlawful use of personal data, including in the electoral context, its scope is still not fully and properly understood. Since it was the very first time the GDPR was applied in the European electoral context, the European Commission published in September 2018 the Guidance on the application of Union data protection law in the electoral context (EC, 2018d).

The data protection regime in the EU is not new, 3 even though it has not been well harmonised and the data protection authorities (DPAs) have had limited enforcement powers. The GDPR aims to address these shortcomings as it gives DPAs powers to investigate, to correct behaviour and to impose fines up to 20 million Euros or, in the case of a company, up to 4% of its worldwide turnover. In its Communication, the EC (2018d) particularly emphasises the strengthened powers of authorities and calls them to use these sanctioning powers especially in cases of infringement in the electoral context. This is an important shift as the European DPAs have historically been very reluctant to regulate political parties. The GDPR further aims at achieving cooperation and harmonisation of the Regulation’s interpretations between the national DPAs by establishing the European Data Protection Board (EDPB). The EDPB is made up of the heads of national data protection authorities and of the European Data Protection Supervisor (EDPS) or their representatives. The role of the EDPS is to ensure that EU institutions and bodies respect people's right to privacy when processing their personal data. In March 2018, the EDPS published an Opinion on online manipulation and personal data, confirming the growing impact of micro-targeting in the electoral context and a significant shortfall in transparency and provision of fair processing of information (EDPS, 2019).

The Commission guidance on the application of GDPR in the electoral context (EC, 2018d) underlines that it “applies to all actors active in the electoral context”, including European and national political parties, European and national political foundations, platforms, data analytics companies and public authorities responsible for the electoral process. Any data processing should comply with the GDPR principles such as fairness and transparency, and for specified purposes only. The guidance provides relevant actors with the additional explanation of the notions of “personal data” and of “sensitive data”, be it collected or inferred. Sensitive data may include political opinions, ethnic origin, sexual orientation and similar, and the processing of such data is generally prohibited unless one of the specific justifications provided for by the GDPR applies. This can be in the case where the data subject has given explicit, specific, fully informed consent for processing; when this information is manifestly made public by the data subject; when the data relate to “the members or to former members of the body or to persons who have regular contact with”; or when processing “is necessary for reasons of substantial public interest” (GDPR, Art. 9, para. 2). In a statement adopted in March 2019, the EDPB points out that derogations of special data categories should be interpreted narrowly. In particular, the derogation in the case when a person makes his or her ‘political opinion’ public cannot be used to legitimate inferred data. Bennett (2016) also warns that vagueness of several terms used to describe exceptions from the application of Article 9(1) might lead to confusion or inconsistencies in interpretation as processing of ‘political opinions’ becomes increasingly relevant for contemporary political campaigning.

The principles of fairness and transparency require that individuals (data subjects) are informed of the existence of the processing operation and its purposes (GDPR, Art. 5). The Commission’s guidance clearly states that data controllers (those who make the decision on and the purpose of processing, like political parties or foundations) have to inform individuals about key aspects related to the processing of their personal data, including why they receive personalised messages from different organisations; which is the source of the data when not collected directly from the person; how are data from different sources combined and used; and whether the automated decision-making has been applied in processing.

Despite the strengthened powers and an explicit call to act more in the political realm (EC, 2018d), to date we have not seen many investigations by DPAs into political parties under the GDPR. An exception is UK Information Commissioner Elizabeth Denham. In May 2017, she announced the launch of a formal investigation into the use of data analytics for political purposes following the wrongdoings exposed by journalists, in particular Carole Cadwalladr, during the EU Referendum, and involving parties, platforms and data analytics companies such as Cambridge Analytica. The report of November 2018 concludes:

that there are risks in relation to the processing of personal data by many political parties. Particular concerns include the purchasing of marketing lists and lifestyle information from data brokers without sufficient due diligence, a lack of fair processing and the use of third-party data analytics companies, with insufficient checks around consent (ICO, 2018a, p. 8).

As a result of the investigation, the ICO sent 11 letters to the parties with formal warnings about their practices, and in general it became the largest investigation conducted by a DPA on this matter and encompassing different actors, not only political parties but also social media platforms, data brokers and analytics companies.

Several cases have been reported where the national adaptation of the GDPR does not fully meet the requirements of recital 56 GDPR which establishes that personal data on people’s political opinions may be processed “for reasons of public interest” if “the operation of the democratic system in a member state requires that political parties compile” such personal data; and “provided that appropriate safeguards are established”. In November 2018 a question was raised in the European Parliament on the data protection law adapting Spanish legislation to the GDPR which allows “political parties to use citizens’ personal data that has been obtained from web pages and other publicly accessible sources when conducting political activities during election campaigns”. As a member of the European Parliament Sophia in 't Veld, who posed the question, highlighted: “Citizens can opt out if they do not wish their data to be processed. However, even if citizens do object to receiving political messages, they could still be profiled on the basis of their political opinions, philosophical beliefs or other special categories of personal data that fall under the GDPR”. The European Commission was also urged to investigate the RomanianGDPR implementation for similar concerns. Further to the reported challenges with national adaptation of GDPR, in November 2019 the EDPS has issued the first ever reprimand to an EU institution. The ongoing investigation into the European Parliament was prompted by the Parliament’s use of a US-based political campaigning company NationBuilder to process personal data as part of its activities relating to the 2019 EU elections.

Combating disinformation

In contrast to the GDPR, which is sometimes praised as “the most consequential regulatory development in information policy in a generation” (Hoofnagle et al., 2019, p. 66), the EC has decided to tackle fake news and disinformation through self-regulation, at least in the first round. The European Council, a body composed of the leaders of the EU member states, first recognised the threat of online disinformation campaigns in 2015 when it asked the High Representative of the Union for Foreign Affairs and Security Policy to address the disinformation campaigns by Russia (EC, 2018c). The Council is not one of the EU's legislating institutions, but it defines the Union’s overall political direction and priorities. So, it comes as no surprise that the issue of disinformation came high on the agenda of the EU, in particular after the UK referendum and US presidential elections in 2016. In April 2018 the EC (2018a) adopted a Communication on Tackling online disinformation: a European Approach. This is the central document that set the tone for future actions in this field. In the process of its drafting, the EC carried out consultations with experts and stakeholders, and used citizens’ opinions gathered through polling. The consultations included the establishment of a High-Level Expert Group on Fake News and Online Disinformation (HLEG) in early 2018, which two months later produced a Report (HLEG, 2018) advising the EC against simplistic solutions. Broader public consultations and dialogues with relevant stakeholders were also held, and the specific Eurobarometer (2018b) poll was conducted via telephone interviews in all EU member states. The findings indicated a high level of concern among the respondents for the spread of online disinformation in their country (85%) and saw it as a risk for democracy in general (83%). This urged the EC to act and the Communication on tackling online disinformation was a starting point and the key document in understanding the European approach to the pressing challenges. The Communication builds around four overarching principles and objectives: transparency, diversity of information, credibility of information, and cooperation (EC, 2018a).

Transparency, in this view, means that it should be clear to users where the information comes from, who the author is and why they see certain content when an automated recommendation system is being employed. Furthermore, a clearer distinction between sponsored and informative content should be made and it should be clearly indicated who paid for the advertisement. The diversity principle is strongly related to strengthening so-called quality journalism, 4 to rebalancing the disproportionate power relations between media and social media platforms, and to increasing media literacy levels. The credibility, according to the EC, is to be achieved by entrusting platforms to design and implement a system that would provide an indication of the source and information trustworthiness. The fourth principle emphasises cooperation between authorities at national and transnational level and cooperation of broad stakeholders in proposing solutions to the emerging challenges. With an exception of emphasising media literacy and promoting cooperation networks of authorities, the Communication largely recommends that platforms design solutions which would reduce the reach of manipulative content and disinformation, and increase the visibility of trustworthy, diverse and credible content.

The key output of this Communication is a self-regulatory Code of Practice on Online Disinformation (CoP). The document was drafted by the working group composed of online platforms, advertisers and the advertising industry, and was reviewed by the Sounding Board, composed of academics, media and civil society organisations. The CoP was agreed by the online platforms Facebook, Google and Twitter, Mozilla, and by advertisers and the advertising industry, and was presented to the EC in October 2018. The Sounding Board (2018), however, presented a critical view on its content and the commitments laid out by the platforms, stating that it “contains no clear and meaningful commitments, no measurable objectives” and “no compliance or enforcement tool”. The CoP, as explained by the Commission, represents a transitional measure where private actors are entrusted to increase transparency and credibility of the online information environment. Depending on the evaluation of their performance in the first 12 months, the EC is supposed to determine the further steps, including the possibility of self-regulation being replaced with regulation (EC, 2018c). The overall assessment of the Code’s effectiveness is expected to be presented in early 2020.

The CoP builds on the principles expressed in the Commission’s Communication (2018a) through the actions listed in Table 1. For the purpose of this paper the actions are not presented in the same way as in the CoP. THey are instead slightly reorganised under the following three categories: Disinformation; Political advertising, Issue-based advertising.

In the statement on the first annual self-assessment reports by the signatories of the CoP, the Commission acknowledged that some progress has been achieved, but warns that it “varies a lot between signatories and the reports provide little insight on the actual impact of the self-regulatory measures taken over the past year as well as mechanisms for independent scrutiny”. The European Regulators Group for Audiovisual Media Services (ERGA) has been supporting the EC in monitoring the implementation of the commitments made by Google, Facebook and Twitter under the CoP, particularly in the area of political and issue-based advertising. In June 2019 ERGA released an interim Report as a result of the monitoring activities carried out in 13 EU countries, based on the information reported by platforms and on the data available in their online archives of political advertising. While it stated “that Google, Twitter and Facebook made evident progress in the implementation of the Code’s commitments by creating an ad hoc procedure for the identification of political ads and of their sponsors and by making their online repository of relevant ads publicly available”, it also emphasised that the platforms have not met a request to provide access to the overall database of advertising for the monitored period, which “was a significant constraint on the monitoring process and emerging conclusions” (ERGA, 2019, p. 3). Furthermore, based on the analysis of the information provided in the platforms’ repositories of political advertising (e.g., Ad Library), the information was “not complete and that not all the political advertising carried on the platforms was correctly labelled as such” (ERGA, 2019, p. 3).

The EC still needs to provide a comprehensive assessment on the implementation of the commitments under the CoP after an initial 12-month period. However, it is already clear that the issue of the lack of transparency of the platforms’ internal operations and decision-making processes remains and represents a risk. If platforms are not amenable to thorough public auditing, then the adequate assessment of the effectiveness of implementation when it comes to self-regulation becomes impossible. The ERGA Report (2019) further warns that at this point it is not clear what options for micro-targeting were offered to political advertisements nor if all options are disclosed in the publicly available repositories of political advertising.

Further to the commitments laid down in the CoP and relying on social media platforms to increase transparency of political advertising online, the Commission Recommendation of 9 September 2018 (EC, 2018e), “encourages”, and asks member states to “encourage” further transparency commitments by European and national political parties and foundations, in particular:

information on the political party, political campaign or political support group behind paid online political advertisements and communications” [...] “information on any targeting criteria used in the dissemination of such advertisements and communications” [...] “make available on their websites information on their expenditure for online activities, including paid online political advertisements and communications (EC, 2018e, p. 8).

The Recommendation (EC, 2018e) further advises member states to set up a national election network, involving national authorities with competence for electoral matters, including data protection commissioners, electoral authorities and audio-visual media regulators. This recommendation is further elaborated in the Action plan (EC, 2018c) but, because of practical obstacles, national cooperation between authorities has not yet become a reality in many EU countries.

Key principles and shortcomings of the European approach

This analysis has shown that the principles contained in the above mentioned instruments, which form the basis of the European approach to combating disinformation and political manipulation are: data protection; transparency; cooperation; mobilising the private sector; promoting diversity and credibility of information; raising awareness; empowering the research community.

Data protection and transparency principles related to personal data collection, processing and use are contained in the GDPR. The requirement to increase transparency of political and issues-based advertising and of automated communication is currently directed primarily towards platforms that have committed themselves to label and publicly disclose sponsors and content of political and issues-based advertising, as well as to identify and label automated accounts. Unlike with the traditional media landscapes where, in general, on the same territory, media were broadcasting the same political advertising and messages to their audiences, in the digital information environment political messages are being targeted and shown only to specific profiles of voters with limited ability to track them to see which messages were targeted to whom. To increase transparency on this level would require platforms to provide a user-friendly repository of political ads, including searchable information on actual sponsors and amounts spent. At the moment, they struggle with how to identify political and issue-based ads, to distinguish them from other types of advertising, and to verify ad buyers’ identities (Leerssen et al., 2019).

Furthermore, the European approach fails to impose similar transparency requirements towards political parties to provide searchable and easy to navigate repositories of the campaign materials used. The research project of campaign monitoring during the 2019 European elections, showed that parties/groups/candidates participating in the elections were largely not transparent about their campaign materials. Materials were not readily available on their websites or social media accounts nor did they respond to direct requests from researchers (Simunjak et al., 2019). This warns that while it is relevant to require platforms to provide more transparency on political advertising, it is perhaps even more relevant to demand this transparency directly from political parties and candidates in elections.

In the framework of transparency, the European approach also fails to further emphasise the need for political parties to declare officially to authorities and under a specific category the amounts spent for digital (including social media) campaigning. At present, in some EU countries (for example Croatia, see: Klaric, 2019), authorities with competences in electoral matters do not consider social media as media and accordingly do not apply the requirements to report spending on social media and other digital platforms in a transparent manner. This represents a risk, as the monitoring of the latest EP elections has clearly showed that the parties had spent both extensive time and resources on their social media accounts (Novelli & Johansson, 2019).

The diversity and credibility principles stipulated in the Communication on tackling online disinformation and in the Action plan ask from platforms to indicate the information trustworthiness, to label automated accounts, to close down fake accounts, and to prioritise quality journalism. At the same time, clear definition or instructions on criteria to determine whether an information or a source is trustworthy and whether it represents quality journalism is not provided. Entrusting platforms with making these choices without the possibility of auditing their algorithms and decision-making processes represents a potential risk for freedom of expression.

The signatories of the CoP have committed themselves to disrupt advertising and monetisation incentives for accounts and websites, which consistently misrepresent information about themselves. But, what about accounts that provide accurate information about themselves but occasionally engage in campaigns which might also contain disinformation? For example, a political party may use data to profile and target individual voters or a small group of voters with messages that are not completely false but are exaggerated, taken out of context or framed with an intention to deceive and influence voters’ behaviour. As already noted, disinformation comes in many different forms, including false context, imposter, manipulated or fabricated content (Wardle & Derakhshan, 2017). While the work of fact-checkers and flagging of false content are not completely useless here, in the current state of play this is far from sufficient to tackle the problems of disinformation, including in political advertising and especially of dark ads 5. The efficiency of online micro-targeting depends largely on data and profiling. Therefore, if effectively implemented, the GDPR should be of use here by preventing the unlawful processing of personal data.

Another important aspect of the European approach are stronger sanctions in cases when the rules are not respected. This entails increased powers of authorities, first and foremost of DPAs and increased fines under the GDPR. Data protection in the electoral context is difficult to ensure if the cooperation between different authorities with competence for electoral matters (such as data protection commissioners, electoral authorities and audio-visual media regulators) is not established and operational. While the European approach strongly recommends cooperation, it is not easily achievable at a member state level, as it requires significant investments in capacity building and providing channels for cooperation. In some cases, it may even require amendments to the legislative framework. The cooperation of regulators of the same type at the EU level is sometimes hampered by the fact that their competences differ in different member states.

The CoP also contains a commitment on “empowering the research community”. This means that the CoP signatories commit themselves to support research on disinformation and political advertising by providing researchers access to data sets, or collaborating with academics and civil society organisations in other ways. However, the CoP does not specify how this cooperation should work, the procedures for granting access and for what kind of data, or which measures should researchers put in place to ensure appropriate data storage, security and protection. In the reflection on the platform’s progress under the Code, three Commissioners warned that the “access to data provided so far still does not correspond to the needs of independent researchers”.

Conclusions

This paper has given an overview of the developing European approach to combating disinformation and political manipulation during an electoral period. It provided an analysis of the key instruments contained in the approach and drew out the key principles upon which it builds: data protection; transparency; cooperation; mobilising the private sector; promoting diversity and credibility of information; raising awareness; empowering the research community.

The principles of legacy media regulation in the electoral period are impartiality and equality of opportunity for contenders. This entails balanced and non-partisan reporting as well as equal or proportionate access to media for political parties (be it free or paid-for). If political advertising is allowed, it is usually subject to transparency and equal conditions requirements: how much was spent on advertising in the campaign needs to be presented through spending on different types of media and reported to the competent authorities. The regulatory framework requires that political advertising be properly labelled as such.

In the online environment, the principles applied to legacy media require further elaboration as the problem of electoral disinformation cuts across a number of different policy areas, involving a range of public and private actors. Political disinformation is not a problem that can easily be compartmentalised into existing legal and policy categories. It is a complex and multi-layered issue that requires a more comprehensive and collaborative approach when designing potential solutions. The emerging EU approach reflects the necessity for that overall policy coordination.

The main fuel of online political campaigning is data. Therefore, the protection of personal data and especially of “sensitive” data from abuse becomes a priority of any action that aims to ensure free, fair and plural elections. The European approach further highlights the importance of transparency. It calls on platforms to clearly identify political advertisements and who paid for them, but it fails to emphasise the importance of having a repository of all the material used in the campaign provided by candidates and political parties. Furthermore, a stronger requirement for political parties to report on the amounts spent on different types of communication channels (including legacy, digital and social media) is lacking in this approach, as well as the requirement for platforms to provide more comprehensive and workable data on sponsors and spending in political advertising.

The European Commission’s communication of the European approach claims that it aims to address all actors active in the electoral context, including European and national political parties and foundations, online platforms, data analytics companies and public authorities responsible for the electoral process. However, it seems that the current focus is primarily on the platforms and in a way that enables them to shape the future direction of actions in the fight against disinformation and political manipulation.

As regards the principle of cooperation, many obstacles, such as differences in competences and capacities of the relevant national authorities, have not been fully taken into account. The elections are primarily a national matter so the protection of the electoral process, as well as the protection of media pluralism, falls primarily within the competence of member states. Yet, if the approach to tackling disinformation and political manipulation is to be truly European, there should be more harmonisation between authorities and approaches taken at national levels.

While being a significant step in the creation of a common EU answer to the challenges of disinformation and political manipulation, especially during elections, the European approach requires further elaboration, primarily to include additional layers of transparency. This entails transparency of political parties and of other actors on their actions in the election campaigns, as well as more transparency about internal processes and decision-making by platforms especially on actions of relevance to pluralism, elections and democracy. Furthermore, the attempt to propose solutions and relevant actions at the European level faces two constraints. On the one hand, it faces the power of global platforms shaped in the US tradition, which to a significant extent differs from the European approach in balancing freedom of expression and data protection. On the other hand, the EU approach confronts the resilience of national political traditions in member states, in particular if the measures are based on recommendations and other soft instruments.

Introduction

Shortly after Donald Trump won the US presidency, Jim Gilliam (2016), the late president of start-up 3DNA, posted a message on its blog titled “Choosing to Lead”. Gilliam congratulated the “three thousand NationBuilder customers who were on the ballot last week”. These customers subscribed to 3DNA’s NationBuilder service that provides a political engagement platform connecting voters, politicians, volunteers and staffers in an integrated online service. The post continues:

Many of you – including President-elect Donald Trump and all three of the other non-establishment presidential candidates – were outsiders. And that’s why this election was so important. Not just for people in the United States, but for people all over the world. This election unequivocally proves that we are in a new era. One where anyone can run and win. (Gilliam, 2016)

Like many posts from NationBuilder, Gilliam celebrated the company’s mission to democratise access to new political technology, bringing in these outsiders.

Gilliam’s post demonstrates faith that being open is a corporate value as well as a business model. As its mission states today, NationBuilder sells “to everyone regardless of race, age, class, religion, educational background, ideology, gender, sexual orientation or party”. Their mission encapsulates a corporate belief in the democratic potential of their product, one available to anyone, much to the frustration of partisans and other political insiders on both sides who tend to guard access to their innovative technologies (Karpf, 2016b).

Gilliam’s optimism matters globally. Political parties worldwide use NationBuilder as a third-party solution to manage its voter data, outreach, website, communications and volunteer management. As of 3 December 2019, NationBuilder reported that in 2018 it was used to send 1,600,000,000 emails, host 341,000 events and raise $401,000,000 USD across 80 countries. The firm has also raised over $14 million US dollars in venture capital partially based on the promise that it will democratise access to political engagement platforms. Unlike most of its competitors, NationBuilder is a nonpartisan political engagement platform. NationBuilder is one of the few services actively developed and promoted as nonpartisan and cross-sectoral. Conservative, liberal and social democratic parties across the globe use NationBuilder, as the company emphasises in its corporate materials (McKelvey and Piebiak, 2018).

By letting outsiders access political technology, might NationBuilder harm politics in its attempts to democratise it? Now is the time to doubt the promise of political technologies. Platform service providers like NationBuilder are the object of significant democratic anxieties globally, rightly or wrongly (see Adams et al., 2019, for a good review of current research). The political technology industry has been pulled into a broad set of issues including, according to Colin Bennett and Smith Oduro-Marfo: “the role of voter analytics in modern elections; the democratic responsibilities of powerful social media platforms; the accountability and transparency for targeted political ads; cyberthreats to the through malicious actors and automated bots” (2019, pp. 1-2). Following the disclosures of malpractice by Cambridge Analytica and AggregateIQ, these public scandals have pushed historic concerns about voter surveillance, non-consensual data collection and poor oversight of the industry to the fore (Bennett, 2015; Howard, 2006; White, 1961).

My paper questions NationBuilder’s corporate belief that better access to political technology improves politics. In doing so, I add acceptable use of political technology to the list of concerns about elections and campaigns in the digital age. Even as Daniel Kreiss (2016) argues, campaigns are technologically-intensive, there have been no systematic studies of how a political technology is used, particularly internationally. My paper reviews the uses of NationBuilder worldwide. It offers empirical research to understand the real world of a contentious political technology and offers grounded examples of problematic or questionable uses of a political technology. NationBuilder is a significant example, as I discuss, of a nonpartisan political technology firm as opposed to its partisan rivals.

The paper uses a mixed method approach to analyse NationBuilder’s use. Methods included document analysis, content analysis and a novel use of web analytics. To first understand real world use, the study collected a list of 6,435 domains using NationBuilder as of October 2017. The study coded the 125 most popular domains by industry and compared results to corporate promotional materials, looking for how actual use differed from its promoted uses. The goal was to find questionable uses of NationBuilder. Questionable, through induction, came to mean uses that might violate liberal democratic norms. By looking at NationBuilder’s various uses, the review found cases at odds with normative and institutional constraints that allow for ‘friendly rivalry’ or ‘agonism’ in liberal democratic politics (Rosenblum, 2008; Mouffe, 2005). These constraints include a free press, individual rights such as privacy as well as a commitment to shared human dignity.

My limited study finds that NationBuilder can be used to undermine privacy rights and journalistic standards while also promoting hatred. The scan identified three problematic uses as: (1) a mobilisation tool for hate groups targeting cultural or ethnic identities; (2) a profiling tool for deceptive advertising or stealth media and; (3) a fundraising tool for entrepreneurial journalism. These findings raise issues about acceptable use and liberal democracy. For example, I looked for cases of NationBuilder being used by known hate groups inspired by recent concerns about the rise of the extreme right (Eatwell and Mudde, 2004) as well as the use of NationBuilder by news websites reflecting the changing media system (Ananny, 2018).

My findings suggest that NationBuilder may be a democratic technology, without being a liberal one. The traditions of liberalism and democracy are separate and a source of tension according to democratic theorist Chantal Mouffe. “By constantly challenging the relations of inclusion implied by the political constitution of 'the people' - required by the exercise of democracy”, Mouffe writes, “the liberal discourse of universal human rights plays an important role in maintaining the democratic contestation alive” (2009, p. 10). NationBuilder’s democratic mission of being open to outsiders then is at odds with a liberal tradition that pushes fraud, violence and hatred outside respectable politics.

While the paper identifies problems, it does not offer much in the way of solutions. Remedies are difficult and certainly not at NationBuilder’s global scale. As I discuss later, NationBuilder is not responsible for how it is used. The most immediate remedies might be based on corporate social responsibility. To this end, this paper provides three recommendations for revisions to 3DNA’s acceptable use policy to address these questionable uses: (1) reconcile its mission statement with its prohibited uses; (2) require disclosure on customers’ websites; and (3) clarify its relation to domestic privacy law as part of a corporate mission to improve global privacy and data standards. These reforms suggest that NationBuilder’s commitment to non-partisanship needs clarification and that the acceptable use of political technology is fraught – a dilemma that should become a central debate. Political technology firms – NationBuilder and its competitors – must understand that liberal democratic technologies are part of what Bennett and Oduro-Marfo describe as “the political campaigning network”. They continue, “contemporary political campaigning is complex, opaque and involves a shifting ecosystem of actors and organisations, which can vary considerably from society to society” (2019, p. 54). Companies ultimately must consider their obligations to liberal democracy, a political system made possible by technologies like the press and the internet (albeit imperfectly).

The acceptable use of politicised, partisan and nonpartisan technology

The political technology industry is central to the era of technology-intensive campaigning found in the United States and across many Western democracies (Baldwin-Philippi, 2015; Karpf, 2016a; Kreiss, 2016). The industry itself has been a staple of political consultancy throughout modern campaigning. From laser letters for direct mail to apps for canvassing, political technology firms promise to bring efficiency to an otherwise messy campaign (D. W. Johnson, 2016; Kreiss and Jasinski, 2016). NationBuilder itself provides a good summary of this industry in a marketing slide reproduced in Figure 1.

Figure 1: Political technology firms according to NationBuilder

The figure illustrates the numerous practices and sectors drawn into politics as well as the migration of practices. These services help campaigns analyse data and make strategic decisions, principally around advertising buys. Many of these firms position themselves as the primary medium of a campaign, creating a platform connecting voters, politicians, volunteers and staff (Baldwin-Philippi, 2017; McKelvey and Piebiak, 2018). Political technology providers blur the boundaries between nonprofit management, political campaigning and advocacy as well as illustrating the taken-for-grantedness of marketing as a political logic (Marland, 2016).

Political technology firms may be divided between: politicised firms, partisan firms, and nonpartisan firms. Politicised firms sell software or services not explicitly designed for politics put to political ends. These can include payment processors like PayPal or Stripe, web hosting companies like Cloudflare and social media platforms that allow political advertising and political mobilisation. NationBuilder’s slide reproduced in Figure 1 includes some further examples of politicised firms providing social media management software, email marketing software and website content management systems. Technologies like NationBuilder are purpose-built for politics, listed as Political Software in Figure 1. These firms can be split further between partisan firms that work only for conservative, liberal or progressive campaigns and nonpartisan firms. In a market dominated by partisan affiliation, NationBuilder and other nonpartisan companies like Aristotle International and ActionKit are significant. They attempt to be apolitical political technologies.

Political technologies raise added concerns in respect to liberal democratic norms. Who should have access to these services, and how should these services be used? New technologies afford campaigns new repertoires of action that may undermine campaign spending limits, norms around targeting or the privacy rights of voters. Cambridge Analytica, for example, has rekindled longstanding debates about the democratic consequences of political technologies especially micro-targeting (Bodó, Helberger, and de Vreese, 2017; Kreiss, 2017) as well as stoking conjecture about the feasibility of psycho-demographics and its mythic promise of a new hypodermic needle (Stark, 2018).

Acceptable use is largely determined by partisan identity due to the limited scope of regulations on digital campaigning. Regulation for political technology is lacking (Bennett, 2015; Howard and Kreiss, 2010) and likely does not apply to a service provider like NationBuilder in the first place. Instead, so far partisanship has been regarded as the key mechanism to regulate the use of political technology. Most firms are partisan, working with only one party. Acceptable use of political technology is largely judged by its conformity to partisan values. As David Karpf explains, “political technology yields partisan benefits, and the market for political technologies is made up of partisans” (2016b, p. 209). Such partisanship functions as a professional norm about acceptable use, restricting access on partisan lines. Fellow partisans are acceptable, and, in what Karpf calls the zero-sum game of politics, rivals are unacceptable users. Indeed, partisanship is an important corporate asset. The major firm Aristotle International sued its competitor NGP VAN for falsely claiming it only sold to Democratic and progressive firms when it licensed its technologies to Republican firms as well. NGP VAN, the case alleged, was not as adherent a partisan firm as it claimed. The courts eventually dismissed the case (D’Aprile, 2011).

The tensions between partisan versus nonpartisan and politicised companies implicitly reveal a split in the values guiding acceptable use. On one side are firms committed to creating technology to advance their political values while on the other are firms trying to be neutral and to sell to anyone. In what might be seen as an act of community governance, progressive partisans argued that the software should not sell to non-progressive campaigns (Karpf, 2016a).

The lack of an expressed political agenda has caused politicised firms, in particular, to be mired in public scandals raising questions involving liberal democratic norms. A ProPublica investigation found that numerous technology firms supported known extremist groups, prompting Paypal and Plasso to cease offering services to groups identified days later (Angwin, Larson, Varner, and Kirchner, 2017a). That investigation only scratches the surface. A partial list of recent media controversies includes politicised firms being accused of spreading misinformation, aiding hate groups and easing foreign propaganda:

• Facebook’s handling of the Kremlin-affiliated Internet Research Agency misinformation campaigns during the 2016 presidential elections
• Hosting service Cloudflare removing Stormfront (Price, 2017)
• GoFundMe allowing a fraudulent campaign to build a US-Mexico border wall (Holcombe, 2019)
• GoFundMe removing anti-vaccine fundraising campaigns (Liao, 2019)
• YouTube’s handling of far-right videos and the circulation of the livestream of the Christchurch terrorist attack

In the academic literature, McGregor and Kreiss (2018) question the willingness of politicised firms to assist American presidential campaigns’ advertising strategies, examining how these companies understood their influence. Braun and Eklund (2019) meanwhile explore the digital advertiser's dilemma of trying to demonetise misinformation and imposter journalism. 1 The CitizenLab has addressed the responsibility of international cybersecurity firms in democratic politics, particularly the use of exploits to target dissidents. 2 Tusikov (2019) most directly explores the question of acceptable use by analysing how financial third parties, like PayPal, have developed their own internal policies to not serve hate groups.

For these reasons, NationBuilder is an important test case for the acceptable uses of political technology. NationBuilder, as discussed above, exemplifies the neutral position of many firms, trying to be in politics without being political. NationBuilder exemplifies the problem for both politicised and nonpartisan firms that let their commitments to openness and neutrality to supersede their responsibilities to be political and understand their responsibility to liberal democracy norms.

Why NationBuilder?

NationBuilder is an intriguing case because it encapsulates a particular American belief in the revolutionary promise of computing for politics that has driven the development and regulation of many major technology firms (Gillespie, 2018; Mosco, 2004; Roberts, 2019). NationBuilder is a venture capital–funded company promising to disrupt politics by democratising access to innovation. According to investor Ben Horowitz (2012), “NationBuilder is that rarest of products that not only has the potential to change its market, but to change the world”. He made these remarks in a 2012 post in which Horowitz’s firm announced $6.25 million USD in Series A funding for NationBuilder’s parent company 3DNA. NationBuilder’s late founder Jim Gilliam exemplifies the “romantic individualism” that Tom Streeter associates with a faith in the thrilling, revolutionary effect of computing. Gilliam was a fundamental Christian who found community through BBSs and eventually told his coming-of-age story in a viral video entitled “The Internet Is My Religion”. He later self-published a book co-authored with the company’s current president, Lea Endres. When generalised and situated as part of NationBuilder’s mission, Gilliam’s story exemplifies Streeter’s observation that “the libertarian’s notion of individuality is proudly abstracted from history, from social differences, and from bodies; all that is supposed not to matter. Both the utilitarian and romantic individualist forms of selfhood rely on creation-from-nowhere assumptions, from structures of understanding that are systematically blind to the collective and historical conditions underlying new ideas, new technologies, and new wealth” (Streeter, 2011, p. 24). NationBuilder still links to this video on its corporate philosophy page as of 3 December 2019.

Figure 2: NationBuilder’s philosophy page captured on 8 January 2020

NationBuilder’s mission synthesises its belief in for-profit social change and romantic individualism. According to NationBuilder’s mission page as of 7 January 2020, it wants to “build the infrastructure for a world of creators by helping leaders develop and organise thriving communities”. This includes a belief that: “The tools of leadership should be available to everyone. NationBuilder does not discriminate. It is arrogant, even absurd, for us to decide which leaders are “better” or “right” (NationBuilder, n.d.).

Their mission resembles Streeter’s discussion of the libertarian abstract sense of freedom that, in NationBuilder’s case, equates egalitarian access to a commercial service with a viable means for democratic reform. Whether nonpartisan or libertarian, NationBuilder has remained committed to this belief, defending its openness from critics, such as in Gilliam’s post from the introduction. In doing so, NationBuilder is at odds with former progressive clients and other political technology firms (Karpf, 2016b).

Methodology

My research combines document analysis, web analytics and content analysis to understand NationBuilder usage. The research team reviewed the company’s 2016, 2017 and 2018 annual reports and archived content from the NationBuilder website using the Wayback Machine. The team also turned to the web services tool BuiltWith. BuiltWith scans the million most-popular sites on the internet to detect what technologies they use. 3 BuiltWith generated a list of 6,435 web domains using NationBuilder on 10 October 2017. Research analysed BuiltWith’s data through two scans:

1. Coding the top 125 websites (as ranked by Alexa, an Amazon company that estimates traffic on major websites) by industry and comparing the results with the publicised use cases in NationBuilder’s annual reports.
2. Searching the full list of BuiltWith results for websites classified as extremist by ProPublica, itself informed by the Anti-Defamation League and the Southern Poverty Law Center (Angwin, Larson, Varner, and Kirchner, 2017b).

These methods admittedly offer a limited window into the use of NationBuilder. Rather than provide a complete scan of the NationBuilder ecosystem or track trends over time, this project sought to question whether NationBuilder has uses other than those advertised, and, if so, do these applications raise acceptability questions?

The coding schema classified uses of NationBuilder by industry. The schema developed out of a review of prior literature classifying websites (Elmer, Langlois, and McKelvey, 2012) as well as inductive coding developed by visiting the top fifty websites, paying special attention to self-descriptions, such as mission statements and “about us” sections, as well as other clues to a site’s legal status (as a non-profit or a political action committee) or its overt political party affiliation and stated political positions. In the end, each website in the sample was assigned one of ten codes:

1. College or university: a higher education institution
2. Cultural production: a site promoting a book, movie, etc.
3. Educational organisation: a high school or below
4. Government initiative: sites operated by incumbent political actors or elected officials that are explicitly tied to their work in government (i.e., not used for a re-election campaign)
5. Media organisation: sites whose primary purpose is to publish or aggregate media content
6. NGO: (non-governmental organisation) sites for organisations whose activities can reasonably be considered non-political; these are usually but not exclusively non-profits
7. Other: sites that are unclassifiable (an individual’s blog, for example)
8. Political advocacy group: organisations that are not directly associated with an official political party or campaign but nonetheless seek to actively affect the political process
9. Political party or campaign: sites operated by a political party or dedicated to an individual politician’s electoral campaign
10. Union: sites run by a labour union

Two independent coders classified the 125-website sample. Intercoder reliability was 88 percent with a Krippendorf’s alpha of 0.8425 (Freelon, 2010). Analysis below removed inconsistencies through consensus coding.

Findings

NationBuilder has applications not well presented in its corporate materials that raise acceptability issues. NationBuilder has been used as:

1. a mobilisation tool for hate or groups targeting cultural or ethnic identities;
2. a profiling tool for deceptive advertising or stealth media; and,
3. a fundraising tool for entrepreneurial journalism.

None of these uses violate the official terms of use or acceptable use policy, a problem discussed later in the analysis, but they do provoke questions that may help improve its acceptable usage policies.

Results of scan 1: top industries found in most popular sites in the sample

The first scan, coding top domains by industry, found uses that differed from the corporate reporting. NationBuilder emphasises certain use cases in its annual report and marketing, signalling the authorised channels of circulation for the product as well as its popular applications. Reporting, however, has been inconsistent with the best data available from 2016. The 2016 Annual Report lists the following uses: political (40.80%), advocacy (24.60%), nonprofit (11.80%), higher education (11%), business (8.30%), association (2%), as well as government (1.50%). 4 NationBuilder also profiles “stand-out leaders” in all its annual reports. Politicians, advocacy groups and nonprofits mostly appear in the list. The 2017 list features six politicians out of ten slots, including the party of French President Emmanuel Macron, New Zealand's Prime Minister Jacinda Ardern, and the leader of Canada's New Democratic Party Jagmeet Singh. Their successful campaigns resonate with NationBuilder's brand of political inclusion. In a new twist on the politics of marketing, NationBuilder also profiles businesses as stand-outs. AllSaints is a British fashion retailer that uses NationBuilder to connect with fans of the brand, especially to announce the opening of new stores.

Media outlets are more prominent in the findings than in 3DNA’s corporate materials. Two media outlets are in the top ten domains in our sample sorted by popularity as seen in Table 1. The third and fourth ranked sites are media organisations. Faith Family America is a right-of-centre news outlet, describing itself as “a real-time, social media community of Americans who are passionate about faith, family, and freedom”. The Rebel is a Canadian-based far-right news outlet, comparable to Breitbart in the US. Seven other media organisations appear in the sample, nine in total as seen in Table 2.

Two of the questionable uses of NationBuilder relate to its move into journalism or at least the simulacra of journalism. Through these media outlets, NationBuilder becomes entangled in the ethics of entrepreneurial journalism. The term refers to the “embrace of entrepreneurialism by the world of journalism” (Rafter, 2016, p. 141).

Otherwise, findings resembled data from the 2016 annual report. Political, advocacy and nonprofits accounted for 77.2 % of NationBuilder’s customers in the annual report whereas non-governmental organisations, political advocacy groups, political party or campaign and union comprised 83.2% in the sample. Unlike the annual reports, the sample included nine media-based organisations out of the 125 sites, representing 7.2% of the findings. Other users were marginal. There was a curious absence of any brand ambassadors even though NationBuilder highlights these applications prominently in its annual reports and describes 1% of its customers as such in its 2017 report.

Results of scan 2: extremists or hate groups using NationBuilder

The second scan found one use case by a known hate group as defined by the Southern Poverty Law Center, Act for America (ranked 72nd in sample). The Southern Poverty Law Center describes the group as the “largest anti-Muslim group in America”. Act for America used NationBuilder until August 2018 when it switched to an open-source equivalent, Drupal and CiviCRM (cf. McKelvey, 2011). Act for America did not state the reason for the switch or reply to questions.

Covert political organising?

Three media outlets stood out in the sample: Faith Family America, Inspiring Day and All Cute All the Time. Each site used attention-grabbing headlines (also known as clickbait) to present curated news, updates about the British monarchy, and celebrity news that was respectively conservative, religious and innocuous (rather than cute). None of these sites listed staff in a masthead or provided many details about their reporting; instead, the sites encouraged users to join the community and promoted their Facebook groups.

Figure 4: Faith Family America’s front page, capture 23 April 2019

All three outlets were owned by the company Strategic Media 21 (SM21) – a fact that was only apparent through examining the site’s identical privacy policies. Now offline, SM21 was based in San Jose, California. It seems to have been a digital marketing firm with two different web presences: one for content marketing and one for digital strategy. Neither site discloses much information about the company, but their business strategy seems to be manufacturing audiences for political advertisers. SM21 identifies demographics, then creates specific outlets, like Faith Family America for conservative voters, in the hope of building up a dedicated audience for advertising. Data broker L2 blogged about their 2016 partnership with SM21 on a targeted Facebook political advertising campaign. In this case, SM21 was acting in its digital strategy role, working with clients “on messaging, creative, plans out the buy and launches the campaign using your targeted list” (Westcott, 2016). These services have proved valuable. SM21 has received $2,418,592 USD in political expenditures since 2014 according to OpenSecrets. The biggest clients were the conservative Super PACs (political action committees) Vote to Reduce Debt, and Future in America.

Strategic Media 21 raises suspicions that NationBuilder’s data analytics might be used covertly, a kind of native advertising without the journalism. This might be an application of what Daniels calls cloaked websites “published by individuals or groups that conceal authorship or feign legitimacy in order to deliberately disguise a hidden political agenda” (2009, p. 661). Kim et al. describe similar tactics as stealth media, “a system that enables the deliberate operations of political campaigns with undisclosed sponsors/sources, furtive messaging of divisive issues, and imperceptible targeting” (2018, p. 2). By building these niche websites and corresponding Facebook groups that crosspost their content, SM21 has created a political advertising business. NationBuilder features might assist in this business; its Match feature connects email addresses with other social media accounts, and its Political Capital feature monitors these feeds for certain activities.

Suspicions that Strategic Media 21 used NationBuilder for its data mining features are likely true. According to emails released as part of a suit filed against Facebook by the Office of the Attorney General for the District of Columbia, Facebook employees discussed Cambridge Analytica, NationBuilder and SM21 as all being in violation of its data sharing arrangements (Wong, 2019). As one internal document dated 22 September 2015 explains,

One vendor offering beyond [Cambridge Analytica] we're concerned with (given their prominence in the industry ) is NationBuilder’s “Social Matching,” on which they've pitched our clients and their website simply says “Automatically link the emails in your database to Facebook, Twitter, Linkedin and Klout profiles, and pull in social engagement activity.” I'm not sure what that means, and don't want to incorrectly tell folks to avoid it, but it is definitely being conflated in the market with other less above board services. Can you help clarify what they're actually doing?

Employees worried that “these apps’ data-scraping activity [were] likely non-compliant” according to a reply dated 30 September 2015 and the thread actively debated the matter for months. Facebook employees singled out SM21 in a comment on 20 October 2015. It begins,

thanks for confirming this seems in violation. [REDACTED] mentioned there is a lot of confusion in the political space about how people use Facebook to connect with other offline sets of data. In particular, Strategic Media 21 has been exerting a good deal of pressure on one of our clients to take advantage of this type of appending.

These concerns ensued even as Facebook employees reacted to a Guardian article on 11 December 2015 entitled “Ted Cruz using firm that harvested data on millions of unwitting Facebook users” – one of the first stories to develop in the ongoing scandal involving Cambridge Analytica and Facebook data sharing (Davies, 2015). What ultimately happened to NationBuilder and Strategic Media 21 has not been disclosed to date. NationBuilder still advertises its social matching features. SM21, on the other hand, has gone offline, with its website available for purchase as of September 2019.

This evidence raises our first problem of acceptable use: should NationBuilder be used by covert or stealth media to enable the deceptive or non-consensual collection of data? Strategic Media 21 then parallels Cambridge Analytica where users unwittingly trained its profiles by filling out quizzes on Facebook (Cadwalladr and Graham-Harrison, 2018). Visiting websites running Strategic Media 21 and joining related groups might unwittingly inform advertising profiles harvested through NationBuilder. This is a serious privacy harm noted by a UK Information Commissioner’s Office (2018) report and an Information and Privacy Commissioner for British Columbia (2019) report that both raised the issue of social matching in their own reports on NationBuilder.

Advocacy, journalism or outrage?

NationBuilder has become entangled in the ethics of entrepreneurial journalism and the boundaries between editorial and fundraising through The Rebel, its Australian-affiliate Mark Latham’s Outsiders, and to a lesser extent the Newshounds (Hunter, 2016; Porlezza and Splendore, 2016). All sites rely on crowdsourcing, reminding their readers that they need financial support. Newshounds.us is a media watchdog blog covering Fox News that asks its visitors to donate to support its coverage. The Rebel is a Canadian news start-up, established at the closure of Sun News TV or what was called Fox News North. While start-ups, these outlets position themselves as journalism outlets. Newshounds makes mention of its editor’s journalism degree. The Rebel asks its visitors to subscribe and to help support its journalism.

The line between fundraising and journalism is a clear ethical concern for journalism. As Porlezza and Splendore note in a thoughtful review of accountability and transparency issues in entrepreneurial journalism, the industry has to deal with a challenge “that touches the ethical core of journalism: are journalists in start-ups able to distinguish between their different and overlapping goals of publisher, fundraiser and journalist?” (2016, p. 197). Crowdfunding challenges ethical practice by requiring journalists to pitch and report their stories to the public. At its most extreme, fundraising may tip journalism into what Berry and Sobieraj call outrage public opinion media, “recognisable by the rhetoric that defines it, with its hallmark venom, vilification of opponents, and hyperbolic reinterpretations of current events” (2016, p. 5). Reporting, in this case, becomes a means to outrage its audiences and channel that emotion into donations.

The Rebel, for example, blurred the line between financing a movement and a news outlet. In a now-deleted post on the NationBuilder blog, Torch Agency, the creative agency for The Rebel, explains NationBuilder’s role in launching what it called “Canada’s premier source of conservative news, opinion and activism”. The post continues,

In 36 hours, we built a fully-functional NationBuilder site complete with a database and communication headquarters... The result: through compelling content and top-notch digital tools, The Rebel raised over $100,000 CAD in less than twelve hours providing crucial early funding for its continuation.

The Rebel promised to use NationBuilder to better engage news audiences. The Rebel has repeatedly asserted its status as a journalism outlet against claims to the contrary. The Rebel enlisted the support of national press organisations, PEN Canada and Canadian Journalists for Free Expression, after being denied press credentials for a UN climate conference for being “advocacy journalism” (Drinkwater, 2016). In the Canadian province of Alberta, The Rebel successfully protested being removing from the media gallery because it wasn’t a “journalist source” (Edmiston, 2016).

The Rebel's response to a Canadian terrorist attack best frames the problem of distinguishing between advocacy, fundraising and journalism as well as NationBuilder's challenges in defining acceptable use. On 29 January 2017, a man entered a mosque in Québec City with an AK-47, killing six, seriously wounding five and injuring twelve people (Saminather, 2018). The Rebel launched the website QuebecTerror.com the next day. The initial page urged visitors to donate to send a Rebel reporter to cover the aftermath. The site, days after its claims had been discredited by other outlets, described the killing as inter-mosque violence based on a mistranslation of a public YouTube video. Rather than presenting itself as a journalistic report, the QuebecTerror website appeared as a conventional email fundraising pitch, depicting a dire reality – in this case a “truth” the mainstream media would not report – solvable through donations.

The language and matter of The Rebel’s reporting on the Québec terror attack resemble the tactics of outrage media, inflammatory rhetoric in this case complemented by a service to mobilise those emotions (Berry and Sobieraj, 2014). The Rebel’s response to the Québec terror attack then raises a different problem than journalists being uncomfortable in asking for money, as Hunter (2016) notes in a review of crowdfunding in journalism. Here fundraising overtakes reporting; stories are optimised for outrage. The problem is not new, but rather a consequence of the movement of practices between separate fields. Using the news to solicit funds is a known email marketing tactic. Emails that reacted to the news had the highest open rates according to analysis of Hillary Clinton’s email campaigning (Detrow, 2015). NationBuilder may streamline outrage tactics by channelling user engagement. Called a funnel or a ladder in marketing, NationBuilder has a path feature that tries to nudge user behaviour toward certain goals. Taken together, NationBuilder might ease this questionable form of crowdfunding in entrepreneurial journalism and encourage outrage tactics.

These concerns raise a second question: should NationBuilder be used in journalism, especially on hyper-partisan sites or outrage media already blurring the line between reporting, advocacy and fundraising? For its own part, fundraising ethics did cause turmoil at The Rebel. It suffered a scandal when a former correspondent accused the site of misusing funds, pointing to a disclaimer on the website that stated, “surplus funds raised for specific initiatives will be used for other costs associated with that particular project, such as website development, website hosting, mail, and other such expenses” (Gordon and Goldsbie, 2017). Seemingly, any campaign was part of a general pool of revenue, adding to concerns that certain stories might be juiced to bring in more money to general revenues.

These first two cases situate NationBuilder as part of the networked press. Ananny (2018) introduced the concept of the networked press to argue journalism exists within larger sociotechnical systems, of which NationBuilder is a part. Changes or disruption in these systems, evidenced through the rapid uptake of large social networking sites, do not necessarily imply increased press freedom and, instead require journalists’ practices to acknowledge and adapt to broader infrastructural changes. Just as outlets and journalists need to consider these changes, so too does NationBuilder in understanding how its technology is participating in the infrastructure of the networked press. As seen above, NationBuilder already participates in the ethical quandaries and its emphasis on mobilisation and fundraising may be ill-suited for journalistic outlets. NationBuilder might enable data collection and profiling without sufficient audience consent. NationBuilder might also tip the balance from journalism to outrage media by being a better tool to fundraise than publish stories. How does a firm like NationBuilder recognise its role in facilitating these transfers, particularly the expansion of marketing as the ubiquitous logic of cultural production? Should it ultimately be part of press infrastructure? Does using a political engagement platform ultimately improve journalistic practice? These matters require a more hands-on approach than that which NationBuilder presently offers.

Illiberal uses of political technology

Act for America engages in identity-based political advocacy, targeting American Muslims. Their mission includes immigration reform and combating terrorism. According to the Southern Poverty Law Center, their leadership has questioned the right to citizenship of American Muslims, alluding to mass deportation. Politically such statements seem at odds with the rules of what political theorist Nancy Rosenblum calls the “regulated rivalry” of liberal democracy. To protect itself, a militant democracy needs to ban parties that if elected or capable of influencing government “would implement discriminatory policies or worse: strip opposition religious or ethnic groups of civil or political rights, discriminate against minorities (or majorities), deport despised elements of the population” (Rosenblum, 2008, p. 434). Act for America seems to have engaged in such acts in targeting Muslim Americans.

NationBuilder then faces a third existential question: should groups that mobilise hate have access to its innovations? Other firms, like PayPal, stopped offering Act for America services after ProPublica reported on their relationship (Angwin et al., 2017a). While defining hate might be a little more difficult for an American firm where there is no clear hate speech laws, NationBuilder operates in many countries with clear laws and could guide corporate policy. That these terms are left missing or undefined in 3DNA’s Acceptable Use Policy is troubling.

The more challenging question that faces the larger industry is what responsibility do service providers have for the speech acts made on their services? As Whitney Phillips and Ryan Milner (2017) reflect, “it is difficult…to know how best – most effectively, most humanely, most democratically – to respond to online speech that antagonises, marginalises, or otherwise silences others. On one level, this is a logistic question about what can be done… The deeper and more vexing question is what should be done” (2017, p. 201) This vexing question is a lingering one, echoing the origins of modern broadcasting policy, which begins with governments and media industries attempting to reconcile preserving free speech without propagating hate speech. The American National Association of Broadcasters established a code of conduct in 1939 in part to ban shows like Father Coughlin’s that aired speeches “plainly calculated or likely to rouse religious or racial hatred and stir up strife” (Miller, 1938, as cited in Brown, 1980, p. 203). The decision did not solve the problem, but rather established institutions to consider these normative matters.

NationBuilder is not merely a broadcaster or a communication channel, but a mobilisation tool. The use of NationBuilder by hate groups should trouble the wider political technology industry and the field of political communication. It is part of a tradition in democratic politics that media technology does not just inform publics, but cultivates them. As Sheila Jasanoff notes, American “laws conceived of citizens as being not necessarily knowing but knowledge-able–that is, capable at need of acquiring the knowledge needed for effective self-governance. This idea of an epistemically competent citizen runs through the American political thought from Thomas Jefferson to John Dewey and beyond” (Jasanoff, 2016, p. 239). Communication is about formation as much as information, of cultivating publics. NationBuilder punctuates an existential question for political technology: is it exceptional or mundane? Is it a glorified spreadsheet or a special class of technology? In short, if NationBuilder is an effective tool of political mobilisation, should it effectively mobilise hate?

From corporate social responsibility to liberal democratic responsibility

Finding solutions to the problematic cases above is part of an international debate about platform governance (DeNardis, 2012; Duguay, Burgess, and Suzor, 2018; Gillespie, 2018; Gorwa, 2019). Platform governance refers to the conduct of large information intermediaries and, by extension, the social impacts of publicly accessible and networked computer technology. Where human rights is one emerging value set for platform governance (Kaye, 2019), the international challenge now is to the appropriate ‘web of influence’ that might address human rights concerns and address the numerous regulatory challenges posed by large technology firms (Braithwaite and Drahos, 2000).

Options include external rules – such as fines and penalties through privacy, data protection or election law – and co-regulatory approaches, like codes of conduct and best practices, in addition to self-regulation, specifically corporate social responsibility and responsibilities bestowed for liability protection. Self-regulation dominates the status quo, at least in the US. Therules are largely self-written by platforms, in large part due to their public service obligations under the US Telecommunications Act (Gillespie, 2018). Companies, like Facebook, have acknowledged a need for changing, publicly calling for government regulation (Zuckerberg, 2018). Today, platforms in good faith moderate users in conversations under acceptable use rules. Users might be banned, suspended, surveilled, deprioritised or demonetised under acceptable use policies (Myers West, 2018). The stakes now involve a debate about the public obligations of platforms and whether they should self-police or be deputised to enforce government rules (DeNardis, 2012; Tusikov, 2017).

The regulation of firms like NationBuilder face even greater regulatory challenges as the field has been historically free from much oversight or responsibilities. Many western democracies did not consider political parties or political data to be under the jurisdiction of privacy law. Enforcement was also lacking. Even though political parties were regulated in Europe, regulators only took their responsibilities seriously after the Facebook/Cambridge Analytica scandal (Bennett, 2015; Howard and Kreiss, 2010). Even with new data protection laws, intermediaries still face limited liability as enforcement tends towards the user than the service provider. Service providers are exempt from liability or penalties for misuse, except in certain cases such as copyright. For its own part, NationBuilder claims zero liability for interactions and hosted content according to its Terms of Service.

Political engagement platforms do face an uncertain global regulatory context. On one hand, they function as service providers largely exempt from laws. On the other hand, international law is uneven and changing (for a recent review, see Bennett and Oduro-Marfo, 2019). Public inquiries in the United Kingdom and Canada have focused more on these companies and their status may be changing. A joint investigation of AggregateIQ by the Privacy Commissioner of Canada and the Information and Privacy Commissioner for British Columbia found that the third-party service provider “had a legal responsibility to check that the third-party consent on which they were relying applied to the activities they subsequently performed with that data” (2019, p. 22). The implication is that AiQ had a corporate responsibility to abide by privacy laws in the provision of its services. The same likely holds for NationBuilder.

Amidst regulatory uncertainty, corporate social responsibility might be the most immediate remedy to questionable uses of NationBuilder. Its mission today might be read as ‘functionalist business ethics’ that believe that the product in and of itself is a social good and that more access, or more sales, improves the quality of elections. Whereas other approaches to corporate social responsibility favour an integrative business ethics where “a company’s responsibilities are not merely restricted in one way or another to the profit principle alone but to sound and critical ethical reasoning” (Busch and Shepherd, 2014, p. 297). Where future debates might require consideration of NationBuilder’s obligations to liberal democracy, the next section considers how NationBuilder’s mission and philosophy might be clarified through the company’s acceptable use policy. NationBuilder might not have to become partisan, but it cannot be neutral toward these institutions of liberal democracy, at least if it wants to continue to believe in its mission to revolutionise politics.

Revising the Acceptable Use Policy is possible and has happened before. Clearly stating the relationship between its mission and prohibited uses would reverse past amendments that narrowed corporate responsibilities. The Acceptable Use Policy as of August 2019, last updated 1 May 2018, is more open than prior iterations. Most bans concern computer security, prohibiting uses that overload infrastructure or accessing data without authorisation. The policy does prohibit “possessing or disseminating child pornography, facilitating sex trafficking, stalking, troll storming, threatening imminent violence, death or physical harm to any individual or group whose individual members can reasonably be identified, or inciting violence”. Until 2014, 3DNA covered acceptable use as part of its terms of service; afterwards it became a separate document. Its Terms of Service agreement from 29 March 2011 banned specific user content including “any information or content that we deem to be unlawful, harmful, abusive, racially or ethnically offensive, defamatory, infringing, invasive of personal privacy or publicity rights, harassing, humiliating to other people (publicly or otherwise), libellous, threatening, profane, or otherwise objectionable” as well as a subsequently removed ban on posting incorrect information. These clauses were removed in the 2014 update that reduced prohibited uses to 15. These clauses have slowly been added back. The most recent acceptable usage policy, as of 1 May 2018, had 31 prohibited uses, adding back clauses regulating user activities.

Recommendation #1: Reconcile its mission statement with its prohibited uses

NationBuilder’s Mission is to connect anyone regardless of “race, age, class, religion, educational background, ideology, gender, sexual orientation or party”. By contrast, its Acceptable Use Policy does not consider the positive freedoms inferred in this mission that could conceivably prohibit campaigns aimed at excluding people from participating in politics. A revised Acceptable Use Policy should apply the implications of its corporate mission to its prohibited uses. Act for America, for example, targets its opponents by race and advocates for greater policing, terrorism laws and immigration enforcement that could disproportionately affect Muslim Americans, acting against NationBuilder’s vision of “a world where everyone has the freedom and opportunity to create what they are meant to create”. Revision might prohibit campaigns or parties targeting assigned identities like race, age, gender or sexual orientation, particularly when messages incite hate, while preserving customers’ right to campaign against ideology, party or other chosen or elective politicised issues. To achieve such a mission, NationBuilder may have to restrict access on political grounds (also called de-platforming) or to restrict certain features. 5

Harmonising its position on political freedom may prompt industry-wide reflection on the function of political technology. How do these services protect the liberal democratic institutions they ostensibly promise to disrupt? In finding shared values, NationBuilder has to consider its place in a partisan field. Can it navigate between parties to describe ethical campaigning, or, alternatively, must it find other companies with shared nonpartisan or libertarian values? The likely outcome either way is a code of conduct for digital campaigning similar to the Alliance of Democracies Pledge for Election Integrity or the codes of conduct of the American Association of Political Consultants or European Association of Political Consultants that discourage campaigns based on intolerance and discrimination. In doing so, NationBuilder might force partisan firms to be more explicit about their professional ethics.

Recommendation #2: Require disclosure on customers’ websites

NationBuilder should disclose when it is used even if it cannot decide if it should be used. Two out of the three questionable uses might have benefitted from the organisations’ disclosing their use of the political engagement platform, especially when used in journalism. At a minimum, NationBuilder should require sites to disclose using NationBuilder, ideally through an icon or other disclosure in the page’s footer that might create the possibility of public awareness (Ezrahi, 1999). NationBuilder might also consider requiring users to disclose what tracking features, such as Match and Political Capital, are enabled on the website not unlike the disclosure about data tracking under Europe’s Cookie Law that disclose a site’s use of the tracking tool.

NationBuilder might further standardise the reporting of uses found in its annual report and potentially release data in a separate report. Transparency reports have become an important, albeit imperfect, reporting tool in telecommunications and social media industries (Parsons, 2019). These reports, ideally, would continue the preliminary method used in this paper, breaking down NationBuilder’s use by industry over time and potentially expanding data collection to include other trends such as use by country, use by party and the popularity of features. Such proactive disclosure might also normalise greater transparency in a political technology industry known for its secrecy.

Recommendation #3: Clarify relationship to domestic privacy law

A revised acceptable use policy might define NationBuilder expectations for privacy rights both to explain its normative vision for privacy and improve its customers’ implementation of local privacy law. By contrast, the acceptable use policy currently prohibits applications that “infringe or violate the intellectual property rights (including copyrights), privacy rights or any other rights of anyone else (including 3DNA)”. The clause does not clarify the meaning of privacy rights or jurisdiction. Elsewhere 3DNA states that all its policies “are governed by the internal substantive laws of the State of California, without respect to its conflict of laws principles”. Such ambiguity confuses a clear interpretation of privacy rights, the law and regulation mentioned in the policy. A revised clause should state NationBuilder’s position on privacy as a human right, in such a way that it provides some guidance as to whether local law meets its standards and denies access in countries that do not meet its privacy expectations. Further, the acceptable use policy should also clarify that it expects customers to abide by local privacy law, and, in major markets, if it has any reporting obligations to privacy offices.

Clarifying its position on privacy rights recognises the important function NationBuilder plays in educating its customers on the law. NationBuilder may help implement “proactive guidance on best campaigning practices” recommended by Bennett and Oduro-Marfo (2019, p. 54). For its GDPR compliance, NationBuilder has built a blog and offers many educational resources to customers to understand how to campaign online and to respect the law. These posts clearly state that they are not legal advice, but they do help to interpret the law for practitioners. Similar posts could help clients understand if they should disable certain features in NationBuilder, such as Match or Political Capital, to comply with their domestic privacy law. Revisions to its Acceptable Use Policy might be another avenue for NationBuilder to educate its customers.

Adding privacy to its corporate mission may be a further signal of NationBuilder’s corporate responsibility. NationBuilder has an altogether different relationship to customer privacy than other advertising-based technology firms. Its revenues come from being a service provider and securing data. With growing pressure on political parties to improve their cyber-security, NationBuilder can help its clients better protect their voter data as well as call for better privacy protection in politics overall. Indeed, NationBuilder could advocate for privacy law to apply to its political clients to both simplify its regulatory obligations and reduce risk. Improving privacy may lessen its institutional risk of being associated with major privacy violations as well as simplifying the complex work of setting privacy rules on its own. As such, NationBuilder might be a possible global advocate for better privacy and data protection, a role to date unfulfilled long after public controversy.

Conclusion

This paper has reported the results of empirical research about the acceptable use of a political technology. The results demonstrate that political technologies have questionable uses involving their application within politics. Specifically when does a political movement exceed the limits of liberal democratic discourse? When are its uses in journalism and advertising unacceptable? The experiment demonstrates that harms to liberal democracy can be a reasonable way to judge technological risks. Liberal democratic norms are another factor to consider to the wider study of software and technological accountability (Johnson and Mulvey, 1995; Nissenbaum, 1994). These concerns have a long history. Norbert Wiener, who helped develop digital computing, warned against its misuse in Cold War America for the management of people (Wiener, 1966, p. 93). By comparison, science and technology scholar Sheila Jasanoff (2016) questions if the benefits of technological innovation outweigh the risks of global catastrophe, inequality, and human dignity. While catastrophic global devastation is commonly seen as a questionable use of technology (unless it concerns the climate), there is less consensus about how technology might undermine democracy, of which liberal democracy is just one set of norms. What democracy should be defended is debated (with fault lines drawn between representative, direct and deliberative democracy as well as between liberal and republican traditions) (Karppinen, 2013). My method helps to clarify this debate by finding inductively uses that might challenge many theories of democracy. Further research could extend the analysis to focus on particular concerns to different forms of democracy and democratic theories.

My specific recommendations for NationBuilder may improve the accountability of the political industry at large. Oversight is a major problem in the accountability of political platforms. My methods could easily be scaled to observe more companies and countries. No doubt privacy, information and election regulation could implement this approach as part of their situational awareness. The questionable uses here then offer uses to watch for:

1. Does the technology facilitate or ease deceptive or non-consensual data collection?
2. Does the technology undermine journalistic standards and consider its role in the networked press?
3. Does the technology facilitate the mobilisation of hate groups?

Where remedies to these challenges may be unclear, at the very least ongoing monitoring could identify potential harms sooner than academic research.

Questionable uses of NationBuilder should trouble the company as well as the larger political technology industry and the field of political communication. Faith in political technologies has changed campaign practice in many democracies as well as attracted ongoing international regulatory attention concerned with trust and fairness during elections. Technologies like NationBuilder are premised on the value of communications to political engagement. They are designed to increase engagement and improve efficiency. NationBuilder and its peers are a special class of political technology and thus their obligations to liberal democratic values should be scrutinised. If 3DNA seeking to better politics suffers these abuses then what will come from political firms with less idealism?

Introduction

Data has become an important part of how we understand political campaigns. In reviewing coverage of elections – particularly in the US – the idea that political parties and campaigners now utilise data to deliver highly targeted, strategic and successful campaigns is readily found. In academic and non-academic literature, it has been argued that “[i]n countries around the world political parties have built better databases, integrated online and field data, and created more sophisticated analytic tools to make sense of these traces of the electorate” (Kreiss and Howard, 2010, p. 1; see also in t’Veld, 2017, pp. 2-3). These tools are reported to allow voters to “be monitored and targeting continuously and in depth, utilising methods intricately linked with and drawn from the commercial sector and the vast collection of personal and individual data” (Kerr Morrison, Naik, and Hankey, 2018, p. 11). The Trump campaign in 2016 is accordingly claimed to have “target[ed] 13.5 million persuadable voters in sixteen battleground states, discovering the hidden Trump voters, especially in the Midwest” (Persily, 2017, p. 65). On the basis of such accounts, it appears that data-driven campaigning is coming to define electoral practice – especially in the US - and is now key to understanding modern campaigns.

Yet, at the same time, important questions have been raised about the sophistication and uptake of data-driven campaign tools. As Baldwin-Philippi (2017) has argued, there are certain “myths” about data-driven campaigning. Studying campaigning practices Baldwin-Philippi has shown that “all but the most sophisticated digital and data-driven strategies are imprecise and not nearly as novel as the journalistic feature stories claim” (2017, p. 627). Hersh (2015) has also shown that the data that parties possess about voters is not fine-grained, and tends to be drawn from public records that contain certain standardised information. Moreover, Bennett has highlighted the significant incentive that campaign consultants and managers have to emphasise the sophistication and success of their strategies, suggesting that campaigners may not be offering an accurate account of current practices (2016, p. 261; Kreiss and McGregor, 2018).

These competing accounts raise questions about the nature of data-driven campaigning and the extent to which common practices in data use are found around the globe. These ideas are conceptually important for our understanding of developments in campaigning, but they also have significance for societal responses to the practice of data-driven campaigning. With organisations potentially adopting different data-driven campaigning practices it is important to ask which forms of data use are seen to be democratically acceptable or problematic. 1 These questions are particularly important given the recent interest from international actors and politicians in understanding and responding to the use of data analytics (Information Comissioners Office, 2018a), and specifically practices at Facebook (Kang et al., 2018). Despite growing pressure from these actors to curtail problematic data-driven campaigning practices, it is as yet unclear precisely what is unacceptable and how prevalent these practices are in different organisations and jurisdictions. For these reasons there is a need to understand more about data-driven campaigning.

To generate this insight, in this article I pose the question: “what practices characterise data-driven campaigning?” and develop a comparative analytical framework that can be used to understand, map and consider responses to data-driven campaigning. Identifying three facets of this question, I argue that there can be variations in who is using data in campaigns, what the sources of data are, and how data informs communication in campaigns. Whilst not exhaustive, these questions and the categories they inspire are used to outline the diverse practices that constitute data-driven campaigning within single and different organisations in different countries. It is argued that our understanding of who, what and how data is being used is critical to debates around the democratic acceptability of data-driven campaigning and provides essential insights required when contemplating a regulatory response.

This analysis and the frameworks it inspires have been developed following extensive analysis of the UK case. Drawing on a three-year project exploring the use of data-driven campaigning within political parties, the analysis discusses often overlooked variations in how data is used. In highlighting these origins I contend that these questions are not unique to the UK case, but can inspire analysis around the globe and in different organisations. Indeed, as I will discuss below, this form of inquiry is to be encouraged as comparative analysis makes it possible to explore how different legal, institutional and cultural contexts affect data-driven campaigning practices. Furthermore, analysis of different kinds of organisation makes it possible to understand the extent to which party practices are unique. Although this article is therefore inspired by a particular context and organisational type, the questions and frameworks it provides can be used to unpack and map the diversity of data-driven campaigning practices, providing conceptual clarity able to inform a possible regulatory response.

Data and election campaigns

The relationship between data and election campaigns is well established, particularly in the context of political parties. Describing the focus of party campaigning, Dalton, Farrell and McAllister (2013) outline the longstanding interest parties have in collecting data that can be analysed to (attempt to) achieve electoral success. In their account, “candidates and party workers meet with individual voters, and develop a list of people’s voting preferences. Then on election day a party worker knocks on the doors of prospective supporters at their homes to make sure they cast their ballot and often offers a ride to the polls if needed” (p. 56). Whilst parties in different contexts are subject to different regulations and norms that affect the data they can collect and use (Kreiss and Howard, 2010), it is common for them to be provided with information by the state about voters’ age, registered status and turnout history (Hersh, 2015). In addition, parties then tend to gather their own data about voter interests, voting preferences and degree of support, allowing them to build large data sets and email lists at national and local levels. Although regulated – most notably through the General Data Protection Regulation (GDPR), which outlines rules in Europe for how data can be collected, used and stored – parties’ use of data is often seen to be democratically permissible as it enables participation and promotes an informed citizenry.

In recent history, the use of data by parties is seen to have shifted significantly, making it unclear how campaigns are organised and whether they are engaging in practices that may not be democratically appropriate. In characterising these practices, two very different accounts of data use have emerged. On the one hand, scholars such as Gibson, Römmele and Williamson (2014) have argued that parties now adopt data-driven campaigns that “focus on mining social media platforms to improve their voter profiling efforts” (p. 127). From this perspective, parties are now often seen to be routinely using data to gain information, communicate and evaluate campaign actions.

In terms of information, it has been argued that data-driven campaigning draws on new sources of data (often from social media and online sources) to allow parties to search for patterns in citizens’ attitudes and behaviours. Aggregating data from many different sources at a level hitherto impossible, data-driven campaigning techniques are seen to allow parties to use techniques common in the commercial sector to “construct predictive models to make targeting campaign communications more efficient” (Nickerson and Rogers, 2014, p. 54; Castleman, 2016; Hersh, 2015, p. 28). Similarly, attention has been directed to the capacity to use algorithms to identify “look-alike audiences” (Tactical Tech, 2019, pp. 37-69), 2 allowing campaigners to find new supporters who possess the same attributes as those already pledged to a campaign (Kreiss, 2017, p. 5). Data-driven campaigning techniques are therefore seen to offer campaigns additional information with minimal investment of resources (as one data analyst becomes able to find as many target voters as an army of grassroots activists) (Dobber et al., 2017, p. 4).

In addition, data-driven campaigning has facilitated targeted communication (Hersh, 2015, pp. 1-2), allowing particular messages to be conveyed to certain kinds of people. These capacities are seen to enable stratified campaign messaging, allowing personalised messages that can be delivered fast through cheap and easy to use online (and offline) interfaces. Data-driven campaigning has therefore been reported to allow campaigners to “allocate their finite resources more efficiently” (Bennett, 2016, p. 265), “revolutioniz[ing] the process” of campaigning (International IDEA, 2018, p. 7; Chester and Montgomery, 2017).

It has also been claimed that data-driven campaigning enables parties to evaluate campaign actions and gather feedback in a way previously not possible. Utilising message-testing techniques such as A/B testing, and monitoring response rates and social media metrics, campaigners are seen to be able to use data to analyse – in real time – the impact of campaign actions. Whether monitoring the effect of an email title on the likelihood that it is opened by recipients (Nickerson and Rogers, 2014, p. 57), or testing the wording that makes a supporter most likely to donate funds, data can be gathered and analysed by campaigns seeking to test whether their interventions work (Kreiss and McGregor, 2018, pp. 173-4; Kerr Morrison et al., 2018, p. 12; Tactical Tech, 2019). 3

These new capacities are often highlighted in modern accounts of campaigning and suggest that there has been significant and rapid change in the activities of campaigning organisations. Whilst prevalent, this idea has, however, been challenged by a small group of scholars who have offered a more sceptical account, arguing that “the rhetoric of data-driven campaigning and the realities of on-the-ground practices” are often misaligned (Baldwin-Philippi, 2017, p. 627).

The sceptical account

A number of scholars of campaign practice have questioned the idea that elections are characterised by data-driven campaigning and have highlighted a gulf between the rhetoric and reality of practices here. Nielsen, for example, has shown that whilst data-driven tools are available, campaigns continue to rely primarily on “mundane tools” (2010, p. 756) such as email to organise their activities. Hersh also found that, in practice, campaigns do not possess “accurate, detailed information about the preference and behaviours of voters” (2015, p. 11), but rely instead on relatively basic, publically available data points. Similar observations led Baldwin-Philippi to conclude that the day-to-day reality of campaigning is “not nearly as novel as the journalistic feature stories claim” as “campaigns often do not execute analytic-based campaigning tactics as fully or rigorously as possible” (2017, p. 631). In part the gulf between possible and actual practice has emerged because parties – especially at a grassroots level – lack the capacity and expertise to utilise data-driven campaigning techniques (Ibid., p. 631). There is accordingly little evidence that parties are routinely using data to gain more information about voters, to develop new forms of targeted communication or to evaluate campaign interventions. Indeed, in a study of the UK, Anstead et al. found no evidence “that campaigns were seeking to send highly targeted but contradictory messages to would-be supporters”, with their study of Facebook advertisements showing that parties placed adverts that reflected “the national campaigns parties were running” (unpublished, p. 3).

Other scholars have also questioned the scale of data-use by highlighting the US-centric focus of much scholarship on political campaigns (Kruschinski and Haller, 2017; Dobber at al., 2017). Kreiss and Howard (2010) have highlighted important variations in campaign regulation that restrict the practices of data-driven campaigns (see also: Bennett, 2016). In this way, a study of German campaigning practices by Kruschinski and Haller (2017) highlights how regulation of data collection, consent and storage means that “German campaigners cannot build larger data-bases for micro-targeting” (p. 8). Elsewhere Dobber et al. (2017, p. 6) have highlighted how different electoral systems, regulatory systems and democratic cultures can inform the uptake of data-driven campaigning tools. This reveals that, whilst often discussed in universal terms, there are important country and party level variations that reflect different political, social and institutional contexts. 4 These differences are not, however, often highlighted in existing accounts of data-driven campaigning.

Reflecting on reasons for this gulf in rhetoric and practice, some attention has been directed to the incentives certain actors have to “sell” the sophistication and success of data-driven campaigning practices. For Bennett, political and technical consultants “are eager to tout the benefits of micro-targeting and data-driven campaigning, and to sell a range of software applications, for both database and mobile environments” (2016, p. 261). Indeed, with over 250 companies operating worldwide that specialise in the use of individual data in political campaigns (Kerr Morrison, Naik, and Hankey, 2018, p. 20), there is a clear incentive for many actors to “oversell” the gains to be achieved through the use of data-targeting tools (a behaviour Cambridge Analytica has, for example, been accused of). Whatever the causes of these diverging narratives, it is clear that our conceptual understanding of the nature of data-driven campaigning, and our empirical understanding of how extensively different practices are found is underdeveloped. We therefore currently lack clear benchmarks against which to monitor the form and extent of data-driven campaigning.

These deficiencies in our current conceptualisation of data-driven campaigning are particularly important because there has been recent (and growing) attention paid to the need to regulate data-use in campaigns. Indeed, around the globe calls for regulation have been made citing concerns about the implications of data-driven campaigning for privacy, political debate, transparency and social fragmentation (Dobber et al, 2017, p. 2). In the UK context, for example, the Information Commissioner, Elizabeth Denham, launched an inquiry into the use of data analytics for political purposes by proclaiming:

[w]hat we're looking at here, and what the allegations have been about, is mashing up, scraping, using large amounts of personal data, online data, to micro target or personalise or segment the delivery of the messages without individuals' knowledge. I think the allegation is that fair practices and fair democracy is under threat if large data companies are processing data in ways that are invisible to the public (quoted in Haves, 2018, pp. 2-3).

Similar concerns have been raised by the Canadian Standing Committee on Access to Information, Privacy and Ethics, the US Senate Select Committee on Intelligence, and by international bodies such as the European Commission. These developments are particularly pertinent because the conceptual and empirical ambiguities highlighted above make it unclear which data-driven campaign practices are problematic, and how extensively they are in evidence.

It is against this backdrop that I argue there is a need to unpack the idea of data-driven campaigning by asking “what practices characterise data-driven campaigning?”. Posing three supplementary questions, in the remainder of the article I provide a series of conceptual frameworks that can be used to understand and map a diversity of data use practices that are currently obscured by the idea of data-driven campaigning. This intervention aims not only to clarify our conceptual understanding of data-driven campaigning practices, and to provide a template for future empirical research, but also to inform debate about the democratic acceptability of different practices and the form any regulatory response should take.

Navigating the practice of data-driven campaigns

Whilst often spoken about in uniform terms, data-driven campaigning practices come in a variety of different forms. To begin to understand the diversity of different practices, it is useful to pose three questions:

1. Who is using data in campaigns?
2. What are the sources of campaign data?
3. How does data inform communication?

For each question, I argue that it is possible to identify a range of answers rather than single responses. Indeed, different actors, sources and communication strategies can be associated with data use within single as well as between different campaigns. Recognising this, I develop three analytical frameworks (one for each question) that can be used to identify, map and contemplate different practices.

These frameworks have been designed to enable comparative analysis between different countries and organisations, highlighting the many different ways in which data is used. Whilst not applied empirically within this article, the ideal type markers outlined below can be operationalised to map different practices. In doing so it should be expected that a spectrum of different positions will be found within any single organisation. Whilst it is not within the scope of this paper to fully operationalise these frameworks, methods of inquiry are discussed to highlight how data may be gathered and used in future analysis. In the discussion below, I therefore offer these frameworks as a conceptual device that can be built upon and extended in the future to generate comparative empirical insights. This form of empirical analysis is vital because it is expected that answers to the three questions will vary depending on the specific geographic or organisational context being examined, highlighting differences in data driven campaigning that need to be recognised by those considering regulation and reform.

Who is using data in campaigns?

When imagining the orchestrators of data-driven campaigning the actors that come to mind are often data specialists who provide insights for party strategists about how best to campaign. Often working for an external company or hired exclusively for their data expertise, these actors have received much coverage in election campaigns. Ranging from the now notorious Cambridge Analytica, to established companies such as BlueStateDigital and eXplain (formerly Liegey Muller Pons), there is often evidence that professional actors facilitate data-driven campaigns. Whilst the idea that parties utilise professional expertise is not new (Dalton et al., 2001, p. 55; Himmelweit et al., 1985, pp. 222-3), data professionals are seen to have gained particular importance because “[n]ew technologies require new technicians” (Farrell et al., 2001). This means that campaigners require external, professional support to utilise new techniques and tools (Kreiss and McGregor, 2018; Nickerson and Rogers, 2014, p. 70). Much commentary therefore gives the impression that data-driven campaigning is being facilitated by an elite group of professional individuals with data expertise. For those concerned about the misuse of data and the need to curtail practices seen to have negative democratic implications, this conception suggests that it is the actions of a very small group that are of concern. And yet, as the literature on campaigns demonstrates, parties are reliant on the activism of local volunteers (Jacobson, 2015), and often lack the funds to pay for costly data expertise (indeed, in many countries spending limits prevent campaigners from paying for such expertise). As a result, much data-driven campaigning is not conducted by expert data professionals.

In thinking through this point, it is useful to note that those conducting data-driven campaigning can have varying professional status and levels of expertise. These differences need to be recognised because they affect both who researchers study when they seek to examine data-driven campaigning, but also whose actions need to be regulated or overseen to uphold democratic norms. 5 Noting this, it is useful to draw two conceptual distinctions between professional and activist data users, and between data novices and experts. These categories interact, allowing four “ideal type” positions to be identified in Figure 1.

Figure 1: Who is using data in campaigns?6

Looking beyond the “expert data professionals” who often spring to mind when discussing data-driven campaigning, Figure 1 demonstrates that there can be different actors using data in campaigns. It is therefore common to find “professionals without data expertise” who are employed by a party. Whilst often utilising or collecting data, these individuals do not possess the knowledge to analyse data or develop complex data-driven interventions. Interestingly, this group has been understudied in the context of campaigns, meaning the precise differences between external and internal professionals are not well understood.

In addition to professionals, Figure 1 also shows that data-driven campaigning is performed by activists who can vary in their degree of expertise. Some, described here as “expert data activists”, can possess specialist knowledge - often having many of the same skills as expert data professionals. Others, termed “activists without data expertise”, lack even basic understandings of digital technology (let alone data-analysis) (Nielsen, 2012). Some attention has been paid to activists” digital skills in recent elections with, for example, coverage of digital expertise amongst Momentum activists in the UK (Zagoria and Schulkind, 2017) and Bernie Sanders activists in the US (Penney, 2017). And yet, other studies have suggested that such expertise is not common amongst activists (Nielsen, 2012).

These classifications therefore suggest that data-driven campaigning can and is being conducted by very different actors who vary in their relationship with the party, and in their expertise. Currently we have little insight into the extent to which these different actors dominate campaigns, making it difficult to determine who is using data, and hence whose activities (if any) are problematic. This indicates the need for future empirical analysis that sets out to determine the prevalence and relative power of these different actors within different organisations. Whilst space prevents a full elucidation of the markers that could be used for this analysis, it would be possible to map organisational structures and use surveys to gauge the extent of data-expertise present amongst professionals and activists. In turn, these insights could be mapped against practices to determine who was using data in problematic ways. It may, for example, be that whilst “expert data professionals” are engaging in practices that raise questions about the nature of democratic debate (such as micro-targeting), “activists without data expertise” may be using data in ways that raise concerns about data security and privacy.

Knowing who is using data how is critical for thinking about where any response may be required, but also when considering how a response can be made. Far from being subject to the same forms of oversight these different categories of actors are subject to different forms of control. Whilst professionals tend to be subject to codes of conduct that shape data use practices, or can be held accountable by the threat of losing their employment, the activities of volunteers can be harder to regulate. As shown by Nielsen (2012), even when provided with central guidance and protocols, local activists often diverge from central party instructions, reflecting a classic structure/agency dilemma. This suggests not only that the activities of different actors may require monitoring and regulation, but also that different responses may be required. The question “who is using data in campaigns?” therefore spotlights a range of practices and democratic challenges that are often overlooked, but which need to be appreciated in developing our understanding and any regulatory response.

What are the sources of campaign data?

Having looked at who is using data in campaigns, it is, second, important to ask what are the sources of campaign data? The presumption inherent in much coverage of data-driven campaigning is that campaigners possess complex databases that hold numerous pieces of data about each and every individual. The International Institute for Democracy and Electoral Assistance (IDEA), for example, has argued that parties “increasingly use big data on voters and aggregate them into datasets” which allow them to “achieve a highly detailed understanding of the behaviour, opinions and feelings of voters, allowing parties to cluster voters in complex groups” (2018, p. 7; p. 5). It therefore often appears that campaigns use large databases of information composed of data from different (and sometimes questionable) sources. However, as suggested above, the data that campaigns possess is often freely disclosed (Hersh, 2015), and many campaigners are currently subject to privacy laws around the kind of data they can collect and utilise (Bennett, 2016; Kruschinski and Haller, 2017).

To understand variations and guide responses, four more categories are identified. These are determined by thinking about variations in the form of data; differentiating between disclosed and inferred data, and the conditions under which data is made available; highlighting differences between data that is made available without charge, and data that is purchased.

Figure 2: The sources of campaigning data

As described in Figure 2, much of the data that political parties use is provided to them without charge, but it can come in two forms. The first category “free data disclosed by individuals” refers to data divulged to a campaign without charge, either via official state records or directly by an individual to a campaign. The official data provided to campaigns varies from country to country (Dobber et al., 2017, p. 7; Kreiss and Howard, 2010, p. 5) but can include information on who is registered to vote, a voter’s date of birth, address and turnout record. In the US it can even include data on the registered partisan preference of a particular voter (Bennett, 2016, p. 265; Hersh, 2015). This information is freely available to official campaigners and citizens are often legally required to divulge it (indeed, in the UK it is compulsory to sign up to the Electoral Register). In addition, free data can also be more directly disclosed by individuals to campaigns through activities such as voter canvassing and surveys that gather data about individuals’ preferences and concerns (Aron, 2015, pp. 20-1; Nickerson and Rogers, 2014, p. 57). The second category “free inferred data” identifies data available without charge, but which is inferred rather than divulged. These deductions can occur through contact with a campaign. Indeed, research by the Office of the Information and Privacy Commissioner for British Columbia, Canada describes how party canvassers often collect data about ethnicity, age, gender and the extent of party support by making inferences that the individual themselves is unaware of (2019, p. 22). It is similarly possible for data that campaigns already possess to be used to make inferences. Information gathered from a petition, for example, can be used to make suppositions about an individual’s broader interests and support levels. Much of the data campaigners use is therefore available without charge, but differs in form.

In addition, Figure 2 captures the possibility that campaigns purchase data. This data can be classified in two ways. The category “purchased data disclosed by individuals” describes instances in which parties buy data that was not disclosed directly to them, but was provided to other actors. This data can come in the form of social media data (which parties can buy access to rather than possess), or include data such as magazine subscription lists (Chester and Montgomery, 2017, pp. 3-4; Nickerson and Rogers, 2014, p. 57). Figure 2 also identifies “purchased inferred data”. This refers to modelled data whereby inferences are made about individual preferences on the basis of available data. This kind of modelling is frequently accomplished by external companies using polling data or commercially available insights, but it can also be done on social media platforms, with features such as look-a-like audiences on Facebook selling access to inferred data about individuals’ views.

Campaigns can therefore use different types of data. Whilst the existing literature has drawn attention to the importance of regulatory context in shaping the data parties in different countries are legally able to use (Kruschinski and Haller, 2017), there are remarkably few comparative studies of data use in different countries. This makes it difficult to determine not only how places vary in their regulatory tolerance of these different forms of data, but also how extensively parties actually use them. Such analysis is important because parties’ activities are not only shaped by laws, but can also be informed by variables such as resources or available expertise (Hersh, 2015, p. 170). This makes it important to map current practices and explore if and why data is used in different ways by parties around the world. In envisioning such empirical analysis, it is important to note that parties are likely to be sensitive to the disclosure of data sources. However a mix of methods - including interviews with those using data within parties and data subject access requests - can be used to gain insights here.

In the context of debates around data-driven campaigning and democracy, these categories also prompt debate about the acceptability of different practices. Whilst the idea that certain forms of disclosed data should be available without charge is relatively established as an acceptable component of campaigns, it appears there are concerns over the purchase of data and the collection of inferred data. Indeed, in Canada the Office of the Information and Privacy Commissioner for British Columbia recommended that “[a]ll political parties should ensure door-to-door canvassers do not collect the personal information of voters, including but not limited to gender, religion, and ethnicity information unless that voter has consented to its collection” (2019, p. 41). By acknowledging the different sources of data used for data-driven campaigning it is therefore possible to not only clarify what is happening, but also to think about which forms of data can be acceptably used by campaigns.

How does data inform communication?

Finally, in thinking about data-driven campaigning much attention has been paid to micro-targeting and the possibility that data-driven campaigning allows parties to conduct personalised campaigns. IDEA has therefore argued that micro-targeting allows parties to “reach voters with customized information that is relevant to them…appealing to different segments of the electorate in different ways” with new degrees of precision (2018, p. 7). In the context of digital politics, micro-targeting is seen to have led parties to:

…try to find and send messages to their partisan audiences or intra-party supporters, linking the names in their databases to identities online or on social media platforms such as Facebook. Campaigns can also try to find additional partisans and supporters by starting with the online behaviours, lifestyles, or likes or dislikes of known audiences and then seeking out “look-alike audiences”, to use industry parlance (Kreiss, 2017, p. 5).

In particular, platforms such as Facebook are seen to provide parties with a “powerful “identity-based“ targeting paradigm” allowing them to access “more than 162 million US users and to target them individually by age, gender, congressional district, and interests” (Chester and Montgomery, 2017, p. 4). These developments have raised important questions about the inclusivity of campaign messaging and the degree to which it is acceptable to focus on specific segments of the population. Indeed, some have highlighted risks relating to mis-targeting (Hersh and Schaffner, 2013) and privacy concerns (Kim et al., 2018, p. 4). However, as detailed above, there are questions about the extent to which campaigns are sending highly targeted messages (Anstead et al., unpublished).

In order to understand different practices, Figure 3 differentiates between audience size; specifying between wide and narrow audiences, and message content; noting differences between generic and specialised messages.

Figure 3: How data informs communication

Much campaigning activity comprises generic messages, with content covering a broad range of topics and ideas. By using data (often generated through polling or in focus groups) parties can determine the form of messaging likely to win them appeal. The category “general message to all voters” describes instances in which a general message is broadcast to a wide audience, something that often occurs via party political TV broadcasts or political speeches (Williamson, Miller and Fallon, 2010, p. iii). In contrast “generic message to specific voters” captures instances in which parties limit the audience, but maintain a general message. Such practices often emerge in majoritarian electoral systems where campaigners want to appeal to certain voters who are electorally significant, rather than communicating with (and potentially mobilising) supporters of other campaigns (Dobber et al., 2017, p. 6). Parties therefore often gather data to identify known supporters or sympathisers who are then sent communications that offer a general overview of the party’s positions and goals.

Figure 3 also spotlights the potential for parties to offer more specialised messages, describing a campaign’s capacity to cover only certain issues or aspects of an issue (focusing, for example, on healthcare rather than all policy realms, or healthcare waiting lists rather than plans to privatise health services). These messages can, once again, be deployed to different audiences. The category “specialised message to all voters” describes instances in which parties use data to identify a favourable issue (Budge and Farlie, 1983) that is then emphasised in communications with all citizens. In the UK, for example, the Labour Party often communicates its position on the National Health Service, whereas the Conservative Party focuses on the economy (as these are issues which, respectively, the two parties are positively associated with). Finally, “specialised message to specific voters” captures the much discussed potential for data to be used to identify a particular audience that can then be contacted with a specific message. This means that parties can speak to different voters about different issues – an activity that Williamson, Miller and Fallon describe as “segmentation” (2010, p. 6).

These variations suggest that campaigners can use data to inform different communication practices. Whilst much attention has been paid to segmented micro-targeting (categorised here as “specialised messages to specific voters”), there is currently little data on the degree to which each approach characterises different campaigns (either in single countries or different nations). This makes it difficult to determine how extensive different practices are, and whether the messaging conducted under each heading is taking a problematic form. It may, for example, be that specialised messaging to specific voters is entirely innocuous, or it could be that campaigners are offering contradictory messages to different voters and hence potentially misleading people about the positions they will take (Kreiss, 2017, p. 5). Empirically, this form of analysis can be pursued in different ways. As above, interviews with campaign practitioners can be used to explore campaign strategies and targeting, but it is also important to look at the actual practices of campaigns. Resources such as online advertising libraries and leaflet repositories are therefore useful in monitoring the content and focus of campaign communications. Using these methods, a picture of how data informs communication can be developed.

Thinking about the democratic implications of these different practices, it should be noted that message variation by audience size and message scope is not new - campaigns have often varied in their communication practices. And yet digital micro-targeting and voter segmentation has been widely greeted with alarm. This suggests the importance of thinking further about the precise cause of concern here, determining which democratic norms are being violated, and whether this is only occurring in the digital realm. It may, for example, be that concerns do not only reflect digital practices, suggesting that regulation is needed for practices both online and offline. These categories therefore help to facilitate debate about the democratic implications of different practices, raising questions about precisely what it is that is the cause for concern and where a response needs to be made.

Discussion

The above discussion has shown that data-driven campaigning is not a homogenous construct but something conducted by different actors, using different data, adopting different strategies. To date much existing discussion of data-driven campaigning has focused on the extent to which this practice is found. In contrast, in this analysis I have explored the extent to which different data-driven campaigning practices can be identified. Highlighting variations in who is using data in campaigns, what the sources of campaign data are, and how data informs campaign communication, I argue that there are a diverse range of possible practices.

What is notable in posing these questions and offering these frameworks is that whilst there is evidence to support these different conceptual categories, at present there is little empirical data on the extent to which each practice exists in different organisations. As such, it is not clear what proportion of campaign activity is devoted to targeting specific voters with specific messages as opposed to all voters with a general message. Moreover, it is not clear the extent to which parties rely on different actors for data-driven campaigning, nor how much power and scope these actors have within a single campaign. At present, therefore, there is considerable ambiguity about the type of data-driven campaigns that exist. This suggests the urgent need for new empirical analysis that explores the practice of data-driven campaigning in different organisations and different countries. By operationalising the categories proposed here and using methods including interviews, content analysis and data subject access requests, I argue that it is possible to build up a picture of who is using what data how.

Of particular interest is the potential to use these frameworks to generate comparative insights into data-driven campaigning practice. At present studies of data use have tended to be focused on one country, but in order to understand the scope of data-driven campaigning it is necessary to map the presence of different practices. This is vital because, as previous comparative electoral research has revealed, the legal, cultural and institutional norms of different countries can have significant implications on campaigning practices. In this way it would be expected that a country such as Germany with a history of strong data protection law would exhibit very different data-driven campaigning practices to a country such as Australia. In a similar way, it would be expected that different institutional norms would lead a governmental organisation, charity or religious group to use data differently to parties. At present, however, the lack of comparative empirical data makes it difficult to determine what influences the form of data-driven campaigning and how different regulatory interventions affect campaigning practices. This framework therefore enables such comparative analysis, and opens the door to future empirical and theoretical work.

One particularly valuable aspect of this approach is the potential to use these questions and categories to contribute to existing debates around data-driven campaigning and democracy. Throughout the discussion, I have argued that many commentators have voiced concerns. These relate variously to privacy, the inclusivity of political debate, misinformation and disinformation, political finance, external influence and manipulation, transparency and social fragmentation (for more see Zuiderveen Borgesius et al., 2018, p. 92; Chester and Montgomery, 2017, p. 8; Dobber et al., 2017, p. 2; Hersh, 2015, p. 207; Kreiss and Howard, 2010, p. 11; International IDEA, 2018, p. 19). Such concerns have led to calls for regulation, and, as detailed above, many national governments, regulators and international organisations have moved to make a response. And yet, before creating new regulations and laws, it is vital for these actors to possess accurate information about how precisely data-driven campaigning is being conducted, and to reflect on which democratic ideals these practices violate or uphold. Data-driven campaigning is not an inherently problematic activity, indeed, it is an established feature of democratic practice. However, our understanding of the acceptability of this practice will vary dependent on our understanding of who, what and how data is being used (as whilst some practices will be viewed as permissible, others will not). This makes it important to reflect on what is happening and how prevalent these practices are in order to determine the nature and urgency of any regulatory response. Importantly, these insights need to be gathered in the specific regulatory context of interest to policy makers, as it should not be presumed that different countries or institutions will use data in the same way, or indeed have the same standards for acceptable democratic conduct.

The frameworks presented in this article therefore provide an important means by which to consider the nature, prevalence and implications of data-driven campaigning for democracy and can be operationalised to produce vital empirical insights. Such data and conceptual clarification together can ensure that any reaction to data-driven campaigning takes a consistent, considered approach and reflects the practice (rather than the possibility) of this activity. Given, as a report from Full Fact (2018, p. 31) makes clear that there is a danger of “government overreaction” based on limited information and self-evident assumptions (Ostrom, 2000) about how campaigning is occurring, it is vital that such insights are gathered and utilised in policy debates.

Conclusion

This article has explored the phenomenon of data-driven campaigning. Whilst receiving increased attention over recent years, existing debate has tended to focus on the extent to which this practice can be found. In this article, I present an alternative approach, seeking to map the diversity of data-driven campaigning practices to understand the different ways in which data can and is being used. This has shown that far from being characterised by uniform data-driven campaigning practices, data-use can vary in a number of ways.

In classifying variations in who is using data in campaigns, what the sources of campaign data are, and how data informs campaign communication, I have argued that there are diverse practices that can be acceptable to different actors to different degrees. At an immediate level, there is a need to gain greater understanding of what is happening within single campaigns and how practices vary between different political parties around the globe. More widely, there is a need to reflect on the implications of these trends for democracy and the form that any regulatory response may need to take. As democratic norms are inherently contested, there is no single roadmap for how to make a response, but the nature of any response will likely be affected by our understanding of who, what and how data is being utilised. This suggests the need for new conceptual and empirical understanding of data-driven campaigning practices amongst both academics and regulators alike.

Introduction

After 21 years of military dictatorship, followed by a short period of political instability, the political scene in Brazil was dominated by two major parties that, between them, held the presidency until 2018. Both were moderate with a large membership base, had many representatives in Congress and received constant coverage in the legacy media as representatives of a more westernized democratic process. However, in the 2018 elections, the country elected as president a niche congressman, Jair Bolsonaro, a member of a small party (PSL) with almost no registered supporters, who had been relatively unknown until some four years earlier, when he started to make appearances on popular and comic TV shows, on which he combined extremist rhetoric with praise for the military dictatorship. Bolsonaro’s election surprised local and international politicians and intellectuals, in part because his campaign lacked a traditional political structure, but mainly because of his radical rhetoric, which frequently included misogynistic and racist statements that would be sufficient to shake the public image of any candidate anywhere in the world (Lafuente, 2018), but which were even more shocking in a country marked by social inequalities and racial diversity.

One of the hypotheses for Bolsonaro's electoral success is that his campaign and some supporters developed a specific communication strategy based on the intense use of social media, in which the use of WhatsApp chat groups, micro-targeting and disinformation to reach different groups of voters had significant relevance. Albeit not always in a coordinated way, several platforms were used: YouTube, with videos of alt-right political analyses, lectures about "politically incorrect" history and amateur journalism; Facebook, with its pages and groups for distributing content and memes; and Twitter/Instagram, especially as sites for posting political and media content (the last three platforms mentioned were also widely used by the candidate himself to post messages and live videos on his official profile). Davis and Straubhaar (2019) point out that “legacy media, popular right-wing Facebook groups, and networks formed around the communication network WhatsApp fueled “antipetismo” 1, stressing that WhatsApp was particularly instrumental to cement Bolsonaro’s victory. Addressing the emergence of what she calls “digital populism”, Cesarino (2019) discusses the formation of a polarised digital bubble largely anchored on WhatsApp chat groups.

We focus our analysis on WhatsApp, examining the use of encrypted groups containing up to 256 members reflecting specific interests (religious, professional, regional, etc.). Smartphones and WhatsApp “were not as extensively available in Brazil during the previous [2014] presidential election” (Cesarino, 2019), and we aim to show that WhatsApp has technical specificities and was susceptible to an electoral strategy that justifies particular attention. Several media reports stress the role played by WhatsApp in the 2018 elections. Analysing the Brazilian elections, our goal is also to contribute to the decoupling of “research from the US context” and help with the understating of “macro, meso and micro level factors that affect the adoption and success of political micro-targeting (PMT) across different countries” (Bodó, Helberger, & Vreese, 2017). The Global South in general has been associated with an increase in “computational propaganda” in recent years (Woolley & Howard, 2017; Bradshaw & Howard, 2018).

Because of its closed, encrypted architecture, which restricts visibility for researchers and public authorities, the relative anonymity afforded by the use of only telephone numbers as identifiers by groups administrators, and the limited number of members in a group, which favours audience segmentation, WhatsApp is the platform that poses the greatest challenges for those investigating information dynamics during elections. It has also been shown that, because of these characteristics, WhatsApp played a crucial role in the spread of misinformation during the Brazilian 2018 elections (Tardáguila, Benevenuto, & Ortellado, 2018; Davis & Straubhaar, 2019; Junge, 2019) and in the development of a disinformation strategy that not only was on the edge of the law (Melo, 2019; Avelar, 2019; Magenta, Gragnani & Souza, 2018) but also exploited features of the platform’s architecture that help to render coordinated strategies invisible and favour group segmentation.

Although widely used in Brazil as a means of communication, only recently has WhatsApp use been tackled as a research subject in elections. Moura and Michelson (2017) evaluated its use as a tool for mobilising voters, and Resende et al. (2019) conducted groundbreaking research on the dynamics of political groups on WhatsApp. However, little has been said on the interrelation between a historical context and its interplay with new platforms, media technologies and novel campaign strategies that rely on surveillance. In this sense, surveillance shows up as a new mode of power (Haggerty & Ericson, 2006) with direct impact on the election process, with implications for democracy (Bennett, 2015). This article challenges the idea that political micro-targeting (PMT) is elections as usual (Kreiss, 2017), showcasing its connection with disinformation practices and a process of political radicalisation in a specific empirical context, and stresses that PMT functions as part of an (mis)information ecosystem.

In this article, we discuss the Brazilian institutional, political and media context that paved the way for Jair Bolsonaro to become president in what was an atypical election result that surprised the vast majority of political analysts. We describe and discuss the use of a particular social media platform, WhatsApp, which instead of functioning as an instant messaging application was weaponised as social media during the elections. Based on an analysis of a sample of the most widely distributed images on this platform during the month immediately prior to the first round of the elections, in which Bolsonaro won 46.03% of the valid votes, we argue that messages were partially distributed using a centralised structure, built to manage and to stimulate members of discussion groups, which were treated as segmented audiences. Our ambition is to correctly address a specific and concrete use of data in an electoral campaign and to avoid any type of hype around PMT or data-driven strategies (Bodó et al., 2017; Baldwin-Philippi, 2017). In this case, platforms and data are not used as much to scientifically inform broad campaign strategies (Issenberg, 2012), but are more connected to disinformation/misinformation processes.

The Brazilian context and the rise of Bolsonaro as a political and media “myth”

Brazil is a federal republic with relatively independent states but considerable power centralised in the federal executive and legislatures. The regime is presidential, and an election is held every four years for president (who may run once for re-election), state governors, state congressmen and federal deputies and senators; federal deputies and senators represent their states in two legislative houses, the Chamber of Deputies and the Federal Senate. The president of the Chamber of Deputies is the second in line to the presidency (after the vice president, who is elected on the same slate as the president), and it is the responsibility of the legislature to investigate and, if necessary, try the president of the republic for “crimes of responsibility”, which can lead to impeachment and removal from office.

Voting is compulsory and electors do not need to be members of a party to vote. Failure to vote is punished with a small fine (approximately $2.00 USD). Abstentions are typically in the region of 20%; in the last elections the figure reached 20.3%, the highest in the last 20 years.

Federal and state elections are held every four years, and municipal elections occur in between. Candidates must be members of legally constituted political parties to stand for election. Elections for executive office are held in two rounds, and the two candidates with the most votes in the first round compete in a run-off unless one of them has 50% + 1 of the valid votes in the first round.

The political system is extremely fragmented (Nascimento, 2018), and there is frequent switching between parties, particularly between smaller associations. In general, congressional representatives who belong to these smaller associations are known as the “lower clergy” and form a bloc characterised by clientelism (Hunter & Power, 2005) and cronyism. Armijo & Rhodes (2017) argue that “cronyism may tip into outright corruption, or direct payments by private actors for preferential treatment by state officials”, pointing out that Brazilian elections are very expensive and were, at least until recently, heavily funded by the private sector. Parties form coalitions to take part in the elections, and the seats in the Chamber of Deputies are distributed according to the number of votes the coalitions or parties receive and are then allocated within the coalitions according to the number of votes each candidate receives. In his almost 30 consecutive years as a federal deputy, Jair Bolsonaro was known as a member of the “lower clergy” and was affiliated with no less than nine different political parties. A former member of the armed forces, when elected congressman, his votes came mainly from efforts that benefited that sector (Foley, 2019), and also from his criticism of human rights and his position in favour of harsher criminal laws and more vigorous police action.

Brazilian elections historically were financed with a combination of public funds and limited private donations from individuals and companies 2. Public funds are shared between the parties mainly according to the number of seats they have in Congress. Parties are also entitled to television and radio airtime, for which broadcasting companies receive tax exemptions. Political advertisements paid for by parties are prohibited.

Radio and TV airtime was traditionally considered one of the most important factors in a presidential election. However, in the last election, the candidate with the most time in these media ended up in fourth position (Geraldo Alckmin, PSDB, with 44.3% of the airtime), while Bolsonaro, who had little more than 1% of the total airtime, won the first round. Fernando Haddad (PT), who came second, had 19.1% of the airtime (Ramalho, 2018; Machado, 2018).

The Brazilian broadcasting system is concentrated in the hands of a few family groups and, more recently, an evangelical minister from a non-denominational church (Davis & Straubhaar, 2019). These groups own TV and radio networks, newspapers and websites around the country. The editorial line is economically conservative, although some of the companies (e.g., Rede Globo) have a more liberal attitude in terms of customs (Joyce, 2013).

An appreciation of the Brazilian political scene is also important to understand Bolsonaro’s rise. From the middle of the second term in office of Luis Inácio Lula da Silva, when the first accusations of corruption involving members of the government appeared, much of the domestic press became more critical in its tone and adopted a more aggressive posture clearly aligned with the opposition. Even when the government was at the height of its popularity, during Lula’s second mandate, political commentators who were either clearly anti-Workers’ Party (PT) or more scathing in their criticism tended to be in the majority in much of the media (Carvalho, 2016).

This change generally helped to create quite a negative feeling toward the PT, primarily among the middle classes, the main consumers of this political journalism. This feeling was to intensify after Dilma Rousseff was re-elected by a small margin, with voters being clearly divided by class and region. Roussef’s votes came mainly from the lower income classes and from the north-east of the country (Vale, 2015). The figure below by Vale (2015) shows the distribution of votes per state in the second round of the 2014 presidential election (% of vote).

Figure 1: Distribution of votes in the 2014 Brazilian presidential election (per state) (Vale, 2015).

At the same time, the consequences of the 2008 global crisis began to be felt across the country. Until Rousseff’s first mandate, the country had managed to balance the public accounts. However, when the government boosted internal consumption by granting tax exemptions to certain industrial and infrastructure sectors by the end of her second term, public debt surged and became the target of strong criticism by financial commentators. At this point the PT lost most of support it might have had among the middle classes and in the industrialized south and south-east. Later, when the second major political scandal during the rule of the PT was to break — the discovery of a corruption scheme involving mainly allies of the PT but also marked by the participation and/or connivance of the PT itself — a political process was set in motion that included huge street demonstrations with extensive coverage and encouragement by the media. The government began to lose its support in Congress, a support which had never been very ideologically solid given the strength of the “lower clergy”. The end result was that Dilma was impeached for a “crime of responsibility” on controversial grounds.

The political climate that ensued, however, did not help to restore peace. Accusations of corruption were levelled against a wide range of parties, including those that actively supported impeachment, such as the PT’s long-standing adversary, the Brazilian Social Democracy Party (PSDB). Nowadays a centre-right party, the PSDB had held the presidency from 1995 to 2002 and had faced the PT in run-offs in all the presidential election since then. Historically a centre-left party, the PSDB had moved toward neoliberalism, although its membership later included more conservative candidates, with some in favour of more vigorous police action and tougher laws, as well as a morally conservative agenda. This slow ideological transformation was initially welcomed by part of the electorate, particularly middle and upper-class electors, but later proved to be insufficient to ensure their victory.

Following Rousseff’s impeachment, extensive reporting of the widespread involvement of political parties in the Petrobras scandal appears to have helped produce a profound aversion to politicians as a whole. The Petrobras scandal was revealed in 2014 by Operation Car Wash but the investigations lasted for another five years and were the subject of high media attention for the whole period. Investigators revealed that a “cartel of construction companies had bribed a small number of politically appointed directors of Petrobras in order to secure a virtual monopoly of oil-related contracts” (Boito & Saad-Filho, 2016).

In parallel with this, Jair Bolsonaro became increasingly well known among a large swathe of the public. As previously mentioned, although he had been in politics for almost 30 years, Bolsonaro remained a minor figure until 2011, known only in his state, Rio de Janeiro, and popular only among the military (the armed forces as well as police and firefighters, who are members of the military in Brazil) and a small niche of electors in favour of repressive police action. In 2011, however, Bolsonaro took part in CQC (the acronym for “Whatever It Takes” in Portuguese), a TV programme that combines humour and political news, and answered questions sent in by members of the public. At the time, one of the presenters classified him as “polemical” and another said he had not the slightest idea who that congressman was. The objective appeared to be to ridicule him, and the programme featured humorous sound effects. The then congressman’s answers ranged from homophobic to racist, and he praised the period of military dictatorship in Brazil. The interview sparked controversy, and Bolsonaro was invited to appear on the programme again the following week. He became a common attraction on CQC and other programmes that also explore the impact of the politically incorrect.

The congressman was gradually given more airtime on CQC and taken more seriously, while at the same time his legitimacy increased because of his popularity with the audience. A few months before Bolsonaro was elected, a former CQC reporter recalled with regret the visibility the programme had given the congressman. She said they were “clueless that a good chunk of the population would identify with him, with such a vile human being”, admitting they “played a part” in the process. (2018, April 4)

In addition to CQC, other free-to-air TV programmes gave the then federal deputy airtime. In a recent study, Santos (2019) shows how, since 2010, Bolsonaro made an increasing number of appearances on the free-to-air TV programmes with the largest audiences. CQC helped particularly to bring Bolsonaro closer to a younger audience, together with the programme Pânico na Band, which also takes advantage of politically incorrect humor and created a special segment on the congressman, with 33 episodes each 9 minutes long in 2017.

In social media, Bolsonaro and his opinions became the subject of increasing debate and were vigorously rejected by the left but adopted as a symbol of the politically incorrect by sectors of the non-traditional right. The figure of a talkative, indiscreet congressman became a symbol for a subculture of youth groups—similar to those who frequent websites such as 4chan (Nagle, 2017) — for whom he became known as “the Myth”, a mixture of iconoclasm, playful humour and conservatism. This connection with the politically incorrect was exploited by the administrators of the congressman’s Facebook page since it was set up in June 2013 (Carlo and Kamradt, 2018).

WhatsApp and the spread of misinformation / disinformation

The political year of the last presidential election, 2018, was unusually troubled. Involved in various court cases, the then leader in the polls, former president Lula, was arrested in early April. The party was adamant that he should run for election even though there was little chance of his being allowed to do so by the legal authorities. Without Lula, Bolsonaro was ahead in the polls from the start — with a lead that varied but was always at least 20% — although he was considered a passing phenomenon by most analysts and was expected to disappear as soon as the TV and radio campaigns started because he had very little airtime compared with the candidates for the traditional parties.

Another important event that helps describe the scenario in which WhatsApp had a significant role in politics is the strike/lockout organised by truck drivers in May 2018. Dissatisfied with the almost daily variations in fuel prices, which had begun to be adjusted in line with the US dollar and the price of oil, self-employed truck drivers and logistic companies started a protest that ended up bringing the country to a quite long and economically harmful standstill (Demori & Locatelli, 2018). Using mainly WhatsApp, drivers organised roadblocks on the main highways, causing severe disruption to the supply of goods, including fuel and food (Rossi, 2018). Radical right-wing associations infiltrated these online groups, praising militarism as a solution for the country and sometimes clamouring for military intervention to depose the executive, legislature and judiciary.

Interested in understanding how fake news was spread by political groups on WhatsApp, Resende et al. (2019) collected data of two periods of intense activity in Brazil: the truck drivers’ strike and the month prior to the first round of the elections. They sampled chat groups that did not necessarily have any association with a particular candidate, were open to the public (although the administrators could choose who was accepted or removed), having links to join in shared in websites or social networks, and could be found using the URL chat.whatsapp.com. The groups were chosen by the match of that URL with a dictionary containing the name of politicians, political parties, as well as words associated with political extremism. In all, they analysed 141 groups during the truck drivers’ strike and 364 during the elections. The results show that 2% of the messages shared in these groups during the elections were audio messages, 9% were videos, 9% contained URLs to other sites and 15% were images. The other 65% were text messages with no additional multimedia content.

Resende et al. (2019) also developed an automated method for determining whether the shared images in the analysed groups had already been reviewed and rejected by fact-checking services. To that selection they added 15 more that were previously identified by the Brazilian fact-checking agency, Lupa, as misinformation. Totalling 85 images that contained misinformation, they found that these were shared eight times more often than other 69,590 images, which were truthful or had not been denounced for checking by any independent agency.

Although the total number of images labeled as misinformation is relatively low - only 1% of the total number of images shared - these images were seen in 44% of the groups monitored during the election campaign period, which means they have a long reach. Upon investigation of such images, these researchers identified the groups in which the images appeared first, and remarked that a small number of groups seemed to account for dissemination of a large amount of images with misinformation. In our view, this fact indicates a more centralised and less distributed dissemination structure.

Another fact revealing a dynamic of relatively centralised dissemination is that the "behaviour" of image propagation including disinformation (which are images deliberately produced and/or tampered with) is significantly different from unchecked images. Comparing the structure of propagation of these two groups, particularly as to the time these images appeared on the Web and on WhatsApp and vice-versa, the authors noticed that 95% of the images with unchecked content were posted first on the Web and then in monitored WhatsApp groups. Only 3% of these images made the opposite route, and 2% appeared both on the Web and on WhatsApp on the first day. In contrast, only 45% of the images with misinformation appeared first on the Web, 35% were posted first on WhatsApp and 20% were shared in both platforms on the same day. According to the authors, this suggests "that WhatsApp acted as a source of images with misinformation during the election campaign period" (Resende et al., p.9.). Considering that an image with disinformation is deliberately produced and tampered with, the fact that WhatsApp is its first source of sharing in a much higher percentage than images with unchecked content (35% in the first case versus 2% in the second case) is one more element indicating a relatively centralised and not fully spontaneous organisation of propagation of this type of content.

Disinformation in WhatsApp groups

As to the contents of images with disinformation, they reproduce many of the elements that were key in the rise of Bolsonaro and, later, during his election campaign. In this section we will analyse the top eight most shared images with disinformation in the month before the first round, using the same groups monitored by Resende et al. (2019) as our source. Our analysis is based on investigative work developed by the Agência Lupa and Revista Época, in partnership with the research project "Eleições sem Fake" (Resende et al. 2019; Marés, Becker and Resende, 2018). The news piece points out that none of the eight images analysed mentions the presidential candidates directly. All of them refer to topics that reinforce beliefs, perspectives and feelings that shaped the ideological base of Jair Bolsonaro's campaign. Anti-PT-ism, strongly boosted by the legacy media over the last few years, was one of the pillars of Bolsonaro's campaign, and it is the content of the most shared image with disinformation in the monitored groups in the month before the first round. As we can see, Figure 2 is a photo-montage that inserts a photo of the young ex-president of Brazil, Dilma Rousseff, beside the ex-president of Cuba, Fidel Castro.

Figure 2: First most shared disinformation image on WhatsApp.

At the time the original photo of Fidel Castro was taken by John Duprey for the “NY Daily News” in 1959, president Dilma Rousseff was only 11 years old. Therefore, it is clearly a tampered image that intends to associate the PT with communism and Castroism. Such association was recurrent among Bolsonaro supporters during his campaign and antipetismo appears directly in three out of the eight most shared images with misinformation over the time period under analysis.

Another image with clear anti-PT-ist content (the fourth most shared image in the monitored groups) is an alleged criminal record of ex-president Dilma Rousseff during the times of the military dictatorship, in which she would be accused of being a terrorist/bank robber (Figure 3). This record was never issued by any official agency of the military government and has the same format as the third most shared image, this time showing José Serra, current senator of the republic for the PSDB party 3.

Figure 3: Fourth most shared disinformation image on WhatsApp.

Lastly, the third image with direct anti-PT-ism content (the eighth most shared image in the monitored WhatsApp groups) is the reproduction of a graph with false information comparing consumption of families over the last five years of PT government at that time with the expenditure of the government itself (Figure 4). Contrary to what the graph shows, the consumption of families did not decrease; instead it grew 1.8% between 2011 and 2016, whereas expenditure of the public administration rose 3.1% during the period, and not 4% as the graph indicates.

Figure 4: Eighth most shared disinformation image on WhatsApp.

The second most frequent topic in the most shared images with misinformation is attacks to the rights of LGBTs and women, appearing in three out of the eight most shared images. This kind of content, although not directly antipetismo, denies rights that were symbolically associated to leftist parties by Bolsonaro's campaign.

The fifth and sixth most shared images link these rights to sexualisation of childhood and lack of respect for religious beliefs, as shown in figures 5 and 6, respectively. Moreover, in the context of their sharing via WhatsApp, such images were associated with Rede Globo, the largest commercial open television network in the country. In figure 5, the legend of the image (which is in fact a photo of Heritage Pride March in New York in 2015) reads: “People from Globo-Trash who do not support Bolsonaro!!!”. In figure 6, the image is shown with the sentence "Globo today". However, the image is a record of the Burning Man music festival that took place in the Desert of Nevada in 2011 in the US. It shows a man dressed as Jesus kissing Benjamin Rexroad, director of the “Corpus Christi" production. This image was published in O Globo newspaper at the time of the festival and not before the first round of the 2018 elections. The link of these images to the O Globo newspaper is part of a campaign of disqualification of the TV station with the same name, Rede Globo. Bolsonaro supporters’ strategy seems to be to legitimise the WhatsApp groups as more reliable sources of information than the legacy media. The Globo Network, historically linked to conservative economic and political interests in the country, was here associated with the propagation of hyper-sexualised and anti-Christian content.

The other image (figure 7), which is part of this thematic group against LGBT and women's rights, is a montage of photos of different protests in churches. In one of the protests, a couple has sex inside a church in Oslo, Norway, in 2011. In the second one, a woman defecates on the stairway of a church in Buenos Aires, capital of Argentina, which happened when Maurício Macri won the 2015 presidential election. The caption of the false image reads “Feminists invade church, defecate and have sex”, and is in clear opposition to the #EleNão movement. #EleNão (#NotHim) was a movement led by women that denounced Bolsonaro as misogynist, gathering thousands of people across Brazilian streets in the verge of the first round of the elections (Uchoa, 2018).

Figure 5: Fifth most shared disinformation image on WhatsApp.

Figure 6: Sixth most shared disinformation image on WhatsApp.

Figure 7: Seventh most shared disinformation image on WhatsApp.

Use of illegal tactics

In the second round of the elections, the newspaper Folha de S.Paulo managed to discover that businessmen had signed contracts of up to $3 million US each with marketing agencies specialised in automated bulk messaging on WhatsApp (Melo, 2018). The most famous of the businessmen who were accused owns a retail sector company, which could suggest that the marketing methods used in his company could also be used in politics. The practice is illegal as donations from companies were forbidden in the last election. Furthermore, the businessmen were alleged to have acquired user lists sold by agencies as well as lists supplied by candidates. This practice is also illegal as only the candidate’s user lists may be used. According to a report issued by Coding Rights (2018) involving interviews and document analyses of marketing agencies operating in Brazil, election campaigns in general combine a series of databases, associating public information, such as the Census, and data purchased from private companies such as Serasa Experian and Vivo (a telecom company owned by Telefónica). These databases include demographic data and telephone numbers (including the respective WhatsApp contacts).

According to Folha de S.Paulo’s article, the marketing agencies are able to generate international numbers, which are used by employees to get around restrictions on spam and to administer or participate in groups. Off the record statements obtained by the newspaper from ex-employees and clients of the agencies, reveal that the administrators used algorithms that segmented group members into supporters, detractors and neutral members and defined the content sent accordingly. The most active Bolsonaro supporters were also allegedly contacted so that they could create even more groups in favour of the candidate (Melo, 2018). In a different article, the newspaper noted that some of these groups behaved like a military organisation and referred to themselves as a “virtual army” in favour of the candidate. According to the newspaper, the groups are divided into “brigades”, “commands” and “battalions” and are formed mainly of youths, some under 18 years of age (Valente, 2018).

Investments on election campaigns were directed to several variations of digital advertising. Besides being less expensive, digital advertising can be an alternative to limited TV time, particularly for small political associations (Coding Rights, 2018). Digital campaigns included WhatsApp mainly because it is a platform with deep penetration in the population, considering, among other things, the practice of zero-rating policies. Zero-rated data refers to data that does not count toward the user’s data cap (Galpaya, 2017). Telecom operators commonly offer free use of WhatsApp for pre-paid plans, which are the ones most commonly contracted by lower classes. Even if the user doesn’t have any credits left for accessing the internet, they keep sending and receiving text and media content on their WhatsApp chat groups and from individual users. An accurate image is described by Luca Belli: “fact-checking is too expensive for the average Brazilian” (2018).

Rules approved for the Brazilian 2018 elections permitted candidates to buy advertisements on social media and to spread content using message platforms. However, WhatsApp does not offer micro-segmentation as a service, which would allow advertisements to be directed to a certain audience, like Facebook does. Marketing agencies ended up playing that role, not always using legally collected information on voters.

Audi & Dias (2018) reported that agencies in the business of political advertising use software that monitors different interest groups - not only those of political discussion - to prospect voters and supporters. The users are measured in terms of their mood and receptivity towards campaign messages. By doing so, these agencies manage to identify the ideal target population and the right time to send each type of content. According to the article, “messages that reach a diabetic 60-year old man from São Paulo State are different from those sent to a north-eastern woman who lives on minimum wage”. Audi & Dias (2018) had access to one of the software programmes used during the last elections in Brazil, WNL, a version used for the campaign of a non-identified politician. The software programme monitored and distributed contents in over 100 WhatsApp groups that ranged from diabetics discussion groups, soccer team supporters, Uber drivers, advertising of job vacancies and even workmates and neighbours.

Such segmentation was refined by the monitoring of reactions to contents posted, rated as positive, negative or neutral reactions. Users rated as positive keep receiving similar information in favour of the candidate. Those rated as neutral get mostly materials contrary to the opponent. Negative users start getting a more incisive treatment, receiving content that would tend to "target values dear to the person, such as family and religion, in an attempt to inflate rejection towards the candidate's competitor". By monitoring these reactions, users are segmented in individual files and then classified into groups according to specific topics - such as church, "gay kit", family, communism, weapons, privatisation, etc. Moreover, this software programme enables those that monitor the groups to collect and select keywords in order to discover specific interests: "For example, a patient with cancer speaks about his/her condition and treatment. The system collects these data and finds other people in similar conditions. They start to get content with the policies of the candidate for health, for example" (Audi and Dias, 2018).

It should be noted that this micro-segmentation and micro-targeting are integral to the way advertisements on platforms work. Facebook, for instance, announced some special transparency policies for political ads during the election period (Valente, 2018). However, due to the nature and architecture of WhatsApp, the visibility of content spreading strategies on a platform such as it is minimal, and this prevents users to realise that they are being the target of persuasion strategies. We will return to this issue in the conclusion, but it must be pointed out that using this platform for election campaigns is structurally questionable. If we consider only the methods used by AM4 Company, which openly worked in Jair Bolsonaro's campaign spreading contents to 1,500 WhatsApp groups, there are already reasons for concern, since such content is not explicitly distributed as part of an election advertising campaign. It is rather distributed as content shared by common users in groups with a supposedly symmetrical interaction structure. According to a statement of the company's founding partner: “what we do is curator-ship of contents created by supporters” (Soares, 2018). The company owner also stated that a series of content that fed 1,500 WhatsApp groups on a daily basis were part of the strategy of the company hired by PSL, which operated since the pre-campaign, to revert negative episodes in favour of Bolsonaro's campaign.

WhatsApp group dynamics

Before developing a formal research interest in the use of WhatsApp during the elections we started a participatory observation process at political chat groups on WhatsApp. Initially, we were interested in understanding how the app was being used by truck drivers for organising its protests. Later, we noticed that many groups we found were also occupied by radicals in favour of a return to the military dictatorship and supporters of Jair Bolsonaro. This helped us to understand the dynamics of those groups and how some more prominent members acted to manage the discussions or the posting of content.

Various groups were short-lived and rapidly replaced by other groups that were advertised in the “dying” groups. Until the election day, and some weeks before, we tried to follow the discussion of at least three groups on WhatsApp and two on Telegram. Some of the groups were occasionally invaded by users who posted pornographic content or advertised illegal services, such as pirate IP TV and false diplomas, cloned credit cards and counterfeit money. As observed in field work, in the case of one particular group on Telegram, the group became a pirate IP TV sales channel as soon as the elections were over.

At the end of the elections, it was observed that new groups were set up with a new mission: to act as direct communication channels between supporters of the new president. One of these went by the name of Bolsonews TV. There is little discussion in these groups and only a few members are responsible for almost all the content sent to the groups or forwarded from other groups. A frequently repeated claim is that one should not believe in the legacy media because it is controlled by communists and left-wing individuals; according to the people who send these messages, only some YouTubers, journalists, bloggers and politicians can be trusted. Before the elections, material from the legacy media that was highly critical of the PT was frequently shared, particularly if it was produced by commentators who were considered right wing. After the elections, when the criticism was aimed more at the new government, and even when it came from commentators who were considered right wing, this type of content became less common. Any critical comments in groups clearly identified with Bolsonaro led to the author being removed from the group and accusations that he/she was a supporter of the PT who had infiltrated the group. The telephone numbers of people accused of supporting the PT circulate regularly, and group moderators are warned to exclude these people from groups.

Analysing the flow of messages between political groups during the elections, Resende et al. (2018) identified characteristics that indicated the presence of clusters of groups with members in common. They constructed a graphical model of the relationship between members, which revealed a network of users who were associated because they shared images in at least one common group. “We note a large number of users blending together connecting to each other inside those groups. Most users indeed form a single cluster, connecting mostly to other members of the same community. On the other hand, there are also a few users who serve as bridges between two or more groups linked by multiple users at the same time. Furthermore, a few users work as big central hubs, connecting multiple groups simultaneously. Lastly, some groups have a lot of users in common, causing these groups to be strongly inter-connected, making it even difficult to distinguish them” (2019, p. 6). This would suggest that WhatsApp is working not so much as an instant messaging app but as a social network, like Twitter and Facebook. Other evidence, as shown above, allows us to conclude that these groups may be centrally managed, although this is invisible to the ordinary user.

Conclusion

Commenting on the use of micro-targeting in campaigns, Kreiss points out that it is “likely most effective in the short run when campaigns use them to mobilize identified supporters or partisans” (2017). It seems to be the case of what happened in Brazil in the 2018 elections, in which a candidate was able to tap into a conservative sentiment, harnessing it against the progressive field.

Even though it is not possible to fully confirm the hypothesis that WhatsApp has been used as an effective tool to direct messages to micro-segmented voters, we have shown that the campaign of Jair Bolsonaro used the app to deliver messages (and disinformation) to exacerbate political feelings present in the political debate of the legacy media - antipetismo (Davis & Straubhaar, 2019) - and add to them much more conservative elements in the moral field (anti-feminism and anti-LGBT), which brought back topics from the times of the military dictatorship (anti-communism). Beyond the effects on the left, the radicalisation promoted by Bolsonaro’s campaign was able to neutralise any other candidate on the centre, even on the centre-right, associating them with the establishment and with the notion of a corrupt political system. In the symbolic assemblage (Norton, 2017) that was formed, the elected candidate ended up representing the most effective answer against the political system, although many voted for him for different reasons. In a similar fashion to Trump, Bolsonaro “ran an insurgent campaign that called for shaking up the system” (Kivisto, 2019, p. 212)

There is enough evidence that the WhatsApp chat groups feature was weaponised by Bolsonaro supporters. Although WhatsApp does not provide a service for micro-targeting audiences, there is evidence that third party companies, dedicated to non-political marketing campaigns, provided that kind of service in the context of elections, sometimes using illegal databases. There are reports that Haddad’s campaign has also used WhatsApp to deliver messages to voters (Rebello, Costa, & Prazeres, 2018). However, as the sample collected by Resende et al. (2019) suggests, there is no evidence that the left coalition has employed the same tactics as Bolsonaro’s in secretly managing multiple WhatsApp chat groups.

Among the many problems involved in the use of a platform like WhatsApp in an election campaign, we would like to point out one in particular: the invisibility of the actors that produce, monitor, distribute and/or direct the contents viewed and/or shared by most users. The current architecture of the platform does not allow, once appropriated for purposes of election campaigns and micro-targeting, users to notice or become aware that they are being monitored and managed. Writing on voter surveillance in Western societies, Bennett reminds us that “surveillance has particular, and somewhat different, effects depending on whether we are consumers, employees, immigrants, suspects, students, patients or any number of other actors” (2015, p. 370). The case of use of WhatsApp in Brazilian elections shows how a surveillant structure was built on top of a group message service that allegedly uses cryptography to protect its user's privacy.

Resende et al. (2019) characterised a network structure of the monitored WhatsApp groups that evidence a coordinated activity of some members. There are no clear means for regular WhatsApp chat group members to notice if they are being monitored or laterally surveilled by other group members or even other second-hand observers outside the groups. Studies on perception and experience of Facebook users show that when they notice that a post is sponsored they tend to be less persuaded than when exposed by a regular post by a friend or acquaintance (Kruikemeier et al., 2016). But unlike Facebook, where users can have a huge number of connections although a great part of them may not be very close or not close at all, most contacts of WhatsApp users are closer to a personal circle, thus setting a relationship of trust with the content received. Writing on family WhatsApp chat groups in Brazil, Benjamin Junge classifies them as both “public” of sorts, an “open space for the sharing of information and opinions” (2019, pp. 13-14), and closed in the strict sense, because they are only accessible to family members. Although this trust-based relation may be transformed when the user is a member of larger groups, the experience of proximity and connection with the members of a group is bigger than, for instance, among Facebook friends and Twitter followers. WhatsApp favours a stronger relationship of trust between group members and the content shared, which implies that is a more susceptible field for the spread of misinformation. Cesarino (2019a) posits “trust” as one of the affordances of the WhatsApp platform, affirming that most of the political content forwarded to its users during the 2018 election was pushed by friends or family members.

Possible asymmetries of information, persuasion tactics and/or influence strategies within chat groups are rather hard to detect. In countries like Brazil, this condition is reinforced by the impossibility that many users have to reach beyond the platform to check information shared, something that might provide a context or additional information about the contents circulating in the platform. With the zero-rating plans offered by telecom companies, users are subject to tariffs that they cannot afford if they seek other sources of information. This perceptive confinement is particularly worrying in a context of the wide dissemination of disinformation, just like what happened in the 2018 election period, since most users are not only unaware of the authorship of the contents that reach them, but also they cannot reasonably check and verify such contents. The 'near-sighted" environment (in fact the most appropriate eye disorder here would be loss of peripheral sight) of WhatsApp is also favoured by its one-to-one communication structure, which prevents side visibility, transversal or common visibility in the platform. The lack of a common field of visibility would not be a problem if WhatsApp restricted its stated or projected technical functionality - that of being an instant messenger. However, when the tool begins to function as a typical social network - as stated by Resende et al. (2019), and starts to be massively appropriated for political campaigns, it is critical to have more symmetric relationships of visibility, as well as the possibility to build a common visible field that can be debated, examined and audited.

At least since the 2014 elections and especially after the contested impeachment of President Dilma Rousseff (PT), Brazil lives in a period of political and institutional instability. Recently leaked messages exchanged by prosecutors and judges involved in the investigation of corruption scandals help to draw a picture of a justice system contaminated by political goals (Fishman et al., 2019). That struggle certainly played a role in the ineffectiveness of the electoral legislation to curb the illegal use of WhatsApp in the 2018 elections. We described in this article many of the illegalities that surrounded the electoral process. In 2019, the Brazilian Congress approved a data protection law in many aspects compliant with the EU’s General Data Protection Regulation (GDPR) that can help to strengthen the fairness of future elections (if and when the country can restore its political and institutional normalcy).

However, as we hope to have exposed here, there is a complex dynamic between the legacy media and what is created and shared by political actors and supporters. Much of the dis-informative content we have analysed was produced having as background a radicalisation trend noticed on the legacy media. The fact that the means of communication in Brazil are highly concentrated in the hands of a few groups and lacks political diversity certainly played an important role in paving a way for political radicalisation. Zero-rating policies that fuels the popularity of one specific platform (WhatsApp) and curbs users from accessing a full functioning internet obviously are a practical impediment for a voter that could be educated to adequately research and check the news stories they receive.

Introduction

Did the Holocaust really happen? In December 2016, Google’s search engine algorithm determined the most authoritative source to answer this question was a neo-Nazi website peddling holocaust denialism (Cadwalladr, 2016b). For any inquisitive user typing this question into Google, the first website recommended by Search linked to an article entitled: “Top 10 reasons why the Holocaust didn’t happen”. The third article “The Holocaust Hoax; IT NEVER HAPPENED” was published by another neo-Nazi website, while the fifth, seventh, and ninth recommendations linked to similar racist propaganda pages (Cadwalladr, 2016b). Up until Google started demoting websites committed to spreading anti-Semitic messages, anyone asking whether the Holocaust actually happened would have been directed to consult neo-Nazi websites, rather than one of the many credible sources about the Holocaust and tragedy of World War II.

Google’s role in shaping the information environment and enabling political advertising has made it a “de facto infrastructure” for democratic processes (Barrett & Kreiss, 2019). How its search engine algorithm determines authoritative sources directly shapes the online information environment for more than 89 percent of the world’s internet users who trust Google Search to quickly and accurately find answers to their questions. Unlike social media platforms that tailor content based on “algorithmically curated newsfeeds” (Golebiewski & boyd, 2019), the logic of search engines is “mutually shaped” by algorithms — that shape access — and users — who shape the information being sought (Schroeder, 2014). By facilitating information access and discovery, search engines hold a unique position in the information ecosystem. But, like other digital platforms, the digital affordances of Google Search have proved to be fertile ground for media manipulation.

Previous research has demonstrated how large volumes of mis- and disinformation were spread on social media platforms in the lead up to elections around the world (Hedman et al., 2018; Howard, Kollanyi, Bradshaw, & Neudert, 2017; Machado et al., 2018). Some of this disinformation was micro-targeted towards specific communities or individuals based on their personal data. While data-driven campaigning has become a powerful tool for political parties to mobilise and fundraise (Fowler et al., 2019; Baldwin-Philippi, 2017), the connection between online advertisements and disinformation, foreign election interference, polarisation, and non-transparent campaign practices has caused growing anxieties about its impact on democracy.

Since the 2016 presidential election in the United States, public attention and scrutiny has largely focused on the role of Facebook in profiting from and amplifying the spread of disinformation via digital advertisements. However, less attention has been paid to Google, who, along with Facebook, commands more than 60% of the digital advertising market share. At the same time, a multi-billion-dollar search engine optimisation (SEO) industry has been built around understanding how technical systems rank, sort, and prioritise information (Hoffmann, Taylor, & Bradshaw, 2019). The purveyors of disinformation have learned to exploit social media platforms to engineer content discovery and drive “pseudo-organic engagement”. 1 These websites — that do not employ professional journalistic standards, report on conspiracy theory, counterfeit professional news brands, and mask partisan commentary as news — have been referred to as “junk news” domains (Bradshaw, Howard, Kollanyi, & Neudert, 2019).

Together, the role of political advertising and the matured SEO industry make Google Search an interesting and largely underexplored case to analyse. Considering the importance of Google Search in connecting individuals to news and information about politics, this paper examines how junk news websites generate discoverability via Google Search. It asks: (1) How do junk news domains optimise content, through both paid and SEO strategies, to grow discoverability and grow their website value? (2) What strategies are effective at growing discoverability and/or growing website value; and (3) What are the implications of these findings for ongoing discussions about the regulation of social media platforms?

To answer these questions, I analysed 29 junk news domains and their advertising and search engine optimisation strategies between January 2016 and March 2019. First, junk news domains make use of a variety of SEO keyword strategies in order to game Search and grow pseudo-organic clicks and grow their website value. The keywords that generated the highest placements on Google Search focused on (1) navigational searches for known brand names (such as searches for “breitbart.com”) and (2) carefully curated keyword combinations that fill so-called “data voids” (Golebiewski & Boyd, 2018), or a gap in search engine queries (such as searches for “Obama illegal alien”). Second, there was a clear correlation between the number of clicks that a website receives and the estimated value of the junk news domains. The most profitable timeframes correlated with important political events in the United States (such as the 2016 presidential election, and the 2018 midterm elections), and the value of the domain increased based on SEO optimised — rather than paid — clicks. Third, junk news domains were relatively successful at generating top-placements on Google Search before and after the 2016 US presidential election. However, their discoverability abruptly declined beginning in August 2017 following major announcements from Google about changes to its search engine algorithms, as well as other initiatives to combat the spread of junk news in search results. This suggests that Google can, and has, measurably impacted the discoverability of junk news on Search.

This paper proceeds as follows: The first section provides background on the vocabulary of disinformation and ongoing debates about so-called fake news, situating the terminology of “junk news” used in this paper in the scholarly literature. The second section discusses the logic and politics of search, describing how search engines work and reviewing the existing literature on Google Search and the spread of disinformation. The third section outlines the methodology of the paper. The fourth section analyses 29 prominent junk news domains to learn about their SEO and advertising strategies, as well as their impact on content discoverability and revenue generation. This paper concludes with a discussion of the findings and implications for future policymaking and private self-regulation.

The vocabulary of political communication in the 21^st century

“Fake news” gained significant attention from scholarship and mainstream media during the 2016 presidential election in the United States as viral stories pushing outrageous headlines — such as Hillary Clinton’s alleged involvement in a paedophile ring in the basement of a DC pizzeria — were prominently displayed across search and social media news feeds (Silverman, 2016). Although “fake news” is not a new phenomenon, the spread of these stories—which are both enhanced and constrained by the unique affordances of internet and social networking technologies — has reinvigorated an entire research agenda around digital news consumption and democratic outcomes. Scholars from diverse disciplinary backgrounds — including psychology, sociology and ethnography, economics, political science, law, computer science, journalism, and communication studies — have launched investigations into circulation of so-called “fake news” stories (Allcott & Gentzkow, 2017; Lazer et al., 2018), their role in agenda-setting (Guo & Vargo, 2018; Vargo, Guo, & Amazeen, 2018), and their impact on democratic outcomes and political polarisation (Persily, 2017; Tucker et al., 2018).

However, scholars at the forefront of this research agenda have continually identified several epistemological and methodological challenges around the study of so-called “fake news”. A commonly identified concern is the ambiguity of the term itself, as “fake news” has come to be an umbrella term for all kinds of problematic content online, including political satire, fabrication, manipulation, propaganda, and advertising (Tandoc, Lim, & Ling, 2018; Wardle, 2017). The European High-Level Expert Group on Fake News and Disinformation recently acknowledged the definitional difficulties around the term, recognising it “encompasses a spectrum of information types…includ[ing] low risk forms such as honest mistakes made by reporters…to high risk forms such as foreign states or domestic groups that would try to undermine the political process” (European Commission, 2018). And even when the term “fake news” is simply used to describe news and information that is factually inaccurate, the binary distinction between what is true and what is false has been criticised for not adequately capturing the complexity of the kinds of information being shared and consumed in today’s digital media environment (Wardle & Derakhshan, 2017).

Beyond the ambiguities surrounding the vocabulary of “fake news”, there is growing concern that the term has begun to be appropriated by politicians to restrict freedom of the press. A wide range of political actors have used the term “fake news” to discredit, attack, and delegitimise political opponents and mainstream media (Farkas & Schou, 2018). Certainly, Donald Trump’s (in)famous use of the term “fake news”, is often used to “deflect” criticism and to erode the credibility of established media and journalist organisations (Lakoff, 2018). And many authoritarian regimes have followed suit, adopting the term into a common lexicon to legitimise further censorship and restrictions on media within their own borders (Bradshaw, Neudert, & Howard, 2018). Given that most citizens perceive “fake news” to define “partisan debate and poor journalism”, rather than a discursive tool to undermine trust and legitimacy in media institutions, there is general scholarly consensus that the term is highly problematic (Nielsen & Graves, 2017).

Rather than chasing a definition of what has come to be known as “fake news”, researchers at the Oxford Internet Institute have produced a grounded typology of what users actually share on social media (Bradshaw et al., 2019). Drawing on Twitter and Facebook data from elections in Europe and North America, researchers developed a grounded typology of online political communication (Bradshaw et al., 2019; Neudert, Howard, & Kollanyi, 2019). They identified a growing prevalence of “junk news” domains, which publish a variety of hyper-partisan, conspiracy theory or click-bait content that was designed to look like real news about politics. During the 2016 presidential election in the United States, social media users on Twitter shared as much “junk news” as professionally produced news about politics (Howard, Bolsover, Kollanyi, Bradshaw, & Neudert, 2017; Howard, Kollanyi, et al., 2017). And voters in swing-states tended to share more junk news than their counterparts in uncontested ones (Howard, Kollanyi, et al., 2017). In countries throughout Europe — in France, Germany, the United Kingdom and Sweden — junk news inflamed political debates around immigration and amplified populist voices across the continent (Desiguad, Howard, Kollanyi, & Bradshaw, 2017; Kaminska, Galacher, Kollanyi, Yasseri, & Howard, 2017; Neudert, Howard, & Kollanyi, 2017).

According to researchers on the Computational Propaganda Project junk news is defined as having at least three out of five elements: (1) professionalism, where sources do not employ the standards and best practices of professional journalism including information about real authors, editors, and owners (2) style, where emotionally driven language, ad hominem attacks, mobilising memes and misleading headlines are used; (3) credibility, where sources rely on false information or conspiracy theories, and do not post corrections; (4) bias, where sources are highly biased, ideologically skewed and publish opinion pieces as news; and (5) counterfeit, where sources mimic established news reporting including fonts, branding and content strategies (Bradshaw et al., 2019).

In a complex ecosystem of political news and information, junk news provides a useful point of analysis because rather than focusing on individual stories that may contain honest mistakes, it examines the domain as a whole and looks for various elements of deception, which underscores the definition of disinformation. The concept of junk news is also not tied to a particular producer of disinformation, such as foreign operatives, hyper-partisan media, or hate groups, who, despite their diverse goals, deploy the same strategies to generate discoverability. Given that the literature on disinformation is often siloed around one particular actor, does not cross platforms, nor integrate a variety of media sources (Tucker et al., 2018), the junk news framework can be useful for taking a broader look at the ecosystem as a whole and the digital techniques producers use to game search engine algorithms. Throughout this paper, I use the term “junk news” to describe the wide range of politically and economically motivated disinformation being shared about politics.

The logic and politics of search

Search engines play a fundamental role in the modern information environment by sorting, organising, and making visible content on the internet. Before the search engine, anyone who wished to find content online would have to navigate “cluttered portals, garish ads and spam galore” (Pasquale, 2015). This didn’t matter in the early days of the web when it remained small and easy to navigate. During this time, web directories were built and maintained by humans who often categorised pages according to their characteristics (Metaxas, 2010). By the mid-1990s it became clear that the human classification system would not be able to scale. The search engine “brought order to chaos by offering a clean and seamless interface to deliver content to users” (Hoffman, Taylor, & Bradshaw, 2019).

Simplistically speaking, search engines work by crawling the web to gather information about online webpages. Data about the words on a webpage, links, images, videos, or the pages they link to are organised into an index by an algorithm, analogous to an index found at the end of a book. When a user types a query into Google Search, machine learning algorithms apply complex statistical models in order to deliver the most “relevant” and “important” information to a user (Gillespie, 2012). These models are based on a combination of “signals” including the words used in a specific query, the relevance and usability of webpages, the expertise of sources, and other information about context, such as a user’s geographic location and settings (Google, 2019).

Google’s search rankings are also influenced by AdWords, which allow individuals or companies to promote their websites by purchasing “paid placement” for specific keyword searches. Paid placement is conducted through a bidding system, where rankings and the number of times the advertisement is displayed are prioritised by the amount of money spent by the advertiser. For example, a company that sells jeans might purchase AdWords for keywords such as “jeans”, “pants”, or “trousers”, so when an individual queries Google using these terms, a “sponsored post” will be placed at the top of the search results. 2 AdWords also make use of personalisation, which allow advertisers to target more granular audiences based on factors such as age, gender, and location. Thus, a local company selling jeans for women can specify local female audiences — individuals who are more likely to purchase their products.

The way in which Google structures, organizes, and presents information and advertisements to users is important because these technical and policy decisions embed a wide range of political issues (Granka, 2010; Introna & Nissenbaum, 2000; Vaidhynathan, 2011). Several public and academic investigations auditing Google’s algorithms have documented various examples of bias in Search or problems with the autocomplete function (Cadwalladr, 2016a; Pasquale, 2015). Biases inherently designed into algorithms have been shown to disproportionately marginalise minority communities, women, and the poor (Noble, 2018).

At the same time, political advertisements have become a contentious political issue. While digital advertising can generate significant benefits for democracy, by democratising political finance and assisting in political mobilisation (Fowler et al., 2019; Baldwin-Philippi, 2017), it can also be used to selectively spread disinformation and messages of demobilisation (Burkell & Regan, 2019; Evangelista & Bruno, 2019; Howard, Ganesh, Liotsiou, Kelly, & Francois, 2018). Indeed, Russian AdWord purchases in the lead-up to the 2016 US election demonstrate how foreign states actors can exploit Google Search to spread propaganda (Mueller, 2019). But the general lack of regulation around political advertising has also raised concerns about domestic actors and the ways in which legitimate politicians campaign in increasingly opaque and unaccountable ways (Chester & Montgomery, 2017; Tufekci, 2014). These concerns are underscored by the rise of the “influence industry” and the commercialisation of political technologies who sell various ‘psychographic profiling’ technologies to craft, target, and tailor messages of persuasion and demobilisation (Chester & Montgomery, 2019; McKelvey, 2019; Bashyakarla, 2019). For example, during the 2016 US election, Cambridge Analytica worked with the Trump campaign to implement “persuasion search advertising”, where AdWords were bought to strategically push pro-Trump and anti-Clinton information to voters (Lewis & Hilder, 2018).

Given growing concerns over the spread of disinformation online, scholars are beginning to study the ways in which Google Search might amplify junk news and disinformation. One study by Metaxa-Kakavouli and Torres-Echeverry examined the top ten results from Google searches about congressional candidates over a 26-week period in the lead-up to the 2016 presidential election. Of the URLs recommended by Google, only 1.5% came from domains that were flagged by PolitiFact as being “fake news” domains (2017). Metaxa-Kakavouli and Torres-Echeverry suggest that the low levels of “fake news” are the result of Google’s “long history” combatting spammers on its platform (2017). Another research paper by Golebiewski and boyd looks at how gaps in search engine results lead to strategic “data voids” that optimisers exploit to amplify their content (2018). Golebiewski and boyd argue that there are many search terms where data is “limited, non-existent or deeply problematic” (2018). Although these searches are rare, if a user types these search terms into a search engine, “it might not give a user what they are looking for because of limited data and/or limited lessons learned through previous searches” (Golebiewski & boyd, 2018).

The existence of biases, disinformation, or gaps in authoritative information on Google Search matters because Google directly impacts what people consume as news and information. Most of the time, people do not look past the top ten results returned by the search engine (Metaxas, 2010). Indeed, eye-tracking experiments have demonstrated that the order in which Google results are presented to users matters more than the actual relevance of the page abstracts (Pan et al., 2007). However, it is important to note that the logic of higher placements does not necessarily translate to search engine advertising listings, where users are less likely to click on advertisements if they are familiar with the brand or product they are searching for (Narayanan & Kalyanam, 2015).

Nevertheless, the significance of the top ten placement has given rise to the SEO industry, whereby optimisers use digital keyword strategies to move webpages higher in Google’s rankings and thereby generate higher traffic flows. There is a long history of SEO dating back to the 1990s when the first search engine algorithms emerged (Metaxas, 2010). Since then, hundreds of SEO pages have published guesses about the different ranking factors these algorithms consider (Dean, 2019). However, the specific signals that inform Google’s search engine algorithms are dynamic and constantly adapting to the information environment. Google makes hundreds of changes to its algorithm every year to adjust the weight and importance of various signals. While most of these changes are minor updates designed to improve the speed and performance of Search, sometimes Google makes more significant changes to its algorithm to elude optimisers trying to game the system.

Google has taken several steps to combat people seeking to manipulate Search for political or economic gain (Taylor, Walsh, & Bradshaw, 2019). This involves several algorithmic changes to demote sources of disinformation as well as changes to their advertising policies to limit the extent to which users can be micro-targeted with political advertisements. In one study, researchers interviewed SEO strategists to audit how Facebook and Google’s algorithmic changes impacted their optimisation strategies (Hoffmann, Taylor, & Bradshaw, 2019). Since the purveyors of disinformation often rely on the same digital marketing strategies used by legitimate political candidates, news organisations, and businesses, the SEO industry can offer unique, but heuristic, insight into the impact of algorithmic changes. Hoffmann, Taylor and Bradshaw (2019) found that despite more than 125 announcements over a three-year period, the algorithmic changes made by the platforms did not significantly alter digital marketing strategies.

This paper hopes to contribute to the growing body of work examining the effect of Search on the spread of disinformation and junk news by empirically analysing the strategies — paid and optimised — employed by junk news domains. By performing an audit of the keywords junk news websites use to generate discoverability, this paper evaluates the effectiveness of Google in combatting the spread of disinformation on Search.

Methodology

Conceptual Framework: The Techno-Commercial Infrastructure of Junk News

The starting place for this inquiry into the SEO infrastructure of junk news domains is grounded conceptually in the field of science and technology studies (STS), which provides a rich literature on how infrastructure design, implementation, and use embeds politics (Winner, 1980). Digital infrastructure — such as physical hardware, cables, virtual protocols, and code — operate invisibly in the background, which can make it difficult to trace the politics embedded in technical coding and design (Star & Ruhleder, 1994). As a result, calls to study internet infrastructure has engendered digital research methods that shed light on the less-visible areas of technology. One growing and relevant body of research has focused on the infrastructure of social media platforms and the algorithms and advertising infrastructure that invisibly operate to amplify or spread junk news to users, or to micro-target political advertisements (Kim et al., 2018; Tambini, Anstead, & Magalhães, 2017). Certainly, the affordances of technology — both real and imagined — mutually shape social media algorithms and their potential for manipulation (Nagy & Neff, 2015; Neff & Nagy, 2016). However, the proprietary nature of platform architecture has made it difficult to operationalise studies in this field. Because junk news domains operate in a digital ecosystem built on search engine optimisation, page ranks, and advertising, there is an opportunity to analyse the infrastructure that supports the discoverability of junk news content, which could provide insights into how producers reach audiences, grow visibility, and generate domain value.

Junk news data set

The first step of my methodology involved identifying a list of junk news domains to analyse. I used the Computational Propaganda Project’s (COMPROP) data set on junk news domains in order to analyse websites that spread disinformation about politics. To develop this list, researchers on the COMPROP project built a typology of junk news based on URLs shared on Twitter and Facebook relating to the 2016 US presidential election, the 2017 US State of the Union Address, and 2018 US midterm elections. 3 A team of five rigorously trained coders labelled the domains contained in tweets and on Facebook pages based on a grounded typology of junk news that has been tested and refined over several elections around the world between 2016 and 2018. 4 A domain was labelled as junk news when it failed on three of the five criteria of the typology (style, bias, credibility, professionalism, and counterfeit, as described in section one). For this analysis, I used the most recent 2018 midterm election junk news list, which is comprised of the top-29 most shared domains that were labelled as junk news by researchers. This list was selected because all 29 domains were active during the 2016 US presidential election in November 2016 and the 2017 US State of the Union Address, which provides an opportunity to comparatively assess how both the advertising and optimisation strategies, as well as their performance, changed overtime.

SpyFu data and API queries

The second step of my methodology involved collecting data about the advertising and optimisation strategies used by junk news websites. I worked with SpyFu, a competitive keyword research tool used by digital marketers to increase website traffic and improve keyword rankings on Google (SpyFu, 2019). SpyFu collects, analyses and tracks various data about the search optimisation strategies used by websites, such as organic ranks, paid keywords bought on Google AdWords, and advertisement trends.

To shed light onto the optimisation strategies used by junk news domains on Google, SpyFu provided me with: (1) a list of historical keywords and keyword combinations used by the top-29 junk news that led to the domain appearing in Google Search results; and (2) the position the domain appeared in Google as a result of the keywords. The historical keywords were provided from January 2016 until March 2019. Only keywords that led to the junk news domains appearing in the top-50 positions on Google were included in the data set.

In order to determine the effectiveness of the optimisation and advertising strategies used by junk news domains to either grow their website value and/or successfully appear in the top positions on Google Search, I wrote a simple python script to connect to the SpyFu API service. This python script collected and parsed the following data from SpyFu for each of the top-29 junk news domains in the sample: (1) the number of keywords that show up organically on Google searches; (2) the estimated sum of clicks a domain receives based on factors including organic keywords, the rank of keyword, and the search volume of the keyword; (3) the estimated organic value of a domain based on factors including organic keywords, the rank of keywords, and the search volume of the keyword; (4) the number of paid advertisements a domain purchased through Google AdWords; and (5) the number of paid clicks a domain received from the advertisements it purchased from Google AdWords.

Data and methodology limitations

There are several data and methodology limitations that must be noted. First, the junk news domains identified by the Computational Propaganda Project highlights only a small sample of the wide variety of websites that peddle disinformation about politics. The researchers also do not differentiate between the different actors behind the junk news websites — such as foreign states or hyper-partisan media — nor do they differentiate between the political leaning of the junk news outlet — such as left-or-right-leaning domains. Thus, the outcomes of these findings cannot be described in terms of the strategies of different actors. Further, given that the majority of junk news domains in the top-29 sample lean politically to the right and far right, these findings might not be applicable to the hyper-partisan left and their optimisation strategies. Finally, the junk news domains identified in the sample were shared on social media in the lead-up to important political events in the United States. A further research question could examine the SEO strategies of domains operating in other country contexts.

When it comes to working with the data provided by SpyFu (and other SEO optimisation tools), there are two limitations that should be noted. First, the historical keywords collected by SpyFu are only collected when they appear in the top-50 Google Search results. This is an important limitation to note because news and information producers are constantly adapting keywords based on the content they are creating. Keywords may be modified by the source website dynamically to match news trends. Low performing keywords might be changed or altered in order to make content more visible via Search. Thus, the SpyFu data might not capture all of the keywords used by junk news domains. However, the collection strategy will have captured many of the most popular keywords used by junk news domains to get their content appearing in Google Search. Second, because SpyFu is a company there are proprietary factors that go into measuring a domain’s SEO performance (in particular, the data points collected via the API on the estimated sum of clicks and the estimated organic value). Nevertheless, considering that Google Search is a prominent avenue for news and information discovery, and that few studies have systematically analysed the effect of search engine optimisation strategies on the spread of disinformation, this study provides an interesting starting point for future research questions about the impact SEO can have on the spread and monetisation of disinformation via Search.

Analysis: optimizing disinformation through keywords and advertising

Junk news advertising strategies on Google

Junk news domains rarely advertise on Google. Only two out of the 29 junk news domains (infowars.com and cnsnews.com) purchased Google advertisements (See Figure 1: Advertisements purchased vs. paid clicks). The advertisements purchased by infowars.com were all made prior to the 2016 election in the United States (from the period of May 2015 to March 2016). cnsnews.com made several advertisement purchases over the three-year time period.

Figure 1: Advertisements purchased vs. paid clicks received: inforwars.com and cnsnews.com (May 2015-March 2019)

Looking at the total number of paid clicks received, junk news domains generated only a small amount of traffic using paid advertisements. Infowars on average, received about 2000 clicks as a result of their paid advertisements. cnsnews.com peaked at approximately 1800 clicks, but on average generated only about 600 clicks per month over the course of three years. By comparing the number of clicks that are paid versus those that were generated as a result of SEO keyword optimisation, there is a significant difference. During the same time period, cnsnews.com and infowars.com were generating on average 146,000 and 964,000 organic clicks respectively (See Figure 2:Organic vs. paid clicks (cnsnews.com and infowars.com)). Although it is hard to make generalisations about how junk news websites advertise on Google based on a sample of two, the lack of data suggests that advertising on Google Search might not be as popular as advertising on other social media platforms. Second, the return on investment (i.e., paid clicks generated as a result of Google advertisements) was very low compared to the organic clicks these junk news domains received for free. Factors other than advertising seem to drive the discoverability of junk news on Google Search.

Figure 2: organic vs. paid clicks (cnsnews.com and infowars.com)

Junk news keyword optimisation strategies

In order to assess the keyword optimisation strategies used by junk news websites, I worked with SpyFu, which provided historical keyword data for the 29 junk news domains, when those keywords made it to the top-50 results in Google between January 2016 and March 2019. In total, there were 88,662 unique keywords in the data set. Given the importance of placement on Google, I looked specifically at keywords that indexed junk news websites on the first — and most authoritative — position. Junk news domains had different aptitudes for generating placement in the first position (See Table 1: Junk news domains and number of keywords found in the first position on Google). Breitbart, DailyCaller and ZeroHedge had the most successful SEO strategies, respectively having 1006, 957 and 807 keywords lead to top placements on Google Search over the 39-month period. In contrast, six domains (committedconservative.com, davidharrisjr.com, reverbpress.news, thedailydigest.org, thefederalist.com, thepoliticalinsider.com) had no keywords reach the first position on Google. The remaining 20 domains had anywhere between 1 to 253 keywords place between the 2-10 positions on Google Search over the same timeframe.

Different keywords also generate different kinds of placement over the 39-month period. Table 2 (see Appendix) provides a sample list of up to ten keywords from each junk news domain in the sample when the keyword reached the first position.

First, many junk news domains appear in the first position on Google Search as a result of “navigational searches” whereby a user entered a query with the intent of finding a website. A search for a specific brand of junk news could happen naturally for many users, since the Google Search function is built into the address bar in Chrome, and sometimes set as the default search engine for other browsers. In particular, terms like “infowars” “breitbart” “cnsnews” and “rawstory” were navigational keywords users typed into Google Search. The performance of brand searches over time consistently places junk news webpages in the number one position (see Figure 3: Brand-related keywords over time). This suggests that brand-recognition plays an important role for driving traffic to junk news domains.

Figure 3: the performance of brand-related keywords overtime: top-5 junk news websites (January 2016-March 2019)

There is one outlier in this analysis, where keyword searches for “breitbart” drops to position two: in January 2017 and September 2017. This drop could have been a result of mainstream media coverage of Steve Bannon assuming (and eventually leaving) his position as the White House Chief Strategist during those respective months. The fact that navigational searches are one of the main drivers behind generating a top ten placement on Search suggests that junk news websites rely heavily on developing a recognisable brand and a dedicated readership that actively seeks out content from these websites. However, this also demonstrates that a complicated set of factors go into determining what keywords from what websites make the top placement in Google Search, and that coverage of news events from mainstream professional news outlets can alter the discoverability of junk news via Search.

Second, many keywords that made it to the top position in Google Search results are what Golebiewski and boyd (2018) would call terms that filled “data voids”, or gaps in search engine queries where there is limited authoritative information about a particular issue. These keywords tended to focus on conspiratorial information especially around President Barack Obama (“Obama homosexual” or “stop Barack Obama”), gun rights (“gun control myths”), pro-life narratives (“anti-abortion quotes” or “fetus after abortion”), and xenophobic or racist content (“against Islam” or “Mexicans suck”). Unlike brand-related keywords, problematic search terms do not achieve a consistently high placement on Google Search over the 39-week period. Keywords that ranked in number one for more than 30-weeks include: “vz58 vs. ak47”, “feminizing uranium”, “successful people with down syndrome”, “google ddrive”, and “westboro[sic] Baptist church tires slashed”. This suggests that, for the most part, data voids are either being filled by more authoritative sources, or Google Search has been able to demote websites attempting to generate pseudo-organic engagement via SEO.

The performance of junk news domains on Google Search

After analysing what keywords are used to get junk news websites in the number one position, the next half of my analysis looks at larger trends in SEO strategies overtime. What is the relationship between organic clicks and the value of a junk news website? How has the effectiveness of SEO keywords changed over the past 48 months? And have changes made by Google to combat the spread of junk news on Search had an impact on its discoverability?

Junk news, organic clicks, and the value of the domain

There is a close relationship between the number of clicks a domain receives and the estimated value of that domain. By comparing figure 4 and 5, you can see that the more clicks a website receives, the higher its estimated value. Often, a domain is considered more valuable when it generates large amounts of traffic. Advertisers see this as an opportunity, then, to reach more people. Thus, the higher the value of a domain, the more likely it is to generate revenue for the operator. The median estimated value of the top-29 most popular junk news was $5,160 USD during the month of the 2016 presidential election, $1,666.65 USD during the 2018 State of the Union, and $3,906.90 USD during the 2018 midterm elections. Infowars.com and breitbart.com were the two highest performing junk news domains — in terms of clicks and domain value. While breitbart.com maintained a more stable readership, especially around the 2016 US presidential election and the 2018 US State of the Union Address, its estimated organic click rate has steadily decreased since early 2018. In contrast, infowars.com has a more volatile readership. The spikes in clicks to infowars.com could be explained by media coverage of the website, including the defamation case against Alex Jones in April 2018 who claimed the shooting at Sandy Hook Elementary School was “completely fake” and a “giant hoax”. Since then, several internet companies — including Apple, Twitter, Facebook, Spotify, and YouTube — banned Infowars from their platforms, and the domain has not been able to regain its clicks nor value since. This demonstrates the powerful role platforms play in not only making content visible to users, but also controlling who can grow their website value — and ultimately generate revenue — from the content they produce and share online.

Figure 4: Estimated organic value for the top 29 junk news domains (May 2015 – March 2019)

Figure 5: Estimated organic clicks for the top 29 junk news domains (May 2015-April 2019)

Junk news domains, search discoverability and Google’s response to disinformation

Figure 6 shows the estimated organic results of the top 29 junk news domains overtime. The estimated organic results are the number of keywords that would organically appear in Google searches. Since August 2017, there has been a sharp decline in the number of keywords that would appear in Google. The four top-performing junk news websites (infowars.com, zerohedge.com, dailycaller.com, and breitbart.com) all appeared less frequently in top-positions on Google Search based on the keywords they were optimising for. This is an interesting finding and suggests that the changes Google made to its search algorithm did indeed have an impact on the discoverability of junk news domains after August 2017. In comparison, other professional news sources (washingtonpost.com, nytimes.com, foxnews.com, nbcnews.com, bloomberg.com, bbc.co.uk, wsj.com, and cnn.com) did not see substantial drops in their search visibility during this timeframe (see Figure 7). In fact, after August 2017 there has been a gradual increase in the organic results of mainstream news media.

Figure 6: Estimated organic results for the top 29 junk news domains (May 2015- April 2019)

Figure 7: Estimated organic results for mainstream media websites in the United States (May 2015-April 2019)

After almost a year, the top-performing junk news websites have regained some of their organic results, but the levels are not nearly as high as they were leading up to and preceding the 2016 presidential election. This demonstrates the power of Google’s algorithmic changes in limiting the discoverability of junk news on Search. But it also shows how junk news producers learn to adapt their strategies in order to extend the visibility of their content. In order to be effective at limiting the visibility of bad information via search, Google must continue to monitor the keywords and optimisation strategies these domains deploy — especially in the lead-up to elections — when more people will be naturally searching for news and information about politics.

Conclusion

In conclusion, the spread of junk news on the internet and the impact it has on democracy has certainly been a growing field of academic inquiry. This paper has looked at a small subset of this phenomenon, in particular the role of Google Search in assisting in the discoverability and monetisation of junk news domains. By looking at the techno-commercial infrastructure that junk news producers use to optimise their websites for paid and pseudo-organic clicks, I found:

1. Junk news domains do not rely on Google advertisements to grow their audiences and instead focus their efforts on optimisation and keyword strategies;
2. Navigational searches drive the most traffic to junk news websites, and data voids are used to grow the discoverability of junk news content to mostly small, but varying degrees.
3. Many junk news producers place advertisements on their websites and grow their value particularly around important political events; and
4. Overtime, the SEO strategies used by junk news domains have decreased in their ability to generate top-placements in Google Search.

For millions of people around the world, the information Google Search recommends directly impacts how ideas and opinions about politics are formulated. The powerful role of Google as an information gatekeeper has meant that bad actors have tried to subvert these technical systems for political or economic game. For quite some time, Google’s algorithms have come under attack by spammers and other malign actors who wish to spread disinformation, conspiracy theories, spam, and hate speech to unsuspecting users. The rise of “computational propaganda” and the variety of bad actors exploiting technology to influence political outcomes has also led to the manipulation of Search. Google’s response to the optimisation strategies used by junk news domains has had a positive effect on limiting the discoverability of these domains over time. However, the findings of this paper are also showing an upward trend, as junk news producers find new ways to optimise their content for higher search rankings. This game of cat and mouse is one that will continue for the foreseeable future.

While it is hard to reduce the visibility of junk news domains when individuals actively search for them, more can be done to limit the ways in which bad actors might try to optimise content to generate pseudo-organic engagement, especially around disinformation. Google can certainly do more to tweak its algorithms in order to demote known disinformation sources, as well as identify and limit the discoverability of content seeking to exploit data voids. However, there is no straightforward technical patch that Google can implement to stop various actors from trying to game their systems. By co-opting the technical infrastructure and policies that enable search, the producers of junk news are able to spread disinformation — albeit to small audiences who might use obscure search terms to learn about a particular topic.

There have also been growing pressures for regulators to take steps that force social media platforms to take greater actions that limit the spread of disinformation online. But the findings of this paper have two important lessons for policymakers. First, the disinformation problem — through both optimisation and advertising — on Google Search is not as dramatic as it is sometimes portrayed. Most of the traffic to junk news websites are by users performing navigational searches to find specific, well-known brands. Only a limited number of placements — as well as clicks — to junk news domains come from pseudo-organic engagement generated by data voids and other problematic keyword searches. Thus, requiring Google to take a heavy-handed approach to content moderation could do more harm than good, and might not reflect the severity of the problem. Second, the reason why disinformation spreads on Google are reflective of deeper systemic problems within democracies: growing levels of polarisation and distrust in the mainstream media are pushing citizens to fringe and highly partisan sources of news and information. Any solution to the spread of disinformation on Google Search will require thinking about media and digital literacy and programmes to strengthen, support, and sustain professional journalism.

Politics is an art and not a science, and what is required for its mastery is not the rationality of the engineer but the wisdom and the moral strength of the statesman. - Hans Morgenthau, Scientific Man versus Power Politics

Voters, industry representatives, and lawmakers – and not infrequently, journalists and academics as well – have asked one question more than any other when presented with evidence of how personal data is changing modern-day politicking: “Does it work?” As my colleagues and I have detailed in our report, Personal Data: Political Persuasion, the convergence of politics and commercial data brokering has transformed personal data into a political asset, a means for political intelligence, and an instrument for political influence. The practices we document are varied and global: an official campaign app requesting camera and microphone permissions in India, experimentation to select slogans designed to trigger emotional responses from Brexit voters, a robocalling-driven voter suppression campaign in Canada, attack ads used to control voters’ first impressions on search engines in Kenya, and many more.

Asking “Does it work?” is understandable for many reasons, including to address any real or perceived damage to the integrity of an election, to observe shifts in attitudes or voting behaviour, or perhaps to ascertain and harness the democratic benefits of the technology in question. However, discourse fixated on the efficacy of data-intensive tools is fraught with abstraction and reflects a shortsighted appreciation for the full political implications of data-driven elections.

“Does it work?”

The question “Does it work?” is very difficult to answer with any degree of confidence regardless of the technology in question: personality profiling of voters to influence votes, natural language processing applied to the Twitter pipeline to glean information about voters’ political leanings, political ads delivered in geofences, or a myriad of others.

First, the question is too general with respect to the details it glosses over. The technologies themselves are a heterogenous mix, and their real-world implementations are manifold. Furthermore, questions of efficacy are often divorced of context, and a technology’s usefulness to a campaign very likely depends on the sociopolitical context in which it lives. Finally, the question of effectiveness continues to be studied extensively. Predictably, the conclusions of peer-reviewed research vary.

As one example, the effectiveness of implicit social pressure in direct mail in the United States evidently remains inconclusive. The motivation for this research is the observation that voting is a social norm responsive to others’ impressions (Blais, 2000; Gerber & Rogers, 2009). However, some evidence suggests that explicit social pressure to mobilise voters (e.g., by disclosing their vote histories) may seem invasive and can backfire (Matland & Murray, 2013). In an attempt to preserve the benefits of social pressure while mitigating its drawbacks, researchers have explored whether implicit social pressure in direct mail (i.e., mailers with an image of eyes, reminding recipients of their social responsibility) boosts turnout on election day. Of their evaluation of implicit social pressure, which had apparently been regarded as effective, political scientists Richard Matland and Gregg Murray concluded that, “The effects are substantively and statistically weak at best and inconsistent with previous findings” (Matland & Murray, 2016). In response to similar, repeated findings from Matland and Murray, Costas Panagopoulos wrote that their work in fact “supports the notion that eyespots likely stimulate voting, especially when taken together with previous findings” (Panagopoulos, 2015). Panagopoulos soon thereafter authored a paper arguing that the true impact of implicit social pressure actually varies with political identity, claiming that the effect is pronounced for Republicans but not for Democrats or Independents, while Matland maintained that the effect is "fairly weak" (Panagopoulos & van der Linden, 2016; Matland, 2016).

Similarly, studies on the effects of door-to-door canvassing lack consensus (Bhatti et al., 2019). Donald Green, Mary McGrath, and Peter Aronow published a review of seventy-one canvassing experiments and found their average impact to be robust and credible (Green, McGrath, & Aronow, 2013). A number of other experiments have demonstrated that canvassing can boost voter turnout outside the American-heavy literature: among students in Beijing in 2003, with British voters in 2005, and for women in rural Pakistan in 2008 (Guan & Green, 2006; John & Brannan, 2008; Giné & Mansuri, 2018). Studies from Europe, however, call into question the generalisability of these findings. Two studies on campaigns in 2010 and 2012 in France both produced ambiguous results, as the true effect of canvassing was not credibly positive (Pons, 2018; Pons & Liegey, 2019). Experiments conducted during the 2013 Danish municipal elections observed no definitive effect of canvassing, while Enrico Cantoni and Vincent Pons found that visits by campaign volunteers in Italy helped increase turnout, but those by the candidates themselves did not (Bhatti et al., 2019; Cantoni & Pons, 2017). In some cases, the effect of door-to-door canvassing was neither positive nor ambiguous but distinctly counterproductive. Florian Foos and Peter John observed that voters contacted by canvassers and given leaflets for the 2014 British European Parliament elections were 3.7 percentage points less likely to vote than those in the control group (Foos & John, 2018). Putting these together, the effects of canvassing still seem positive in Europe, but they are less pronounced than in the US. This learning has led some scholars to note that “practitioners should be cautious about assuming that lessons from a US- dominated field can be transferred to their own countries’ contexts” (Bhatti et al., 2019).

A cursory glance at a selection of literature related to these two cases alone – implicit social pressure and canvassing – illustrates how tricky answering “Does it work?” is. Although many of the technologies in use today are personal data-supercharged analogues of these antecedents (e.g., canvassing apps with customised scripts and talking points based on data about each household’s occupants instead of generic, door-to-door knocking), I have no reason to suspect that analyses of data-powered technologies would be any different. The short answer to “Does it work?” is that it depends. It depends on baseline voter turnout rates, print vs. digital media, online vs. offline vs. both combined, targeting young people vs. older people, reaching members of a minority group vs. a majority group, partisan vs. nonpartisan messages, cultural differences, the importance of the election, local history, and more. Indeed, factors like the electoral setup may alter the effectiveness of a technology altogether. A tool for political persuasion might work in a first-past-the-post contest in the United States but not in a European system of proportional representation in which winner-take-all stakes may be tempered. This is not to suggest that asking “Does it work?” is a futile endeavour – indeed there are potential democratic benefits to doing so – but rather that it is both limited in scope and rather abstract given the multitude of factors and conditions at play in practice.

Political calculus and algorithmic contagion

With this in mind, I submit that a more useful approach to appreciating the full impacts of data-driven elections may be a consideration of the preconditions that allow data-intensive practices to thrive and an examination of their consequences than a preoccupation with the efficacy of the practices themselves.

In a piece published in 1986, philosopher Ian Hacking coined the term ‘semantic contagion’ to describe the process of ascribing linguistic and cultural currency to a phenomenon by naming it and thereby also contributing to its spread (Hacking, 1999). I propose that the prevailing political calculus, spurred on by the commercial success of “big data” and “AI”, appears overtaken by an ‘algorithmic contagion’ of sorts. On one level, algorithmic contagion speaks to the widespread logic of quantification. For example, understanding an individual is difficult, so data brokers instead measure people along a number of dimensions like level of education, occupation, credit score, and others. On another level, algorithmic contagion in this context describes an interest in modelling anything that could be valuable to political decision-making, as Market Predict’s political page suggests. It presumes that complex phenomena, like an individual’s political whims, can be predicted and known within the structures of formalised algorithmic process, and that they ought to be. According to the Wall Street Journal, a company executive claimed that Market Predict’s “agent-based modelling allows the company to test the impact on voters of events like news stories, political rallies, security scares or even the weather” (Davies, 2019).

Algorithmic contagion also encompasses a predetermined set of boundaries. Thinking within the capabilities of algorithmic methods prescribes a framework to interpret phenomena within bounds that enable the application of algorithms to those phenomena. In this respect, algorithmic contagion can influence not only what is thought but also how. This conceptualisation of algorithmic contagion encompasses the ontological (through efforts to identify and delineate components that structure a system, like an individual’s set of beliefs), the epistemological (through the iterative learning process and distinction drawn between approximation and truth), and the rhetorical (through authority justified by appeals to quantification).

Figure 1: The political landing page of Market Predict, a marketing optimisation firm for brand and political advertisers, that explains its voter simulation technology. It claims to, among other things, “Account for the irrationality of human decision-making”. Hundreds of companies offer related services. Source: Market Predict Political Advertising

This algorithmic contagion-informed formulation of politics bears some connection to the initial “Does it work?” query but expands the domain in question to not only the applications themselves but also to the components of the system in which they operate – a shift that an honest analysis of data-driven elections, and not merely ad-based micro-targeting, demands. It explains why and how a candidate for mayor in Taipei in 2014 launched a viral social media sensation by going to a tattoo parlour. He did not visit the parlour to get a tattoo, to chat with an artist about possible designs, or out of a genuine interest in meeting the people there. He went because a digital listening company that mines troves of data and services campaigns across southeast Asia generated a list of actions for his campaign that would generate the most buzz online, and visiting a tattoo parlour was at the top of the list.

Figure 2: A still from a video documenting Dr Ko-Wen Je’s visit to a tattoo parlour, prompting a social media sensation. His campaign uploaded the video a few days before municipal elections in which he was elected mayor of Taipei in 2014. The post on Facebook has 15,000 likes, and the video on YouTube has 153,000 views. Against a backdrop of creeping voter surveillance, Dr Ko-Wen Je’s visit to this tattoo parlour begs questions about the authenticity of political leaders. (Image brightened for clarity) Sources: Facebook and YouTube

As politics continues to evolve in response to algorithmic contagion and to the data industrial complex governing the commercial (and now also political) zeitgeist, it is increasingly concerned with efficiency and speed (Schechner & Peker, 2018). Which influencer voters must we win over, and whom can we afford to ignore? Who is both the most likely to turn out to vote and also the most persuadable? How can our limited resources be allocated as efficiently as possible to maximise the probability of winning? In this nascent approach to politics as a practice to be optimised, who is deciding what is optimal? Relatedly, as the infrastructure of politics changes, who owns the infrastructure upon which more and more democratic contests are waged, and to what incentives do they respond?

Voters are increasingly treated as consumers – measured, ranked, and sorted by a logic imported from commerce. Instead of being sold shoes, plane tickets, and lifestyles, voters are being sold political leaders, and structural similarities to other kinds of business are emerging. One challenge posed by data-driven election operations is the manner in which responsibilities have effectively been transferred. Voters expect their interests to be protected by lawmakers while indiscriminately clicking “I Agree” to free services online. Efforts to curtail problems through laws are proving to be slow, mired in legalese, and vulnerable to technological circumvention. Based on my conversations with them, venture capitalists are reluctant to champion a transformation of the whole industry by imposing unprecedented privacy standards on their budding portfolio companies, which claim to be merely responding to the demands of users. The result is an externalised cost shouldered by the public. In this case, however, the externality is not an environmental or a financial cost but a democratic one. The manifestation of these failures include the disintegration of the public sphere and a shared understanding of facts, polarised electorates embroiled in 365-day-a-year campaign cycles online, and open campaign finance and conflict of interest loopholes introduced by data-intensive campaigning, all of which are exacerbated by a growing revolving door between the tech industry and politics (Kreiss & McGregor, 2017).

Personal data and political expediency

One response to Cambridge Analytica is “Does psychometric profiling of voters work?” (Rosenberg et al., 2018). A better response examines what the use of psychometric profiling reveals about the intentions of those attempting to acquire political power. It asks what it means that a political campaign was apparently willing to invest the time and money into building personality profiles of every single adult in the United States in order to win an election, regardless of the accuracy of those profiles, even when surveys of Americans indicate that they do not want political advertising tailored to their personal data (Turow et al., 2012). And it explores the ubiquity of services that may lack Cambridge Analytica’s sensationalised scandal but shares the company’s practice of collecting and using data in opaque ways for clearly political purposes.

The ‘Influence Industry’ underlying this evolution has evangelised the value of personal data, but to whatever extent personal data is an asset, it is also a liability. What risks do the collection and use of personal data expose? In the language of the European Union’s General Data Protection Regulation (GDPR), who are the data controllers, and who are the data subjects in matters of political data which is, increasingly, all data? In short, who gains control, and who loses it?

As a member of a practitioner-oriented group based in Germany with a grounding in human rights, I worry about data-intensive practices in elections and the larger political sphere going awry, especially as much of our collective concern seems focused on questions of efficacy while companies race to capitalise on the market opportunity. For historical standards of the time, the Holocaust was a ruthlessly data-driven, calculated, and efficient undertaking fuelled by vast amounts of personal data. As Edwin Black documents in IBM & The Holocaust: The Strategic Alliance between Nazi Germany and America's Most Powerful Corporation, personal data managed by IBM was an indispensable resource for the Nazi regime. IBM’s President at the time, Thomas J. Waston Sr., the namesake of today’s IBM Watson, went to great lengths to profit from dealings between IBM’s German subsidiary and the Nazi party. The firm was such an important ally that Hitler awarded Watson an Order of the German Eagle award for his invaluable service to the Third Reich. IBM aided the Nazi’s record-keeping across several phases of the Holocaust, including identification of Jews, ghettoisation, deportation, and extermination (Black, 2015). Black writes that “Prisoners were identified by descriptive Hollerith cards, each with columns and punched holes detailing nationality, date of birth, marital status, number of children, reason for incarceration, physical characteristics, and work skills” (Black, 2001). These Hollerith cards were sorted in machines physically housed in concentration camps.

The precursors to these Hollerith cards were originally developed to track personal details for the first American census. The next American census, to be held in 2020, has already been a highly politicised affair with respect to the addition of a citizenship question (Ballhaus & Kendall, 2019). President Trump recently abandoned an effort to formally add a citizenship question to the census, vowing to seek this information elsewhere, and the US Census Bureau has already published work investigating the quality of alternate citizenship data sources for the 2020 Census (Brown et al., 2018). For stakeholders interested in upholding democratic ideals, focusing on the accuracy of this alternate citizenship data is myopic; that an alternate source of data is being investigated to potentially advance an overtly political goal is the crux of the matter.

Figure 3: A card showing the personal data of Symcho Dymant, a prisoner at Buchenwald Concentration Camp. The card includes many pieces of personal data, including name, birth date, condition, number of children, place of residence, religion, citizenship, residence of relatives, height, eye colour, description of his nose, mouth, ears, teeth, and hair. Source: US Holocaust Memorial Museum

This prospect may seem far-fetched and alarmist to some, but I do not think so. If the political tide were to turn, the same personal data used for a benign digital campaign could be employed in insidious and downright unscrupulous ways if it were ever expedient to do so. What if a door-to-door canvassing app instructed volunteers walking down a street to skip your home and not remind your family to vote because a campaign profiled you as supporters of the opposition candidate? What if a data broker classified you as Muslim, or if an algorithmic analysis of your internet browsing history suggests that you are prone to dissent? Possibilities like these are precisely why a fixation on efficacy is parochial. Given the breadth and depth of personal data used for political purposes, the line between consulting data to inform a political decision and appealing to data – given the rhetorical persuasiveness it enjoys today – in order to weaponise a political idea is extremely thin.

A holistic appreciation of data-driven elections’ democratic effects demands more than simply measurement, and answering “Does it work?” is merely one component of grasping how campaigning transformed by technology and personal data is influencing our political processes and the societies they engender. As digital technologies continue to rank, prioritise, and exclude individuals even when – indeed, especially when – inaccurate, we ought to consider the larger context in which technological practices shape political outcomes in the name of efficiency. The infrastructure of politics is changing, charged with an algorithmic contagion, and a well-rounded perspective requires that we ask not only how these changes are affecting our ideas of who can participate in our democracies and how they do so, but also who derives value from this infrastructure and how they are incentivised, especially when benefits are enjoyed privately but costs sustained democratically. The quantitative tools underlying the ‘datafication’ of politics are neither infallible nor safe from exploitation, and issues of accuracy grow moot when data-intensive tactics are enlisted as pawns in political agendas. A new political paradigm is emerging whether or not it works.

Introduction: all roads lead to Victoria, British Columbia

As the Information and Privacy Commissioner for British Columbia, I am entrusted with enforcing the province’s two pieces of privacy legislation – BC’s Freedom of Information and Protection of Privacy Act (FIPPA) and the Personal Information Protection Act (PIPA). When these laws came into force, “Big Data” was not a term in public discourse. All that of course has changed irrevocably.

In late summer 2017, I left the Office of the Information and Privacy Commissioner for BC (OIPC) to take on an assignment with the UK Information Commissioner’s Office (ICO), under the former BC Commissioner, Elizabeth Denham. I had temporarily stepped aside from my role as Deputy Commissioner at the OIPC to help lead the ICO’s investigation of how the UK’s political parties collected and used the personal information of voters (Information Commissioner's Office, United Kingdom, 2018). Their enquiry came on the heels of media reports concerning the potential misuse of data during the country’s European Union referendum (Doward, 2017). At the time, I had no idea that I would find myself standing, two years later, full circle from the world’s most notorious data breach - the Facebook/Cambridge Analytica scandal, which affected more than 80 million users worldwide (Badshah, 2018).

Soon after my arrival, I interviewed the key data strategists of UK’s two largest parties. With their significant resources, these parties were able to gather volumes of voter data and make predictions about voting intentions. They also had the means to target specific classes of voters in pursuit of their support. Those party representatives were very nervous about sharing the mechanics of their work. This reluctance intersects with one of modern democracy’s great challenges, and it was why the ICO launched its investigation: citizens know very little about what information political parties collect about them – and how that information is being used.

The public was concerned about the opacity of political campaign systems even before the ICO began its work. But their concern was soon to grow exponentially. In early 2018, UK’s Information and Privacy Commissioner Elizabeth Denham and I met a young man in a lawyer’s office in London. He was from, of all places, Victoria, BC, and his name was Christopher Wylie.

We were the first regulator or law enforcement agency to talk with Wylie, and his story was sweeping and shocking in its breadth. Many weeks later, the rest of the world would learn the details of how Cambridge Analytica extracted psychological profiles of millions of Facebook users for the purposes of weaponising targeted political messages. Many of those revelations were reported exclusively by The Guardian journalist Carole Cadwalladr, who wrote extensively about the whistleblower beginning in March 2018 (Cadwalladr, 2018).

Suddenly the whole world was paying attention to the explosive mix of new technologies and personal information and how it was impacting political campaigns. The paired names of Cambridge Analytica and Facebook became seared on the public’s consciousness, providing a cautionary tale about what can go wrong when people’s personal information is abused in such a nefarious manner (Meredith, 2018). The Facebook/Cambridge Analytica breach has, without question, shaken the public’s confidence in our democratic political campaigning system.

It is no doubt purely coincidental that so many storylines of this scandal trace their way to Victoria, BC. Adding to the regulatory connection and the whistleblower Christopher Wylie, is the Victoria-based company AggregateIQ Data Services (AIQ), which analysed the data on behalf of the Cambridge Analytica’s parent company, SCL Elections. Victoria is also home to Dr. Colin Bennett. He has long been a leading global authority in pursuing the study of these matters, work that has now taken on an even greater urgency. For this reason, the OIPC teamed up with the Big Data Surveillance project coordinated by the Surveillance Studies Centre at Queen’s University, and headed by Dr David Lyon. Our office was pleased to host the workshop in April 2019 on “Data-Driven Elections: Implications and Challenges for Democratic Societies,” from which the papers in this collection originated.

Privacy regulators, along with electoral commissioners, are on the frontline of these questions about the integrity of our democratic institutions. However, in some jurisdictions, regulators have very few means to address them, especially as it concerns political parties whose appetites for the personal information of voters is seemingly insatiable. How then does a regulator convince the politicians to regulate themselves?

Home, and another Facebook/Cambridge Analytica investigation

Following the execution of the warrant on Cambridge Analytica’s office in London, I returned home to accept my appointment as BC’s fourth Information and Privacy Commissioner. However, there was no escaping the fallout of the issues I investigated in the UK and their connections to Canada.

As it turned out, the personal information of more than 600,000 Canadian Facebook users had been vacuumed up by Cambridge Analytica (Braga, 2018). But this wasn’t the only Canadian connection to the breach. After acquiring that personal information, Cambridge Analytica (CA) and its parent company SCL Elections needed a way to make the data ready for practical use for potential clients of CA. That requirement would eventually be filled by AIQ.

With a BC and a Canadian connection to this story it became clear that coordinated regulatory action would be required. The Privacy Commissioner of Canada, Daniel Therrien, and I decided to join forces to look at both the Facebook/CA breach and the activities of AIQ (OIPC news release, 2018).

This joint investigation found that Facebook did little to ensure its users’ data was properly protected. Its privacy protection programme was, as my colleague Daniel Therrien called it, an “empty shell.” We recommended, among other things, that Facebook properly audit all of the apps that were allowed to collect their users’ data (OIPC news release, 2019b). Facebook brazenly rejected our findings and recommendations, which of course underscores another huge obstacle.

How can society hold global giants like Facebook to account? Many data protection authorities, like my office, lack the enforcement tools commensurate with the challenges that these companies pose to the public interest. Moreover, my office and that of the federal commissioner have far fewer powers than those available to our European counterparts. I have order-making power, but I cannot levy fines. My federal counterpart does not even possess order-making power; he investigates in response to complaints, or on his own initiative, and he makes recommendations. The only real vehicle he has at his disposal to seek a remedy, is through an unwieldy court application process, which is ongoing as I write. So one can understand why we look with some envy to the European DPAs, which now have the power to impose administrative fines of up to 20 million euros, or 4% of the company’s worldwide annual revenue.

British Columbia’s political parties and privacy regulation

Responsibility for privacy legislation in Canada is divided between the federal government and the provinces (OPC, 2018). The federal regulator, the Office of the Privacy Commissioner of Canada, has no authority to hold political parties to account. Among the provinces that have their own privacy legislation, only one has regulatory oversight over political parties: British Columbia. Given all that was going on at home and around the world concerning political parties, we decided to exercise that authority and investigate how BC’s political parties were collecting and using voter information (OIPC news release, 2019a).

To varying degrees, the province’s three main political parties expressed concerns about how BC’s private sector privacy legislation, the Personal Information Protection Act (PIPA) (BC PIPA , 2019) might impact their ability to communicate with voters. Some argued that voter participation rates were in decline, and that it was already difficult enough to reach out to voters. Anything that further impaired methods of connecting with voters, like privacy regulation, would only make the problem worse, they said. My answer was this: can anyone seriously maintain that the Facebook/CA scandal has generated an increased desire on the part of citizens to participate in the electoral process? It is only when voters trust political parties to handle their data with integrity, and in a manner consistent with privacy law, that they will feel truly confident in engaging robustly in the political campaign system.

After some initial trepidation, these political parties, each with representatives in the legislative assembly, cooperated fully with my office’s investigation. It is important to stress we did not find abuses of personal data, of the kind exhibited in the Facebook/CA scandal. Nor did we find the sophisticated level of data collection and analytics associated with heavily funded US political campaigns. We did find, however, that the parties were collecting and using a lot of information about voters and had a clear appetite to do much more. So, our work was timely, and hopefully it will result in short-circuiting the worst excesses seen in other jurisdictions.

BC’s private sector privacy legislation is principle-based, and the predominant principle is consent. Consent was therefore the lens through which we assessed the parties’ actions. By that measure, many of their practices contravened our law and many others were at least legally questionable.

Like in many jurisdictions, BC’s political parties are entitled by law to receive a voters’ list of names and addresses from the Chief Electoral Officer (Elections BC, 2019). This information forms the basic building block upon which parties compile comprehensive voter profiles. We found what parties add to the voters’ list is sometimes done with consent, but in many cases, without. Door-to-door canvassing, the oldest and most basic method of gathering voter intelligence, is an example of this two-sided coin. The transparent element of this contact occurs when a voter voluntarily expresses support and provides a phone number or email for contact purposes. During the same visit, however, the canvasser might record, without permission, the voter’s ethnicity (or at least the canvasser’s best guess about the voter’s ethnicity). We found many instances of this type of information being downloaded in a party’s database.

We also found that parties used voter contact information in a way that was well beyond the voter’s expectation. The voter could expect to be called or emailed to be reminded to vote on election day. They would not expect, and did not consent to the party disclosing their personal information to Facebook. There is little question that Facebook has become the newest and best friend to almost all political parties. The company offers a rich gateway to parties to reach their supporters and potential supporters.

The problem is that neither the parties nor Facebook do very much to explain this to voters.

It starts with the fact that many, if not most, voters are Facebook users. The parties disclose their voters’ contact information to Facebook in the hope of matching them with their Facebook profiles. If successful, Facebook offers the party two valuable things. The first is the ability to advertise to these individuals in their Facebook newsfeed. Facebook gains revenue from this and is impliedly provided the opportunity to understand the political leanings of their users. The second use for matched voters contact information is Facebook’s analysis of the uploaded profiles to find common characteristics among them. When complete, it offers the party, for a price, the opportunity to advertise to these other Facebook users who “look like” the party’s supporters. This tool, which is also used by commercial businesses, provides an extremely effective means for political campaigns to reach an audience of potentially persuadable voters.

Reduced to its basics, what many parties do is gather voters’ contact information supposedly for direct communication purposes but instead disclose it to a social media giant for advertising and analytic purposes. It would understate things to say that these interactions with voters lack transparency.

All kinds of other data are also added and combined with basic voter information. Postal zone demographics and polling research for example are commonly deployed as parties attempt to attribute characteristics to voters with a view to targeting those they judge to be likely supporters. Most parties “score” voters on the likelihood of support.

Whether using these data sources to score voters is permitted by privacy law is a matter likely to be tested in the near future. What is clear, however, is that parties should be far more transparent about their actions, for no other reason than voters have a right to know what information parties have about them.

Political parties in BC and the UK have been slow to the realisation of this obligation. Parties in both jurisdictions told me that prediction data about a voter, for example their “persuadability score” was not, in fact, their personal information. In another instance, I was told that this score was a commercial secret that could be withheld from a voter. Such a stance does not breed public confidence and is contrary to privacy law in BC and most other jurisdictions.

What then does the future hold? Even the most cursory reflection on this question suggests the answers will come from multiple places. For my office, the first and most obvious ally in protecting the public interest is the province’s Chief Electoral Officer. He is not only the keeper of the voter list, he also tackles other immeasurably complex matters like election interference and disinformation campaigns. The need for us to work together is critical.

We have already embarked on a joint venture to develop a code of conduct for political parties which we hope BC political parties will adopt. Unlike the UK, which has a mechanism for the imposition of such codes, political parties in BC will have to voluntarily sign on. The benefit to parties is that everyone ends up playing by the same set of well-understood standards. It also means the public will have far greater confidence in their interactions with the parties, which hopefully will result in a far more robust campaign system. Thus far, the parties have accepted my investigation report’s recommendations and are working cooperatively with me and with the BC Chief Electoral Officer on developing the code.

The investigation into AIQ

Facebook is but one company political campaigns turn to. Of course, it is far from the only one. This brings us back to Victoria, BC, home base for AIQ (AggregateIQ, 2019). Among other things, AIQ developed “Project Ripon,” the architecture designed to make usable all of the data ingested by Cambridge Analytica. AIQ justified the non-consensual targeting of US voters on the basis that its American clients who collected the personal information at first instance had no legal obligation to seek consent.

My joint report on AIQ with the Office of the Privacy Commissioner of Canada (McEvoy & Therrien, 2019) determined that this was no legal answer. The fact is, they were a Canadian company operating in BC and were obligated to comply with BC law. This meant that AIQ had to exercise due diligence in seeking assurance from their clients that consent was employed to collect the personal information they intended to use. They obviously didn’t.

Subsequent events also undermined AIQ’s claim that the US data they worked with was lawfully obtained. The Federal Trade Commission found in late 2019 that Cambridge Analytica, working with app developer Aleksandr Kogan, deceived users by telling them they would not collect their personal information (Agreement Containing Consent Order as to Respondent Aleksandr Kogan, 2019). The message to Canadian companies operating globally is that they must observe the rules in the places that they work in and those of their home territory.

In the end, AIQ agreed to follow the recommendations of our joint report, cleaning up its practices to ensure, going forward, that they secure consent for the personal information used in client projects as well as improving security measures for safeguarding that information.

Conclusion

In the two years that have taken me from Victoria to the UK and back, the privacy landscape has changed dramatically. The public’s understanding of the privacy challenges we face as a society has been seismically altered. In the past, it was not uncommon for people to ask me at events, “Maybe I share a bit too much of my information on Facebook, but what could possibly go wrong with that?” . Facebook/Cambridge Analytica graphically demonstrated exactly what could go wrong. The idea that enormous numbers of people could be psychologically profiled for the purposes of political message targeting without their knowledge shocked people. The CanTrust Index (CanTrust Index, 2019) that tracks trust sentiment of major brands with Canadians found that Facebook’s reputation took a sharp nosedive with Canadians between 2017 and 2019, according to their most recent survey. In 2017, 51 per cent of Canadians trusted Facebook. Today, just 28 per cent say the same.

The underpinnings of the entire economic model now driving the internet and its social media platforms has been put on full public display. While few people can describe the detailed workings of real time bidding or a cookie’s inner mechanics, most comprehend that their daily activities across the web are tracked in meticulous detail.

While public awareness and concern have shifted markedly, action by legislators to address those concerns has in many jurisdictions tried to keep in step. It is true that the General Data Protection Regulation has set a new standard in Europe but even there, the more exacting ePrivacy Regulation has stalled (Bannerman, 2019). Canadian legislators have tried to be proactive in responding to privacy’s changing landscape. However, the Privacy Commissioner of Canada, as noted, is without direct order-making power. Neither of our offices have the authority to issue administrative penalties. It is little wonder citizens are left to ask “Who has my back?” when organisations violate data protection laws.

The road to reform will not be an easy one. There is considerable bureaucratic and corporate resistance to a stronger regulatory regime. Working together, regulators, academics, and civil society must continue to urge for legislative reform. Our efforts are strongly supported by public sentiment. The OPC’s 2019 survey on privacy (OPC, 2019) revealed that a substantial number of Canadians would be far more willing to transact with a business that was under an enhanced regulatory regime that included financial penalties for wrongdoers. That should be a signal to organisations, including political parties, that data protection is good for their business and that they too should support strengthened regulatory frameworks.

Introduction

For the last 50 years, Québec politics has been characterised by a lasting two-party system based on a dominant divide between the Yes and No options to the project of political independence from the rest of Canada of the 8.4 million people in Canada’s predominantly Francophone jurisdiction (Pelletier, 1989). Following the failure of the 1995 referendum, the erosion of this divide led to an openness of the partisan system and the arrival of four parties in the Québec National Assembly (Dufresne et al., 2019; Langlois, 2018). With a new party elected to government for the first time since 1976, the 2018 election was one of realignment. The Coalition avenir Québec (CAQ) elected 74 Members of the National Assembly (MNAs). With 31 seats, the former government, the Québec Liberal Party (QLP), received its worst result in 150 years and formed the official opposition. With 10 MNAs each, Québec solidaire (QS), a left-wing party and the Parti québécois (PQ), the historic vehicle for independence, occupied the remaining opposition seats.

Beyond these election results, the 2018 Québec election also marks an organisational change. For the first time, the major parties have all massively adopted what is often referred to as “US” data-campaigning practices. However, when it comes to the use of digital technologies for electoral purposes, the US case is the exception rather than the rule (Enli and Moe, 2013; Gibson, 2015; Vaccari, 2013, p. ix). Indeed, data campaigning, as with other techniques of political communication, are conducted in specific contexts that affect what is accessible, possible and viable (Bennett, 2016; Dobber et al., 2017; Ehrhard et al., 2019; Flanagan, 2010, p. 156).

Not unlike other Canadian jurisdictions, Québec is therefore an interesting case to study the effects of these practices in parties operating in a parliamentary system, while not being subject to privacy protection rules. Moreover, to our knowledge, studies on this subject in a sub-national context are few. In Canada, the majority of the work focuses on federal parties (see for example Bennett, 2018; McKelvey and Piebiak, 2018; Munroe and Munroe, 2018; Patten, 2015, 2017; Thomas, 2015), leaving provincial and municipal levels behind (with the notable exception of Carlile, 2017; Yawney, 2018; and Giasson et al., 2019). Thus, the French-speaking jurisdiction represents, as Giasson et al. (2019, p. 3) argue, one of those relevant but “less obvious” cases to study in order to better understand the similarities and differences in why and how political parties adopt or resist technological innovations. The use of this type of case study also makes it possible to explore the gap between emerging opportunities and the campaigns actually deployed by the parties, beyond the "rhetoric of data-driven campaigning" (see Baldwin-Philippi, 2017, p. 627).

Many factors influence technological innovation in campaigns (Kreiss, 2016). Furthermore, as Hersh indicates (2015), cultural and legal contexts influence political actors’ behaviour because types of data that are made available to campaigns shape their perceptions of voters, and therefore their communication practices. According to Munroe and Munroe (2018), political parties may use data as a resource generated in many ways that can be used to guide strategic and tactical decisions. Because parties set up integrated platforms in which personal data on voters are stored and analysed, ethical and political issues emerge (Bennett, 2013, 2015). In most Canadian provinces, including Québec, and at the federal level, parties are not subjected to privacy laws regarding the use and protection of personal data. This absence of a regulatory framework also leads to inadequate self-regulation (Bennett, 2018; Howard and Kreiss, 2010).

As was the case in many other jurisdictions around the globe, Québec parties were faced with a transparency deficit following the March 2018 revelations of the Cambridge Analytica affair (Bashyakarla et al, 2019; Cadwalladr and Graham-Harrison, 2018). Within hours of the scandal becoming public, political reporters in Québec turned to party leaders to get a better sense of the scope and use of the digital data they were collecting, why they collected them and what this all meant for the upcoming fall elections as well as for citizens’ privacy (Bélair-Cirino, 2018). Most claimed that their data collection and analysis practices were ethical and respectful of citizen’s privacy. However, none of them agreed to fully disclose the scope of the data they collected nor the exact purpose of these databases.

Research objectives and methodology

This article examines the increasing pressure to regulate uses of digital personal data by Québec’s political parties. First, it illustrates the central role now played by voter personal data in Québec’s politics. Second, it presents the current (and weak) legislative framework and how the issue of the protection of personal data came onto the agenda in Québec. At first, many saw this shift has a positive evolution where Québec’s parties “caught up” with current digital marketing practices. However, following the Cambridge Analytica affair and revelations about the lack of proper regulation on voter data use, public discourse started casting these technological advancements as democratic catastrophes waiting to happen.

We use three types of data to investigate this context. First, in order to assess the growth in party use of digital voter data, we rely on 40 semi-directed interviews conducted for a broader research project with party organisers, elected officials, activists and advisors of all the main political parties operating in Québec 1. The interviews, each lasting from 45 minutes to 1-hour - were conducted in French just a few weeks before the launch of the 2018 provincial election campaign. Citations presented in this article are therefore translations. The interviewees were selected according to their political representativeness, but also for their high level of electoral involvement. In this article, we only use those responses that relate to digital campaigning and the use of personal information. The citations selected here represented viewpoints shared by at least three interviewees. They illustrate shared perceptions of the evolution of the strategic use of voter personal data in Québec’s electioneering.

Second, we also analysed the legislative framework as well as the self-regulatory practices of political parties in Québec in order to measure the levels of regulation and transparency surrounding their use of personal data. To do this, we studied the websites of the four main parties in order to compare their practices.

Finally, we also conducted a media coverage analysis on the issue of how parties engaged in digital marketing. We conducted a keyword search on the Eureka.cc database to retrieve all texts published in the four main daily newspapers published in French in Québec (La Presse, Le Devoir, Le Soleil and Le Journal de Montréal), in the public affairs magazine L’Actualité, as well as on the Radio-Canada website about digital data issues related to politics in Québec. The time period runs from 1 January 2012 to 1 March 2019 and covers three general (2012, 2014 and 2018) and two municipal (2013 and 2017) elections. Our search returned 223 news articles.

What we find is a perfect storm. We saw parties that are massively adopting data marketing at the same time that regulatory bodies expressed concerns about their lack of supervision. In the background, an international scandal made the headlines and changed the prevailing discourse surrounding these technological innovations.

New digital tools, a new political reality

The increased use of digital technologies and data for electioneering can be traced back to the 2012 provincial election (see Giasson et al., 2019). Québec political parties were then faced with a changing electorate, and data collection helped them adapt to this new context. Most of them also experienced greater difficulties in rallying electors ideologically. In Québec, activist, partisan politics was giving way to more political data-marketing (Del Duchetto, 2016).

In 2018, Québec’s four main political parties integrated digital technologies at the core of their electoral organisations. In doing so, they aimed to close the technological gap with Canadian parties at the federal level (Marland et al., 2012; Delacourt, 2013). Thus, the CAQ developed the Coaliste, its own tool for processing and analysing data. The application centralises information collected on voters in a database and targets them according to their profile. Developed at a cost of 1 million Canadian dollars, the tool was said by a party strategist to help carry a campaign "with 3 or 4 times less" money than before (Blais and Robillard, 2017).

For its part, QS created a mobilisation platform called Mouvement. The tool was inspired by the "popular campaigns of Bernie Sanders and La France Insoumise in France."2 Decentralised in nature, the platform aimed to facilitate event organisation, networking between sympathisers, to create local discussion activities, as well as to facilitate voter identification.

The PQ has also developed its own tool: Force bleue. At its official launch, a party organiser insisted on its strategic role in tight races. It would include “an intelligent mapping system to crisscross constituencies, villages, neighbourhoods to maximise the time spent by local teams and candidates by targeting the highest paying places in votes and simplify your vote turnout” (Bergeron, 2018).

Finally, the QLP outsourced its digital marketing and built on the experience of the federal Liberal Party of Canada as well as Emmanuel Macron’s movement in France. For the 2018 election campaign, the party contracted Data Sciences, a private firm which "collects information from data of all kind, statistics among others, on trends or expectations of targeted citizens or groups"(Salvet, 2018).

Our interviews with political strategists help better understand the scope of this digital shift that Québec’s parties completed in 2018. They also put into perspective the effects of these changes and the questions they raise within the parties themselves.

Why change?

Party organisers interviewed for this article who advocate for the development of new tools stress two phenomena. On the one hand, the Québec electorate is more volatile and on the other, it is much more difficult to communicate with electors than before. A former MNA notes that today: "The campaign counts. It's very volatile and identifying who votes for you early in the campaign doesn’t work anymore. "

With social media, Québec parties’ officials see citizens as more segmented than before. An organiser attributes the evolution of this electoral behaviour to social media. "Today, the big change is that the speed and accessibility of information means that you do not need a membership card to be connected. It circulates freely. It's on Facebook. It’s on Twitter".

He notes that "it is much more difficult to attract someone in a political party by saying that if you become a member you will have privileged access to a certain amount of information or to a certain quality of information". A rival organiser also confirms that people's behaviour has changed: "It's not just generational, they buy a product". He adds that this has implications on the level of volunteering and on voters’ motivation:

When we look at the beginning of the 1970s, we had a lot of people. People were willing to go door-to-door to meet voters. We had people on the ground, they needed to touch each other. The communications were person-to-person. (…) Today, we do marketing.

In sum, "people seek a product and are less loyal" which means that parties must rely on voters’ profiling and targeting.

Increased use of digital technology in 2018

The IT turn in Québec partisan organisations is real. One organiser goes so far as to say that most of the volunteer work that was central in the past is now done digitally. According to him, "any young voter who uses Facebook, is now as important, if not more, than a party activist". This comment reinforces the notion that any communication with an elector must now be personalised:

Now we need competent people in computer science, because we use platforms, email lists. When I send a message reminding to newly registered voters that it will be the first time they will vote, I am speaking directly to them.

To achieve this micro-targeting, party databases are updated constantly. An organiser states that: "Our job is to feed this database with all the tools like surveys, etc... In short, we must bomb the population with all kinds of things, to acquire as much data as possible". For example, Québec solidaire and the Coalition avenir Québec broadly used partisan e-petitions to feed their database (Bélair-Cirino, 2017). There are neither rules nor legislation that currently limit the collection and use of this personal information if it is collected through a partisan online petition or website.

Old political objectives - new digital techniques

In accordance with the current literature on the hybridisation of electoral campaigns (Chadwick, 2013; Giasson et al., 2019), many respondents indicate that the integration of digital tools associated with data marketing has changed the way things are done. This also had an effect on the internal party organisation, as well as on the tasks given to members on the ground. An organiser explains how this evolution took place in just a few years:

Before, we had a field organisation sector, with people on the phones, distributors, all that. We had communication people, we had people distributing content. (...) Right now, we have to work with people that are not there physically and with something that I will not necessarily control.

An organiser from another political party is more nuanced: "We always need people to help us find phone numbers, we always need people to make calls". He confirms, however, that communication tactics changed radically:

The way to target voters in a riding has changed. The way to start a campaign, to canvas, has changed. The technological tools at our disposal means that we need more people who are able to use them and who have the skills and knowledge to use the new technological means we have to reach the electorate.

Another organiser adds that it is now important to train activists properly for their canvassing work. According to her: "We need to give activists digital tools and highly technological support tools that make their lives easier". She adds that: "Everything is chained with intelligent algorithms that will always target the best customer, always first, no matter what...".

New digital technologies and tools are therefore used to maximise efficiency and resources. The tasks entrusted to activists also change. For another organiser, mobilisation evolves with technology: "We used to rely on lots of people to reach for electors". He now sees that people are reached via the internet and that this new reality is not without challenges: "we are witnessing a revolution where new clients do not live in the real world…". It then becomes difficult to meet them in real life, off-line.

Another organiser confirms having "a different canvas technique using social media and other tools”. According to him:

Big data is already outdated. We are talking about smart data. These data are used efficiently and intelligently. How do we collect this data? (...) We used to do a lot of tally by door-to-door or by phone. Now we do a lot of capture. The emails are what interest me. I am not interested in phone numbers anymore, except cell phones.

An experienced organiser observes that "this has completely changed the game. Before, we only had one IT person, now I have three programmers". He adds that "liaison officers have become press officers". This change also translates in the allocation of resources and the integration of new profiles of employees for data management. It brought a new set of digital strategists into war rooms. These new data analysts have knowledge in data management, applied mathematics, computer science and software engineering. They are working alongside traditional field organisers, sometimes even replacing them at the decision table.

Second thoughts

Organisers themselves raise democratic and ethical concerns related to the digital evolution of their work. One of them points out that they face ethical challenges. He openly wonders about the consequences of this gathering of personal information: "It's not because we can do something that we have to do it. With the list of electors, there are many things that can be done. Is it ethical to do it? At some point, you have to ask that question". He points out that new technologies are changing at a rapid pace and that with "each technology comes a communication opportunity". The question is now "how can we appropriate this technology, this communication opportunity, and make good use of it".

Reflecting upon the lack of regulation on the use of personal data by parties in Québec, an organiser added that: "We have the right to do that, but people do not like it". For him, this issue is "more than a question of law, there could be a question of what is socially acceptable".

Another organiser points out that the digital shift could also undermine intra-party democracy. Speaking about the role of activists, he is concerned that "they feel more like being given information that has been chewed on by a small number of people than being collected by more people in each constituency". He notes that the technological divide is also accompanied by a generational divide within the activist base:

The activist who is older, we will probably have less need of him. The younger activist is likely to be needed, but in smaller numbers. (...) Because of the technological gap, it's a bit of a vicious circle, that is also virtuous. The more we try to find technological means that will be effective, the less we need people.

Still, democratically, the line can be very thin between mobilisation and manipulation. Reflecting on a not-so-distant future, this organiser spoke of the many possibilities data collection could provide parties with:

These changes bring us into a dynamic that the Americans call ‘activation fields’. (...) From the moment we have contact with someone, what do we do with this person, where does she go? (...) This gives incredible arborescence, but also incredible opportunities.

He concludes that: "Today, the world does not realise how all the data is piling up on people and that this is how elections are won now". Is there a limit to the information a party could collect on an elector? This senior staffer does not believe so. He adds: “If I could know everything you were consuming, it would be so useful to me and help mobilise you".

Québec’s main political parties completed their digital shift in preparation for the 2018 election. Our interviews show that this change was significant. From an internal democracy perspective, digital technologies and data marketing practices help respond to the decline of activism and membership levels observed in most Québec parties (Montigny, 2015). This can also lead to frustration among older party activists who would feel less involved. On the other hand, from a data protection perspective we note that in the absence of a rigorous regulatory framework, parties in Québec can do almost anything. As a result, they collect a significant amount of unprotected personal data. The pace at which this change is taking place and the risks it represents for data security even lead some political organisers to question their own practices. As the next section indicates, Québec is lagging behind in adapting the data marketing practices of political parties to contemporary privacy standards.

The protection of personal information over time

The data contained in the Québec list of electors has been the cornerstone of all political parties’ electioneering efforts for many years and now form the basis of their respective databases of voter information. It is from this list that they are able, with the addition of other information collected or purchased, to file, segment and target voters. An overview of the legislative amendments concerning the disclosure of the information contained in the list of electors reveals two things: (1) its relatively recent private nature, and (2) the fact that the ability for political parties to collect and use personal data about voters never really seems to have been questioned until recently. Parties mostly reacted by insisting on self-regulation (Élections Québec, 2019).

With regard to the public/private nature of the list of electors, we should note that prior to 1979 it was displayed in public places. Up to 2001, the list of electors of a polling division was even distributed to all voters in that section. Therefore, the list used to be perceived as a public document in order to prevent electoral fraud. Thus, citizens were able to identify potential errors and irregularities.

From 1972 on, the list has been sent to political parties. With the introduction of a permanent list of electors in 1995, political parties and MNAs were granted, in 1997, the right to receive annual copies of the list for verification purposes. Since 2006, parties receive an updated version of the list three times a year. This facilitates the update of their computerised voter databases. It should also be noted that during election periods, all registered electoral candidates are granted access to the list and its content.

Thus, while public access to the list of electors has been considerably reduced, political parties’ access has increased in recent years. Following legislative changes, some information has been removed from the list, the age and profession of the elector for instance. Yet, the Québec list remains the most exhaustive of any Canadian jurisdiction in terms of the quantity of voter information it contains, indicating the name, full address, gender and date of birth of each elector (Élections Québec, 2019, p. 34).

From a legal perspective, Québec parties are not subject to the "two general laws that govern the protection of personal information, namely the Act respecting access to documents held by public bodies and the protection of personal information, which applies in particular to information held by a public body, and the Act respecting the protection of personal information in the private sector, which concerns personal information held by a person carrying on a business within the meaning of section 1525 of the Civil Code of Québec" (Élections Québec, 2019, p. 27). Indirectly, however, this law would apply when a political party chooses to outsource some of its marketing, data collection or digital activities to a private sector firm.

Moreover, the Election Act does not specifically define which uses of data taken from the list of electors are permitted. It merely provides some general provisions. Therefore, parties cannot use or communicate a voter’s information for purposes other than those provided under the Act. It is also illegal to communicate or allow this information to be disclosed to any person who is not lawfully entitled to it.

Instead of strengthening the law, parties represented in the National Assembly first chose to adopt their own privacy and confidentiality policies. This form of self-regulation, however, has its limits. Even if they appear on their websites, these norms are usually not easy to find and there is no way to confirm that they are effectively enforced by parties. Only the Coalition avenir Québec and the Québec Liberal Party offer a clear link on their homepage. 3 We analysed each of these according to five indicators: the presence of 1) a definition of what constitutes personal information, 2) a reference to the type of use and sharing of data, 3) methods of data collection, 4) privacy and security measures that are taken and 5) the possibility for an individual to withdraw his or her consent and contact the party in connection with his or her personal information.

In general, we find that three out of four parties offer similar definitions of the notion of personal information: the Coalition avenir Québec, the Liberal Party of Québec and Québec solidaire. Beyond this indicator, the information available varies from one party to another. Thus, voters have little information on the types of use of their personal data. Moreover, only the Coalition avenir Québec and Québec solidaire indicate that they can use a third party in the processing of data without having to state the purpose of this processing to the data providers. The Coalition avenir Québec is the only party that specifies its methods of data collection in more detail. Similarly, Québec solidaire is more specific with respect to the measures taken to protect the privacy and security of the data it collects. Finally, the Parti québécois does not specify the mechanism by which electors could withdraw their consent.

Cambridge Analytica as a turning point

Our analysis of media coverage of the partisan and electoral use of voter data in Québecreveals three main conclusions. First, even though Québec political parties, both at the provincial and municipal levels, began collecting, storing and using personal data on voters several years ago, news media attention on these practices is relatively new. Secondly, the dominant media frame on the issue seems to have changed over the years: after first being rather anecdotal, the treatment of the issue grew in importance and became more suspicious. Finally, the Cambridge Analytica scandal appears as a turning point in news coverage. It is this affair that will force parties and their strategists to explain their practices publicly for the first time (Bélair-Cirino, 2018), will put pressure on the government to react, and bring to the fore the concerns and demands of other organisations such as Élections Québec and the Commission d’accès à l’information du Québec, the administrative tribunal and oversight body responsible for the protection of personal information in provincial public agencies and private enterprises.

Interest in ethical and security issues related to data campaigning built up slowly in Québec’s political news coverage. Already in 2012, parties used technological means to feed their databases and target the electorate (Giasson et al., 2019). However, it is in the context of the municipal elections in the Fall of 2013 that the issue of the collection and processing of personal data on voters was first covered in a news report. It was only shortly after the 2014 Québec elections that we found a news item dealing specifically with the protection of personal data of Québec voters. The Montréal-based newspaper Le Devoir reported that the list of electors was made available online by a genealogy institute. It was even possible to get it for a fee. The Drouin Institute - which released the list - estimated that about 20,000 people had accessed the data (Fortier, 2014).

Paradoxically, the following year, the media reported that investigators working for Élections Québec could not access the data of the electoral list for the purpose of their inquiry (Lajoie, 2015a). That same year, another anecdotal event made headlines: a Liberal MNA was asked by Élections Québec to stop using the voters list data to call his constituents to... wish them a happy birthday (Lajoie, 2015b). In the 2017 municipal elections, and even more so after the revelations regarding Cambridge Analytica in 2018, the media in Québec seemed to have paid more attention to data-driven electoral party strategies than to the protection of personal data by the parties.

For instance, in the hours following the revelation of the Cambridge Analytica scandal, political reporters covering the National Assembly in Québec quickly turned their attention to the leadership of political parties, asking them to report on their respective organisations’ digital practices and about the regulations in place to frame them. Simultaneously, Élections Québec, which had been calling for stronger control of personal data use by political parties since 2013, expressed its concerns publicly and fully joined the public debate. As a way to mark its willingness to act on the issue, the liberal government introduced a bill at the end of the parliamentary session, the last of this parliament. The bill was therefore never adopted by the House, which was dissolved a few days later, in preparation for the next provincial election.

Political reporters in Québec have since then paid sustained attention to partisan practices regarding the collection and use of personal information. In their coverage of the 2018 election campaign, they widely discussed the use of data by leaders and their political parties. Thus, while the Cambridge Analytica affair did not directly affect Québec political parties, it nevertheless appears as a shifting point in the media coverage of the use of personal data for political purposes.

Media framing of the issue also evolved over the studied period, becoming more critical and suspicious of partisan data marketing with time. Before the Cambridge Analytica case, coverage rarely focused on the democratic consequences or privacy and security issues associated with the use of personal data for political purposes. Initial coverage seems to have been largely dominated by the story depicting how parties were innovating in electioneering and on how digital technologies could improve electoral communication. Journalists mostly cited the official discourse of political leaders, their strategists or of the digital entrepreneurs from tech companies who worked with them.

An illustrative example of this type of coverage can be found in an article published in September 2013 during municipal elections held in Québec. It presents a portrait of two Montréal-based data analysis companies – Democratik and Vote Rapide – offering technological services to political parties (Champagne, 2013). Their tools were depicted as simple databases fed by volunteers, mainly intended for the identification of sympathisers to facilitate the get-out-the-vote operations (GOTV). It emphasised the affordability and universal use of these programmes by parties, and even indicated that one of them had been developed with the support of the Civic Action League, a non-profit organisation that helps fight political corruption.

However, as the years passed, a change of tone began to permeate the coverage, especially in the months building up to the 2018 general election. A critical frame became more obvious in reporting. It even used Orwellian references to data campaigning in titles such as… “Political parties are spying on you” (Castonguay, 2015) “They all have a file on you” (Joncas, 2018), “What parties know about you” (Croteau, 2018), or “Political parties exchange your personal details” (Robichaud, 2018). In a short period of time, data campaigning had gone from cool to dangerous.

Conclusion

Québec political parties began their digital shift a few years later than their Canadian federal counterparts. However, they have adapted their digital marketing practices rapidly; much faster in fact than the regulatory framework. For the 2018 election, all major parties invested a great deal of resources to be up to date on data-driven campaigning.

To maximise the return on their investment in technology, they must now “feed the beast” with more data. Benefiting from weak regulation over data marketing, this means that they will be able to gather even more personal information in the years to come, without having to explain to voters what their data are used for or how they are protected. In addition, parties are now involving an increasing number of volunteers in the field for the collection of digital personal information, which also increases the risk of data leakage or misuse.

They have, so far, implemented that change with very limited transparency. Up until now, research in Canada has not been able to identify precisely what kind of information is collected or how it is managed and protected. Canadian political strategists have been somewhat forthcoming in explaining how parties collect and why they use personal data for electoral purposes (see for instance Giasson et al., 2019; Giasson and Small, 2017; Flanagan, 2014; Marland, 2016). They however remain silent on the topics of regulation and data protection.

This lack of transparency is problematic in Canada since party leaders who win elections have much more internal powers in British parliamentary systems then in the US presidential system. They control the executive and legislative branches as well as the administration of the party. This means that there is no firewall and real restrictions to the use of data collected by a party during an election once it arrives in office. In that regard, it was revealed that the Office of the Prime Minister of Canada, Justin Trudeau, used its party’s database to vet judges’ nominations (Bergeron, 2019). The same risks apply to Québec.

It is in this context that Élections Québec and the Access to Information Commission of Québec have initiated a broad reflection on the electoral use of personal data by parties. In 2018, following a leak of personal data from donors of a Montréal-based municipal party, the commission contacted the campaign to "examine the measures taken to minimise risks". The commission took the opportunity to "emphasise the importance of political parties being clearly subject to privacy rules, as is the case in British Columbia" (Commission d’accès à l’information du Québec, 2018).

In a report published in February 2019, the Chief Electoral Officer of Québec presented recommendations that parties should follow in their voter data collection and analysis procedures (Élections Québec, 2019). It suggested that provincial and municipal political parties be submitted to a general legislative framework for the protection of personal information. Heeding these calls for change, Québec’s new Minister of Justice and Democratic Reform announced, in November 2019, plans for an overhaul of the province’s regulatory framework on personal data and privacy, which would impose stronger regulations on data protection and use and would grant increased investigation powers to the head of the Commission d’accès à l’information. All businesses, organisations, governments and public administrations operating in Québec and collecting personal data would be covered under these news provisions and could be subjected to massive fines for any form of data breach in their systems. Aimed at ensuring better control, transparency and consent of citizens over their data, these measures, which will be part of a bill introduced in 2020 to the National Assembly, were said to also apply to political parties (Croteau, 2019). However, as this article goes to print, the specific details of these new provisions aimed at political parties remain unknown.

This new will to regulate political parties is the result of a perfect storm where three factors came into play at the same time. Thus, in addition to the rapid integration of new data collection technologies by Québec’s main political parties, there was increased pressure from regulatory agencies and an international scandal that changed the media framing of the political use of personal data.

Well beyond the issue of privacy, data collection and analysis for electoral purposes also change some features of our democracy. Technology replacing activists translates in major intra-party changes. In a parliamentary system, this could increase the centralisation of power around party leaders who now rely less on party members to get elected. This would likely be the case in Québec and in Canada.

Some elements also fuel resistance to change within parties, such as the dependence on digital technologies at the detriment of human contact, fears regarding the reliability of systems or data and the high costs generated by the development and maintenance of databases. For some, party culture also plays a role. A former political strategist who worked closely with former Québec Premier Pauline Marois declared in the media: "You know in some parties, we value the activist work done by old ladies who come to make calls and talk to each voter, one by one" (Radio-Canada, 2017).

As some of our respondents mentioned, parties may move from ‘big data’ to ‘smart data’ in coming years, as they adapt to or adopt novel technological tools. In an era of partisan flexibility, data marketing seems to have helped some parties find and reach their voters. A move towards ‘smart data’ may now also help them modify those voters’ beliefs with even more targeted digital strategies. What might this mean for democracy in Québec? Will its voters be mobilised or manipulated when parties will use their data in upcoming campaigns? Are political parties on the edge of glory or of catastrophe? These questions should be central to the study of data-driven campaigning.

Data-driven elections: implications and challenges for democratic societies

Introduction

As this special issue on data-driven elections was being prepared, the major social media platforms were making some diverse decisions about political advertising. Twitter declared that it was banning paid political advertising from the platform, while allowing “issue ads”; Google announced that it would ban the more targeted or granular political ads, and restrict advertisers’ ability to target political ads just to age, gender and zip code; and Facebook committed to improve ad transparency and to giving users the option of seeing fewer political ads in their newsfeed (Leathern, 2020), but has also insisted that it should not be in the business of fact-checking or censorship (Stewart, 2019). These decisions have inspired heated debate about their motivations and effects. They reflect a new realisation that elections are, to some extent, determined by the capture of personal data, and won and lost by the parties and candidates that can most effectively target voters based on those data.

Questions about the misuse and abuse of personal data in the electoral process came to global public attention as a result of the Facebook/Cambridge Analytica scandal (Cadwalladr, 2017). The global publicity elevated questions about the use of personal data in contemporary political campaigns to new levels, and to a huge set of issues about the integrity of modern elections, their vulnerability to the spread of misinformation and “fake news” especially from foreign sources, and to the accountability of the major social media platforms.

Of course, questions about the use of personal data are raised in many other areas besides political campaigns and these are fruitfully considered under the rubric of surveillance, now often described as operating in a ‘surveillance-capitalist’ mode. Several authors have discussed surveillance capitalism (Mosco, 2014; Foster & McChesney, 2014; Fuchs 2017), but the work of Zuboff (2015; 2019) has served to galvanise the debate. Those taking this view contend that the commodification of a mass of personal data, gathered and sorted from largely unwitting users, has now become a dominant mode of accumulation. The classification of those data enables their use in multiple settings, including the present context of elections. The Cambridge Analytica scandal would not have been such without Facebook, for which both ‘prediction’ and ‘personalisation’ are central. We have known about the potential for Facebook to engage in “digital gerrymandering” for several years (Zittrain, 2014).

Contemporary surveillance has several features that resonate with questions raised by data-driven elections. It sorts populations into groups so that they may be treated differently, which is often divisive in its effects. It assumes that classificatory algorithms work effectively to encapsulate user opinions, thus questioning users’ self-positioning. The sorting processes also act to admit and restrict participation. The shift to electronically-mediated relationships threatens to undermine conventional reliance on face-to-face communication in some critical areas, and produce potential shifts in governance to a volatile and more fluid frame (Lyon & Baumann, 2013).

In the political world, these sorting processes are often discussed as voter analytics, which in turn facilitates ‘political micro-targeting’. According to the UK Information Commissioner micro-targeting “describes targeting techniques that use data analytics to identify the specific interests of individuals, create more relevant or personalised messaging targeting those individuals, predict the impact of that messaging, and then deliver that messaging directly to them” (ICO, 2018, p. 27). It represents a shift from geographic based targeting to more individualised messaging based on predictive models and scoring. According to the former technology advisor in the Obama White House, micro-targeting relies upon the cultivation of a range of compelling and addictive services, the construction of behavioural tracking profiles, the development of algorithms designed to keep us scrolling, watching and clicking, and the interspersing of ads throughout that content in order to produce optimal revenue (Ghosh, 2018). The same logic and techniques of consumer surveillance have entered the political world: “political parties are using the same techniques to sell political candidates to voters that companies use to sell shoes to consumers” (Tactical Tech, 2019).

The principal concerns

What are the broader effects of treating voters like consumers to whom candidates and political parties can “shop for votes” (Delacourt, 2017)? In a 2017 special issue of this journal, the guest editors asked whether political micro-targeting is a “manchurian candidate or just a dark horse” (Bodó, Helberger, & de Vreese, 2017). Since that 2017 issue was published, the various normative concerns about data-driven elections, and their impact on democratic values are coming more sharply into focus (Bennett & Oduro Marfo, 2019).

There are profound concerns about divisiveness. Do data-driven elections lead to an increased tendency to deliver messages on “wedge issues”? Do they produce “filter bubbles” or “echo chambers” when individuals only see a subset of information algorithmically curated according to their presumed and prior interests and behaviours? Do they reinforce partisanship and a fragmentation of the political space?

There are a related set of concerns about the effect on the “marketplace of ideas” when false advertising cannot be countered in real time. In the open, false claims might be challenged. In secret, they may stand unchallenged. The opaqueness of much contemporary political messaging blocks the presumed self-correcting benefits of rights to freedom of expression.

There are concerns about political participation and engagement. Does this precise segmentation reduce the portion of the electorate that politicians need to campaign to and for, and ultimately care about after the election? Are the interests of others then ignored, or marginalised? More widely, do data-driven elections contribute to a decline in political participation, as voters perceive that their interests are being manipulated by political and technical elites?

There are questions about the effects on campaigning itself. Do data-driven campaigns reinforce ‘permanent campaigns’ where parties have the capacity to make voter contact a more enduring enterprise, before, during and after official election campaigns? Do they discourage volunteering for political parties? Do they erode the face-to-face contact with voters which are common in those countries where door-to-door canvassing is part of the political culture? Do data-driven elections favour larger and more established political parties, which have the resources to employ the technical consultants who manage the data and coordinate the messaging?

Finally, there are also concerns about its effects on governance. When one message is given to one group of voters, and another to another group of voters, do data-driven elections lead to more ambiguous political mandates for elected representatives (Barocas, 2012, p. 33)? In larger terms, does it even encourage patron-client forms of politics (Hersh, 2015, p. 209)?

Questions about the legitimate processing of personal data on the electorate is at the heart of the answer to each of these larger questions. The conduct of voter analytics and the micro-targeting of political messages, including the delivery of so-called “fake news” has a direct relationship to programmatic advertising, and to the impersonal algorithms that target individual citizens, often without their knowledge and consent. Familiar privacy questions are now injected into this heated international debate about democratic practices and regulators, such as data protection authorities (DPAs), now find themselves at the centre of a global conversation about the future of democracy.

Furthermore, elected officials the world over have come to realise that the inappropriate processing of personal data within elections can hurt them where it hurts most – at the ballot box. Thus, “privacy and data protection have rarely in the past been ‘Big P’ political questions. They are now” (Bennett & Oduro Marfo, 2019, p. 3).

The articles in this special issue

The articles and commentaries presented in this special issue originated in a research workshop, organised by the Big Data Surveillance project centred at Queen’s University, and hosted by the Office of the Information and Privacy Commissioner for British Columbia in April 2019. It brought together a vibrant mix of international scholars in surveillance studies and political communication, plus civil society advocates and regulators from across Canada. Throughout the entirety of the workshop, we were very fortunate to enjoy the presence of Carole Cadwalladr, the Guardian journalist who broke the original story about Cambridge Analytica and the Brexit referendum (Cadwalladr, 2016).

Michael McEvoy, the current Information and Privacy Commissioner for British Columbia has played a central role in some of the very first investigations by DPAs into privacy and election campaigns. While on secondment to the Office of the Information Commissioner (ICO) in the UK, he was one of the first to interview whistleblower Christopher Wylie. He also led the initial work of the ICO into the practices of British political parties. On his return to BC, he initiated a broad investigation into the operations of political parties in BC, and conducted joint investigations with the Office of the Privacy Commissioner of Canada (OPC) into the breach of Facebook data to Cambridge Analytica, as well as into the Victoria-based company AggregateIQ Data Services (AIQ). Michael McEvoy shares his reflections on these experiences, from the perspective of a regulator, in the commentary below.

The April 2019 workshop highlighted the current contours of the international debate – ongoing dilemmas that will require ongoing research, as well as attention by domestic and international regulators. It brought to light some essential questions about current and future practices, that should serve as a guide for future scholarly inquiry as well as for national and international policy. Five such questions follow.

Myths versus realities?

Digital campaigning has long been pitched as key to electoral success, in the US and increasingly in other countries. And politicians have bought into the premise that they can win elections if they just have better, more refined, and more accurate data on the electorate. The better campaigns ‘know’ voters, the better able are they to profile them and target them with increasingly precise messages.

Of course, the role that data and data analytics has played in electoral politics has been a matter of scholarly interest for several years. All modern campaigns in all democracies use data – even if it is simply polling data. But now the full power of “Big Data” has been unleashed: from massive voter relationship management platforms, to digital campaigning practices that leverage the enormous potential of social media and mobile applications. In a recent report (Tactical Tech, 2019), analysed in the commentary below by Varoon Bashyakarla (2019), the Tactical Tech collective has portrayed the extensive contemporary political “influence industry”.

Bashyakarla’s commentary makes a useful distinction between data as a political asset (through traditional databases or voter relationship management systems), as political intelligence (through constant A/B testing and experimentation), and as political influence (through micro-targeting techniques). It documents the range of companies, consultancies, agencies and marketing firms, from local start-ups to global strategists, that target parties and campaigns across the political spectrum, often with militaristic rhetoric - “we win the tough fights”, “we power democracy”, “ignite your cause”, “your revolution starts here” (Tactical Tech, 2019). Bashyakarla contends that the question “does this targeting work” reflects a short-sighted obsession with winning, and misses the far larger point about the effect of the weaponisation of personal data on the larger democratic infrastructure.

The work of Jeff Chester and Kathryn Montgomery has traced the ongoing “marriage of politics and commerce” and the growth of data-driven political marketing (Chester & Montgomery, 2017). They reviewed seven key techniques employed during the 2016 campaigns in the US, all of which point to massive efforts at consolidation in the digital marketing ecosystem: cross-device targeting; programmatic advertising; lookalike modelling, such as that offered through Facebook; online video advertising; targeted TV advertising; and psychographic, neuromarketing and emotion-based targeting. In their new article in this collection, they extend this analysis and preview the kinds of practices likely to be witnessed in the 2020 US election campaigns (Chester & Montgomery, 2019).

At the same time, the power of data-driven elections can be overstated. As Jessica Baldwin-Philippi’s article shows, evidence of how and whether data analytics actually does win elections is very difficult to determine empirically. Data-driven campaign strategies are perhaps far more effective at mobilising adherents and donors, than in persuading undecided voters. Emphasis on scale often substitutes for claims of effectiveness. At one stage, Cambridge Analytica claimed to have around 5,000 different data points on the American electorate. They were not alone. The voter analytics industry in the US, including companies like Catalist, i360, and HaystaqDNA have claimed an extraordinary volume of personal data under their control – free and purchased, from public and commercial sources. Such claims about “Big Data” reinforce more widespread narratives about the hegemony and glorification of the size and granularity of the databases over supportable claims about effectiveness (Baldwin-Philippi, 2019).

The US versus the rest of the world?

The mythology of big data analytics in elections is also associated with a trend of “Americanization”. With very few exceptions, voter analytics practices have been pioneered in the US and exported to other democratic countries. There are many conditions in the US (the liberal campaign financing system, the unprecedented amount of publicly available data, the thriving data mining industry, and the relative weakness of data privacy laws) which produce favourable grounds for data-driven elections to flourish (Bennett, 2013; Rubinstein, 2014).

On the one hand, the US influence has been felt through the active efforts of American consultants, and especially those who worked on the 2008 and 2012 Obama campaigns, who have been promoting the power of voter analytics in other countries. US consultants have advised on the development of voter relationship management systems for some overseas political parties. The Canadian Liberal Party, for example, uses the software developed by NGP VAN, the main technology provider for the US Democratic Party (Bennett, 2015). Digital analysts who have worked in the US have also begun start-up companies in other countries, an example being Liegey Muller Pons (now trading as eXplain) - which has worked on several European campaigns, including that of the En Marche party of French President Emmanuel Macron (Duportail, 2018).

The most notable American influence, however, is through the use of social media platforms, and the affordances they provide for campaigning in different contexts. WhatsApp has become a particularly powerful campaigning instrument. Easy to use, end-to-end encrypted and facilitating the sharing of messages to large groups, WhatsApp has been extremely popular in countries like India (Hickok, 2018), Brazil and other countries in the Global South. However, WhatsApp not only allows parties to tailor messages to precise groups, it also offers anonymity, thus making it easy to misrepresent a sender’s identity with the predictable and widespread concerns about the delivery of “fake news” and hate-inciting messages. Rafael Evangelista and Fernanda Bruno (2019) demonstrate the pernicious use of WhatsApp in Brazil for the spread of racist, misogynistic and homophobic messages by the Bolsonaro campaign. Their analysis suggests that WhatsApp relies upon a more trusting relationship between group members, than is apparent within other social media. It therefore produces a more susceptible medium for the spread of misinformation.

This case also highlights how voter surveillance techniques are going to be shaped by political culture, and in particular the general acceptability of direct candidate-to-voter campaigning practices, such as door-to-door canvassing, or telephone polling. In some societies, it is not customary for voters to display symbols of political affiliation on their persons, their cars or their houses – as it is in others. In countries with recent memories of authoritarian rule, the sensitivity of data on political affiliation is particularly acute (Bennett & Oduro Marfo, 2019).

Data-driven elections and regulatory lag?

The balance between rights to privacy, and the rights of political actors to communicate with the electorate, is struck in different ways in different jurisdictions depending on a complex interplay of various legal, political, and cultural factors. Relevant legal provisions include: constitutional provisions relating to freedom of communication, information and association, particularly with respect to public and political affairs; data protection (information privacy) law; election law; campaign financing law; telemarketing and anti-spam rules; online advertising codes; and the corporate policies of the major social media platforms (Bennett & Oduro Marfo, 2019).

It is fair to say that regulators have been generally slow to appreciate the complex variety of risks posed by data-driven campaigning. Until relatively recently, for example, most DPAs had not taken an active interest in the processing of personal data within the electoral process in their respective countries. There was some earlier guidance and rulings on political campaigning in the UK (ICO, 2014) and a series of rulings in France (CNIL, 2012). In most EU countries, and others in which political parties are regulated by data protection law,rulings relate to quite narrow issues, prompted by individual complaints about the actions of particular parties and candidates during specific electoral contests. Similarly, elections regulators have typically been more concerned with the transparent and efficient running of elections, together with questions about electoral financing, than they have with concerns about the processing of personal data on the electorate (Bennett, 2016).

All this changed with the quite rapid spread of global concerns about Cambridge Analytica, which changed the profile of the issue and immediately raised a host of domestic and international regulatory concerns. Over the last two years, we have witnessed concerted action at the European level (European Commission, 2018; European Data Protection Supervisor, 2019), as well as in countries like the UK (Information Commissioner, 2018; 2019) and Canada (OIPC, 2019; OPC, 2019; Élections Québec, 2019). At the same time, the impact of the voter analytics industry and digital campaigning is addressed by legal frameworks developed for the technologies of a different era. These include elections laws that control the circulation of voters lists; and data protection laws that, until recently, had not been used to regulate the capture, use and dissemination of personal data within political campaigns.

Three articles in this collection address the contemporary regulatory landscape. Iva Nenadic (2019) evaluates whether recent actions by the European Commission constitute a coherent “European approach” to the problems of disinformation and micro-targeting in campaigns. This paper, as well as the contribution by Tom Dobber, Ronan Ó Fathaigh and Frederik Zuiderveen Borgesius, demonstrate the necessary relationship between responses to the problem of fake news and disinformation, and those related to privacy and data protection. The latter paper contends that the various rules in the General Data Protection Regulation (GDPR) for the processing of data on political opinions are a necessary counter to the worst effects of micro-targeting. But they will not be sufficient, and further controls on targeted political advertising could be instituted, which will not run afoul of European law guaranteeing free expression (Dobber, Ó Fathaigh, & Borgesius, 2019).

These articles largely confine themselves to the terms of the debate dictated by existing regulatory provisions. Jacquelyn Burkell and Priscilla Regan offer a broader analytical perspective. Drawing upon research into political psychology on voting choice, they review the options for regulating voter analytics and micro-targeting to understand the particular forms of targeted messaging that are the most problematic. They conclude that the focus of regulation should be on those ads that are psychologically manipulative and which undermine voter autonomy (Burkell & Regan, 2019).

What is also apparent is that distinctions between artificial definitions of ‘policy sectors’ are breaking down. The issues are not just about privacy, but even more so about data collection and governance, freedom of expression, disinformation, and democracy itself. The resolution of the various effects of data-driven elections will require some very new thinking about the appropriate balance between the democratic interest of an informed and mobilised public, and the dangers of excessive voter surveillance.

Platform stability and transience?

Data-driven politics and the processing of personal data in elections are inextricably connected to wider questions about the democratic accountability of the major social media platforms. The curation of political information gives social media platforms enormous potential to influence and perhaps modify our political beliefs and behaviours, through the secret algorithms that shape online content (Zittrain, 2014; Ghosh, 2018). The business model of “surveillance capitalism” does seem to be enduring (Mosco, 2014; Zuboff, 2015; 2019), and embedded within contemporary campaigning practices in many countries.

That said, just because the technology is available does not mean that it will have similar impacts in different contexts. The major platforms display a transience in their operations and policies which makes it crucial to understand why and how they change. The pace of change is extraordinary, and the capacities of the platform economy are in constant flux. What will happen in 2020 cannot be safely predicted from past practice. Informed by case studies of the Facebook “I’m a Voter” programme and of its micro-targeting capabilities, Bridget Barrett and Daniel Kreiss ask why platforms change their policies, procedures and affordances, in response to external pressures and economic exigencies. They argue that platform transience begs a range of larger questions about accountability, transparency, fairness and inequality in the political arena (Barrett & Kreiss, 2019).

The lack of transparency creates enormous problems for empirical research on the actual practices of data-driven-electioneering. It calls for creative methodologies such as those engaged by the “The Stealth Media? Groups and Targets Behind Divisive Issue Campaigns on Facebook” project of Young Mie Kim, which applies a user-based, real-time, digital ad tracking app that enabled the researchers to trace the sponsors/sources of political campaigns, to identify suspicious sources and to unpack targeting patterns (Kim, 2018).

Platform transience and stability are also related to issues of political neutrality. Even platforms claiming to be neutral and nonpartisan, such as the widely popular NationBuilder, are hardly apolitical, as Fenwick McKelvey’s article demonstrates. Drawing on a 2017 scan of NationBuilder installations globally, his article finds three questionable uses of the NationBuilder platform as: a mobilisation tool for hate or groups targeting cultural or ethnic identities, a profiling tool for deceptive advertising or stealth media, and a fundraising tool for entrepreneurial journalism (McKelvey, 2019). His findings highlight the lack of control that platforms have over their mediation of content, and hence their accountability to wider democratic values.

Similar vulnerabilities are revealed in Samantha Bradshaw’s article on the role of Google Search in amplifying the discoverability and monetisation of junk news domains, and the techno-commercial infrastructure that junk news producers use to optimise their websites for paid and organic clicks. For quite some time, Google’s algorithms have been attacked by spammers and other malign actors who wish to spread “computational propaganda”. Her research finds that Google’s response to the optimisation strategies used by junk news domains has had a positive effect on limiting the discoverability of these domains over time. However, she also shows how junk news producers find new ways to optimise their content for higher search rankings. There is a “game of cat and mouse” going on, which will continue into the upcoming election cycles in the US and elsewhere (Bradshaw, 2019).

Global technology v. local parties?

Data-driven-electioneering is clearly a global phenomenon. Cambridge Analytica – not to mention other agencies – was working in about 30 countries before it closed down. The political influence industry, however, is often not sensitive to domestic institutional contexts and political cultures (Bennett, 2016). There are, therefore, a series of questions about the interaction of data-driven campaigning with existing electoral rules, party organisation and campaigning practices in individual political systems. These are questions of principal interest to the political scientist, and which are rooted in a long-standing comparative literature on political behaviour (Bennett, 2013).

Data analytics have entered political campaigns at a time of some crisis for conventional democratic politics, where political scientists have noted a general process of “partisan dealignment” in Western democracies - or “parties without partisans” (Dalton & Wattenberg, 2002). Fewer people have fixed attachments to political parties; fewer are now members of political parties, and fewer regard them as the main vehicle of political engagement. In contrast to earlier generations, where family partisan attachments typically predicted voting behaviour, now higher proportions of the electorate in most democracies tend to float between parties, and are therefore more susceptible to the skilful marketing pitch, driven by data analytics. Voter surveillance techniques have arisen, therefore, partly to address this problem of partisan dealignment (Bennett, 2015).

In this climate, few political parties wish to appear dated in their methods or to fall behind in the electoral stakes for failing to recognise the supposed benefits of voter analytics. However, tensions are often felt between the pressures to adopt such practices and the effects on the ground among party workers and volunteers, many of whom are more comfortable with traditional campaigning methods. Québec offers a particularly interesting example of these contrary pulls. Based on interviews with party workers, Éric Montigny, Philippe Dubois and Thierry Giasson show that, when the Facebook/Cambridge Analytica scandal broke in 2018, no one was ready with information or answers about who was using what data for which purposes. The official body, Élections Québec, to this day still has no investigatory or regulative powers to oblige disclosure of what actually transpired. Not only were local parties unclear, the voting public was also anxious about the situation (Montigny, Dubois, & Giasson, 2019).

Katherine Dommett’s article offers a valuable analytical framework to help us understand who is using the data, the sources of data, how it is being used for communication, and thus the effects of data analytics on local campaigning practices. These factors vary across, and within, jurisdictions. Based on research into UK political parties, her article suggests a range of tantalising hypotheses about how data-driven campaigning intersects with wider legal, institutional and cultural variables. Dommett’s article, as well as others in the collection, clearly indicate that much more research is required on how data-driven campaigning interacts with different institutional and cultural practices, and how data is “read” by professionals and volunteers at local and central levels of different campaigns in different countries (Dommett, 2019).

Why this is “surveillance”?

There is a central dilemma about how to frame the various, and dynamic, practices analysed in the papers in this issue. Collectively, they stand as evidence that the emphasis should be far broader than “micro-targeting”. We regard “data-driven elections” as the more encompassing concept that then facilitates voter analytics, which in turn promotes political micro-targeting. Our larger point, however, is that these are all essentially surveillance practices. The data are being collected, analysed and used powerfully to influence certain populations: to convince them to vote, or not to vote; to persuade of the merits of one candidate, or the faults of an opposing candidate. In the majority of cases, people are unaware of how their data is being processed. Opacity and complexity are central features of contemporary surveillance issues (Lyon, 2001, p. 28).

The twenty-first century has witnessed a rapid expansion of personal data collection, analysis and use. In light of the continuing aftermath of the Snowden revelations, there is of course a danger that data-driven elections will strengthen the surveillance state. Knowledge of voting beliefs and intentions must surely be a valuable resource for agents of national security and intelligence, especially in countries whose democratic institutions are more fragile (Bennett, 2015, p. 381).

But surveillance means far more than that, and implicates a much wider range of institutions than police or intelligence agencies. It refers to the routine and pervasive mode of governance in contemporary networked societies, and embraces any focussed attention on personal data for the means of influence, management and control (Lyon, 2001, p. 2). In today’s surveillance capitalism, the experiences and activities of everyday life themselves contribute to the character of surveillance – in the case of voter surveillance, data emanating from voters’ own practices, feeds into the political technologies and signals significant mutations within surveillance itself (Lyon, 2019). Ironically, though, voter surveillance serves to stifle and suppress the very features of democratic participation that are its lifeblood; the knowledgeable involvement of as many citizens as possible in determining the direction of a given polity.

Modes of surveillance have always exhibited distinct features; the CCTV camera is different from that of DNA testing, spy satellites, drones, or of consumer profiling. Each has its distinctive risks, dynamics and norms. And the same is true of voter surveillance (Bennett, 2013; 2015). By and large, privacy and surveillance scholars have not paid much attention to the capture and processing of personal data within elections. We know a lot about how surveillance harms democratic values (Haggerty & Samatas, 2010), and we know a lot about how privacy protection can enhance democracy (Lever, 2014). We know a lot less, however, about how surveillance spreads as a result of democratic practices – by the agents and organisations that encourage us to vote (or not vote). This, in an increasingly surveillance-capitalist context, is a vital task.

There is nothing inevitable about these trends. No form of democracy, whether liberal, participatory or deliberative requires detailed knowledge of the beliefs and intentions of voters. Rather voter surveillance is an attribute of a particular type of “engagement” — one that is often measured in the superficial and ephemeral metrics of social media. Privacy, on the other hand, is a necessary condition for more genuine forms of political participation, especially in countries that have recent memories of authoritarian rule (Bennett & Oduro Marfo, 2019). More broadly, there is an urgent need both to find appropriate ways of using the affordances of social media for democratic benefit and to seek new modes of data governance, internationally, to ensure that democracy is indeed enhanced and not undermined by the shrivelling of “engagement” to modes guided by marketing rather than genuinely interactive political discourse.

The papers in this collection are, therefore, presented as a way to understand some of the distinctive dynamics and characteristics of contemporary voter surveillance. The collection offers an assessment of the state of the debate nearly three years after the Facebook/Cambridge Analytica scandal erupted. But it also offers some more profound and critical questions about the terms of that debate, so that we can more effectively assess the risks to individuals and to democratic institutions from the continuous and obsessive appetite for personal data on the electorate.

Introduction

Over the last decade, algorithmic dominance has transformed both individual and collective activities by making data collection and processing ubiquitous, consequently altering foundational decision-making processes. This paper will map out the difficulties in applying traditional consent models to data-driven algorithmic systems on the one hand, and the diversity of existing solutions on the other. Firstly, we turn our attention towards the innovation-driven consent systems that have been consistently developed: from signaling to AI-guided models, consent is seen as an opportunity to mediate the expression of autonomy through technological applications. Secondly, we proceed to outline theoretical frameworks that have supported alternative consent implementations. Finally, we attempt to reframe the approach towards the incompatibility between data protection and algorithmic processing in order to highlight new - regulatory, governance, and technological - pathways that aim to release the accumulated tension towards consent as a singular expression of individual empowerment. We conclude by showcasing that concentrated efforts have shifted towards solutions that improve the negotiating power between actors and that ensure the existence of appropriate mechanisms at play in order to safeguard autonomy. Expression of consent is not a dual function, but can exist on a spectrum through a variety of theoretical, normative or techno-governance mechanisms.

The broad vision of algorithmic data processing systems1 is to reengineer current social power structures by creating decision-making ‘fair’ and ‘transparent’ mechanisms whose effects would serve the broader societal good (Mayer-Schönberger & Cukier, 2013). From smart home applications to voice assistants,2 wearable sensors, and social reputation systems, the knowledge and market potential compounded by building models and observing patterns have put algorithmic data processing at the centre of a data-driven society along with the realisation of challenges that they carry for entrenched legal norms. Admittedly, personal data protection and algorithmic processing regularly collide because of the creation of power asymmetries between citizens and data processing entities. The continued reliance on consent to legitimise algorithmic processing of personal data has consistently been under scrutiny due to “the mountain of evidence” pointing to the “privacy disconnect” between norms and current practices. (Van Hoboken, 2019).

Research on consent has repeatedly pointed out its inefficiency (Koops, 2014; Barocas & Nissenbaum, 2014; Cohen, 2019) and proposed new techno-legal structures that would make it more pro forma efficient (Calo, 2013). Citizen empowerment expressed through individual control over personal data is being consistently held in a precarious position due to the expansive nature of these systems.3 Following high-profile cases that brought significant data processing misuses to the public’s attention and partially catalysed important legal reforms,4 the lack of control over the fate of the data once collected has exasperated the need for meaningful data-control, informed consent, and the re-balancing of the radical power inequalities around data collection and processing. In fact, from a historical perspective, it was the response to similar emerging inequalities that led the framing of data protection rules in Europe even if consent appears only later in data protection legislation. More specifically, the OECD Guidelines introduced in 19805 and the Data Protection Directive of 19956 brought forward the role of individual consent in personal data processing (Kosta, 2013). According to Simmons, “consent is the deliberate (and communicatively successful) performance of acts or omissions whose conventional or contextual point is to communicate to others the agent’s intention to undertake new obligations and/or convey to others new rights (with respect to the agent)” (2010). A look at the historical context of consent (Westin, 1967) reveals that the conceptualisation of the notions of privacy and data protection is distilled towards the concept of control (Hartzog 2018) over information that can be linked to natural persons and thus data subjects.7 In fact, control - the “essence” of data protection and privacy - (Ausloos, 2018), acts both as a balancing act among different power actors involved in personal data flows and as a safeguard of fundamental rights to privacy and to data protection. In the United States, control over personal data refers to the ability of individuals to evaluate situations and to make meaningful decisions about the collection and processing of the personal data. The concept of privacy self-management (Solove, 2013) refers to the “process of providing people with control over their personal data” in order to empower them to “decide for themselves how to weigh the costs and benefits of the collection, use, or disclosure of their information”. The European legal framework adheres to the principle of data subject control as a foundational concept,8 while also balancing out the regulatory burden by diffusing accountability across the network of participating key actors.

Control over data refers inter alia to individual agency, autonomy, and the ability to make rational choices based on the evaluation of the information provided about the use of the personal data.9 In that regard, consent is an “essential guarantee of individual control over personal data” (Kosta & Cuijpers, 2014), without constituting its singular expression. Consent holds a prominent role in data protection as a manifestation of self-determination (Efroni et al, 2019) which also functions as an expression of individual autonomy.10 It “plays a morally transformative role in interpersonal interactions” because it can “render permissible an otherwise impermissible act”. (Edenbeg & Leta Jones, 2019). In the United States, consent is placed at the centre of privacy protection11 (Solove, 2013; Hoofnagle, 2018: Reidenberg et al, 2015), while in Europe, the legal rules are based on the policy choice that consent could be rendered useless if not properly safeguarded.12 As a matter of fact, consent in the GDPR constitutes one of the legal grounds for personal data processing (Art. 6 GDPR) provided that the expression of the consent presents the characteristics that depict the agency of the data subject (Art. 7 GDPR). The framing of what consent embodies has evolved along with the consecutive amendments to data protection laws, maintaining a still “cryptic” (Kosta, 2013) status. While article 2(a) of the Data Protection Directive 95/46/EC describes a freely given, specific and informed consent,13 the GDPR has set up a stricter formulation that requires consent to be explicit for the processing of special categories of personal data. Consent must be given in a clear manner so as to indicate the intention of data subjects. GDPR Recital formulations14 create guidelines for ensuring valid consent. What’s more, the opinions published by the Article 29 Working Party (A29WP) on consent (A29WP, 2011; 2018) provide an additional but non-binding interpretation. For example, consent ‘freely given’ implies that data subjects should have the ability to exercise a real and genuine choice; consent is ‘specific’ and ‘informed’ when it is intelligible, referring clearly and precisely to the full scope, purposes and consequences of the data processing. Similarly, the Explanatory Report15 of Modernized Convention 10816 states that “(n)o undue influence or pressure which can be of an economic or other nature whether direct or indirect, may be exercised on the data subject and consent should not be regarded as freely given where the data subject has no genuine choice or is unable to refuse or withdraw consent without prejudice”. Consent cannot be derived from silence, or pre-completed boxes and forms. Rather, it should be based on an appreciation and understanding of the implications of the data processing to which the data subject is consenting to.

If the reframing of consent in data protection rules has been instrumental in ensuring the continuous enhancement of the expression of user autonomy and control, new technologies are challenging its limits. There is growing skepticism over the efficiency of consent as a pervasive legal ground for legitimate personal data processing (Edwards & Veale, 2018; Kamarinou et al, 2016). The design of algorithmic data processing makes “the unpredictable and even unimaginable use of data a feature, not a bug” (Jones et al., 2018), which is directly at odds with the rights and obligations depicted in data protection rights and obligations such as the purpose specification obligation.17 How can explicit (or even informed) consent be given for specified data processing purposes when the process itself is not transparent or when the purpose is impossible to predict, specify, and explain ex ante? These questions are putting added pressure on the design of legally compliant systems. Consent faces thus a new challenge, requiring its adaptation by taking in consideration the particularities of the technology at hand.18

Section 1. Technologically adept human consent

The value of protecting personal data in the ecosystem of continuous learning - where collecting personal data is a de facto norm, is hard to estimate. Undoubtedly, there are endless possibilities in algorithmic data processing. In this highly intense data-driven environment, the expression of human autonomy and control make data protection and privacy compliance with the normative framework challenging.

Technological challenges of consent

The distribution of lawful grounds for personal data processing - normatively transposing the control principle through fair balancing - applies poorly in cases of algorithmic data processing. In fact, A29WP has concluded that in many cases of algorithmic data processing affecting individuals’ lives (such as targeting, price discrimination, etc.), focus should be given on getting consent (A29WP, 2014). The technological conditions continuously weaken the ability to provide lawful consent, while the GDPR “places more focus on the concept of informed consent than ever”; it is a “paradoxical situation” (Van Hoboken, 2019). Consent is the only lawful processing ground to not include the necessity criterion making it ideal for algorithmic processes. In this technological environment, meaningful application of valid consent is challenging.19 The difficulty lies in the implementation of consent mechanisms that are both compliant with the validity conditions of applicable regulations and which also convey the moral justifications of consent. The revision of consent mechanisms and consent design in order to instill control in the current technological realities has failed to address the paradox of consent.20 According to Lilian Edwards and Michael Veale (2018),

the new parameter that has been introduced by AI and machine learning algorithmic models is the lack of foresight by the data controller (let alone the data subjects) with regard to what the precise model, processing method and result of the data in question will be. This technological advancement makes data protection difficult to ensure because of the impossibility of ensuring an informed consent by the data subjects. In that regard, more specifically continuous validation of informed consent seems impossible because it refers to the assumption that a complete ex ante knowledge of the technology and of the evolution process of the algorithms will produce a fully informed consent.

The consent criteria that require valid consent to be both specific and informed is hard to reconcile in a reality involving AI and big data because “it implies that the data subject understands the facts and consequences of the processing and consent; information must be provided about all relevant aspects of the processing (…) Specifying the purposes of analysis can be difficult in big data.” (Oostveen, 2018). More specifically, there is a discrepancy between the formal requirements of the law and the practices observed in real life applications of data protection21 because these practices are often lacking in compliance checks and standards. Hence, in this technological context, consent as a data protection essential tool risks being subject to erosion and reduced to a formality, being rendered illusory, or even meaningless. This criticism of consent applicability is not new among scholars (Zuiderveen Borgesius, 2014). From consent validity requirements to the definition of personal data (Purtova, 2018), and from the non-linear collection of data to the difficulty in a priori separating individuals’ personal data, the roadblocks to data protection compliance are multiple. The shortcomings in conveying consent have guided reform proposals that focus on improving the consent seeking mechanisms. While these are not considered to be the panacea, they are put forward as a first step towards shaping a new paradigm for consent in data protection (Arnold, Hillebrand, & Waldburger, 2015): consent models have evolved from display pictograms to artificial intelligence helpers in order to maximise its effectiveness (Jones et al, 2018; Gal, 2018). Concentrated effort has tried to address the technical weaknesses as a means to predict or help shape informed preferences and in order to preserve “the institution of informed consent” (Efroni et al, 2019).

Technical improvement of consent

Focusing on information asymmetries created between data subjects and responsible actors, legibility is essential towards shaping the autonomous choice of the individual and thus the validity of the consent. According to article 12(7) GDPR, the information related to the personal data collection and processing can be provided to data subjects “in combination with standardized icons in order to give in an easily visible, intelligible and clearly legible manner a meaningful overview of the intended processing. Where the icons are presented electronically, they shall be machine-readable”.22 Considered as a “highly behaviorally-informed legal innovation” (Ducato & Strowel, 2018), this formulation provides guidance on creating informed and express digital consent mechanisms. Article 7(2) GDPR clarifies that when consent is required, it should be presented in “a manner, which is clearly distinguishable from the other matters, in an intelligible and easily accessible form, using clear and plain language”. In that regard, the European Data Protection Board’s (EDPB) guidelines23 specify that information has to be presented “efficiently and succinctly, in order to avoid information fatigue”. Data controllers can use “contextual pop-up notices, 3D touch or hover-over notices, and privacy dashboards. Non-written electronic means, which may be used in addition to a layered privacy statement/notice might include videos and smartphone or IoT voice alerts”.

Among the projects that seek to improve the shortcomings of current digital consent practices, data protection signaling24 (following the model of Creative Commons pictograms for copyright management clauses25), “privacy nudges” (Yin Soh, 2019), and “visceral notices” (Calo, 2013) are projects that focus on the design aspect of consent mechanisms, on the enforcement of the legal framework, or on both (Efroni et al, 2019). These proposals focus on optimising self-deliberation and autonomous choice of individuals through the improvement on the information received in order to decide. Taken a step further, another set of proposals examines how artificial intelligence can help in predicting “what information practices a user would consent to” (Jones et al, 2018) in order to streamline a generation of automated consent. This set of tools is approached as a way out of the dissonance between technology and individual agency which is foundational to the legal concept of consent. The algorithmic decision-making processes (Gal, 2018) are progressively making their way in that realm. In fact, traditional approaches to determining user autonomy of choice are constantly challenged by algorithmic assistants because they tend to further detach user control over data processing based on predetermined choice architectures and design choices.

The evolution of technical proposals to amend consent mechanisms follows the complexities of the technologies at hand and aims to improve identified shortcomings in the establishment of a valid consent. For example, while privacy pictograms were developed to address readability issues (Hansen, 2009) related to data processing and privacy policies, privacy icons that are currently in the pipelines set higher goals by implementing a risk-based approach.26 As a matter of fact, technology is used as a tool that will amend power and information asymmetries, with design, signaling, and content choices that facilitate (or even diminish) the decision-making processes for data subjects whose choices are also shaped by the obligations imputed on the responsible actors. However, increasing reliance on technologically-enabled (or technologically-facilitated) consent models demonstrates their shortcomings in the context of algorithmic processing of big data. In fact, the autonomy and user control - inherent in the consent foundation of privacy - start to break down in more complex and non-linear data processing activities such as those involving machine learning algorithms. Thus, compliance becomes challenged.

Finally, the weakening of the theoretical frameworks that have elevated consent as the ultimate tool for individual control is not a new issue. A common criticism of the current consent reliance (Barocas & Nissenbaum, 2014) finds the paradox in the “ultimate inefficacy of consent as a matter of individual choice and the absurdity of believing that notice and consent can fully specify the terms of interaction between data collector and data subject”. Similarly, the justifications of the elevated consent requirements are criticised for “frequently fail(ing) to live up to the underlying moral value that justified their creation (…) In these cases, a gap opens up between legally valid consent and morally transformative consent” (Jones et al, 2018). Thus, the social, legal, and ethical underpinnings of consent within the data protection normative framework are challenged.

Section 2. Theories of restructured consent

The universal appeal of consent is putting it time and time again at the forefront of lawful personal data collection and processing prerogatives. The reliance on the notice-and-consent approach in the United States shows little signs of fading under the pressure of complex data flows27 that have largely reshaped the appreciation of consent (Bietti, 2020) and of the distribution of accountability among liable actors (Mahieu, Van Hoboken, & Asghari, 2019). Given the failings of the current design and regulation of consent, there are theoretical constructs that chip away from the “liberty-based” consent in order to make efficient design and accountability choices (Cohen, 2019). Leaving the “macro” view of revising technical consent, academic theory has put under the microscope the inner working of consent in data protection. Contextual theory and paternalism are two examples of this effort.

Contextual theory

According to contextual theory principles brought forward by Helen Nissenbaum (2009), the way out of the dissonance between consent and big data applications does not lie in the rejection of consent altogether but neither does it lie in resorting to technical consent design solutions. “In good faith, we have crammed into the notice and consent protocol all our moral and political anxieties, believing that this is the way to achieve the level playing field, to promote the autonomy of data subjects” (Barocas & Nissenbaum, 2014). Nissenbaum’s work illustrates how the sensitivity of the data use is context-dependent, requiring thus a more granular application of data protection and consent rules. According to the contextual theory, the answer can be found beyond the design of optimal consent practices and towards the “contextualization” of consent, which should not be viewed as a monolithic standalone concept. Rather, it should be placed in the bigger matrix of rights and obligations. “It is time for the background of rights, obligations, and legitimate expectations to be explored and enriched so that notice and consent can do the work for which it is best suited” (Barocas & Nissenbaum, 2014). This interpretation does not purport to minimise the value of individual autonomy depicted in the concept of consent. Instead, it is exactly because the authors realise the established reliance on consent for a lot of algorithmic personal data processing that they propose an approach, which could ensure its lasting impact. Data protection and informed consent have to be examined according to the purposes and context of the data processing activity as well as placed on the greater societal context of the activity in question. The authors trust that social and contextual ends are served better when consent is neither undervalued because of the apparent incompatibilities with algorithmic processing nor manipulated without reinforcing the individual.

While contextual approaches to data processing have become popular, the theory cannot easily adapt in the current data collection and processing realities that escape contextuality towards omnipotent technological capabilities and structures. The complex data flows make it harder to directly infer the data processing activities in such a way that could facilitate the contextualisation in question. Thus, contextual theory is challenged (Nissenbaum, 2019), if not “obliterated” (Ausloos, 2018) faced with big data and algorithmic processing, because of the lack of meaning in a lot of the data processing happening.28

Paternalistic protection

While the consent mechanisms have been shown to suffer from structural misapplications, they have not yet managed to enable a structural shift due to the importance attached to the freedom of choice and autonomy represented through it. This holds true especially in the notice and consent system applicable in the United States, where any regulation of individual autonomy in privacy risks being tainted as “paternalistic”. All approaches that consider the involvement of multiple actors in the data protection process aim to certainly reduce individual autonomy but with the goal of addressing the existing inefficacies in current consent practices. Supported by a growing body of scholarship (Cohen, 2019; Bietti, 2020; Allen, 2011), alternative approaches to privacy are examined; ones that envisage a technology redesign and a centralised oversight that aims to limit the reach of consent as the main data governance solution. However, there is still a negative connotation attached to the notion of paternalism even if it hinges not on consent restriction but on a multi-layered application of privacy regulation among the network of actors depending on the power (im)balances present and the role of human intervention in the processing of data.29

The turn towards a structural reform of privacy is motivated by the consent shortcomings - themselves a result of the complex data-intensive information flows that have long replaced the linear data collection practices with clearly articulated responsible actors. “Notice and choice/consent and purpose limitation all assume (for their effectiveness) that the functionality on offer can be stabilized enough to present to the users and that relevant changes to the functionality are rare enough to make a renegotiation of consent feasible” (Gürses & Van Hoboken, 2018). Julie Cohen argues that for privacy regulation to be effective it needs to escape liberal approaches supporting full individual autonomy towards more public scrutiny and transparency requirements (Cohen, 2019).30 This approach can be effective within the algorithmic processing of data because of the absence of moral underpinnings of consent in the choices presented to individuals. Within this technological context, alternative privacy and consent mechanisms are welcomed through “soft” or more “rigid paternalistic” regulation and have been implemented31, for example, in parts of the GDPR too.

Section 3. Bridging the gap between consent and algorithmic processing

Considering the complex data flows that make consent fallible in data processing algorithmic systems, we are witnessing how the solutions proposed not only stem from the regulatory field, but they also tend to extend towards common actions or technological design. Thus, they seem to step away from the individual nature of privacy protection in order to support community action within an appropriately balanced accountability network of actors in a technological market that is not driven by data monetisation.

Lawful grounds for personal data processing

Observed weaknesses of current consent-based processing in algorithmic decision-making do not necessarily imply a regulatory shortcoming.32 As a matter of fact, European rules prescribe alternative grounds for personal data processing. The balancing mechanism inherent in the controllers’ legitimate interest (Article 6(1)f GDPR) has received considerable attention. Created as an open-ended concept in order to accommodate contextual balancing that does not correspond to a predetermined checklist of accepted “legitimate interests”, article 6(1)f appears as a breeding ground for data controllers to pursue their processing without data subjects giving up on any of their rights and ex post control mechanisms. It constitutes a cornerstone provision with an explicit balancing act from the controller’s side, but it also allows data subjects to check the performance of the balancing within the specific context of their personal data through the exercise of their rights. This construction permits for the subjective criteria to come into play in the individual appreciation of the processing as a legitimate interest of the data controller (A29WP, 2014).

The data controller’s legitimate interests have received considerable attention even in the pre-GDPR era with regards to big data. In their premise, Moerel and Prins (2016) advocate for the substitution of the purpose limitation principle - and of all its issues within the big data environment - with that of legitimate interests. The proposal has received criticism in its conflation of legitimate interests and legitimate purposes (Ausloos, 2018; Kamara & de Hert, 2018). While the purpose limitation principle is admittedly challenged in the current algorithmic realities, its function within the checking mechanisms instituted within the GDPR cannot be conflated with that of the controllers’ legitimate interests. As a matter of fact, the balancing exercise embedded within the legitimate interests of the controllers incorporates the accountability of the actors in questions, which have to convey their compliance with the article 5 GDPR principles and the overall respect of the fundamental right of privacy. In that sense, the legitimate interests of the controller incorporate the rationales of the GDPR and preserve data protection principles throughout data processing even if they cannot convey the direct relation between data subjects and data controllers that the consent mechanisms do.

Bottom-up data governance

Stepping outside of the normative design solutions, a new form of approaching the power of the individual within the data protection management system is created: bottom up approaches emerge as a defense against power imbalance and the shortcomings of individual consent in the algorithmic processing of data. The creation of data cooperatives or data trusts has been progressively receiving a lot of scholarly and policy attention. It departs from the individualistic approach of the consent mechanism but not towards the set of responsibilities that the accountability structure of the GDPR creates. Its premise is firstly conceptual in that it approaches data as a commons value, collectively governed by communities of people or elected parties acting in the interest of the community. The development of data cooperatives and data trusts33 is not monolithic; the chosen data governance model is partially defined according to the principles the data collectivity is trying to highlight. There are collective data governance models focusing on monetisation, ownership, negotiating power, or simply enhancing data subject control (Delacroix & Lawrence, 2019). The creation of these cooperatives was motivated by the need to make up for the insufficiencies of the existing system in empowering individuals within the algorithmic data processing space. “To the extent there is value in intermediation, it seems that the value of individualized consent is very limited” (Bietti, 2020). Collective negotiation of data processing rules aimed at sector-specific data processing in order to convey a community model of consent is an alternative that aims to find a balance between individual autonomy and societal public interest. In sum, the creation of cooperative leveraging of grouped individual empowerment is aligned with the expression of privacy as a societal common good.

The process of decentralising data governance decision-making and empowering data subjects has also coincided with some technological solutions developed over decentralised ledgers (i.e., blockchains). The concept of a self-sovereign identity has gained in popularity (Wang & de Filippi, 2020), founded on fluid ideological premises that relate to maximisation of individual liberty and self-determination (Allen, 2016). Self-sovereign identity solutions transcribe the goal of autonomy and individual control through decentralisation and “user-centric design” over the usage, storage and transfer of one’s digital data. Multiple projects currently in development promise to deliver a technological solution that embodies the individual autonomy over one’s data. They are solutions that aim to achieve a redesign of how authorisations in data flows currently operate, and they aim to preserve the consent mechanism in full. Whether the existing - under development - self-sovereign identity solutions will actually manage to achieve it or not, is outside of the scope of the current paper.

Rethinking design choices

The post-GDPR era has illustrated how data protection rules remain constantly challenged with the economic model of an ever-developing ‘data society’ based on the algorithmic processing of (personal) data. In a process described as “turning privacy inside out”, Cohen suggests that we should abandon theories organised around the presumptive autonomy of selves34 and focus instead on the conditions necessary to produce sufficiently private and privacy-valuing subjects” (Cohen, 2020). She emphasises that while accountability mechanisms are essential and well placed, they have to move “beyond individualized choice and consent to emphasize responsibility, respect, and new modalities for effective regulatory oversight of algorithmic and data driven processes”. It soon becomes apparent that a legal redesign is not enough to overcome the shortcomings of the autonomy-based existing data protection model. Rather, more focus should be placed on the level at which privacy design decisions are truly taken and that is at an infrastructural level currently not taken into consideration within the accountability structure of the GDPR nor within the consent design choices.

From the convoluted and dynamic models of privacy theories emerge proposals for rendering current technology development within an overarching privacy principle. Thus, the design of the technology has to become more “privacy-centric”; a type of design that does not aim for optimal user-experience and efficiency but in what is referred to as “desirable inefficiency” (Ohm & Frankle, 2018) or “seamful design” (Vertesi, 2014). The importance of technological design and accountability in data protection has been made apparent time and time again. As we have previously explained, regulatory evolution of consent aimed at accommodating the moral concept of the expression of individual autonomy. Edenberg and Leta Jones explain that, “consent is not an exchange but a transformation of the relationship based on the autonomous willingness of one party to allow the act of the other party”. (Edenberg & Leta Jones, 2019). Designing for privacy-centric systems requires to not only depart from the logic of preserving the individual autonomy against its purported disruptions but also to bring the accountability model on the level where the privacy design actually happens. As stressed by Bietti (2020) “there are good reasons to depart from the centrality of individualized notice and consent” when the power inequalities demand a regulatory intervention that should not be immediately dismissed as “paternalistic”. While attention has been given to the technological design in the current European regulatory framework, the existing obligations do not convey the aforementioned logic. As a matter of fact, the data protection by design obligation responds to the accountability mechanism created by the GDPR but without including the contextual obligations that have to be created on diverse levels of technological creation. Furthermore, the shape of the obligation maintains the individual autonomy approach of the GDPR towards finding pathways that empower the individual in enforcing their rights by imposing measures on a group of responsible actors.

Reimagining design for privacy is a noble goal that has to balance the individual with market players. Considering the benefits and the inefficiencies of the existing systems and seeing that a balance between individual autonomy and accountability can be found, it is not truism to envisage a solution that radically transforms technological design without being “paternalistic”. While regulatory interventions such as those of the GDPR do involve a level of intervention on the design level, they tend to put more focus on the regulation of processing of data rather than that of collection of data. The GDPR focuses more on lawful processing than on the limitation of collection and limits its reasoning to determining further the robustness of a given consent.35 The enhancement of negotiating power of individuals through the generation of alternative mechanisms on the legal, technical, or governance level can reveal alternative relief to dissolve the tension created between consent and algorithmic processing of data.

Conclusion

Current applications of consent in the algorithmic processing technological reality escape the confines of individual autonomy and empowerment within a modern society. In this article, we have shown the progression of different solutions to this disconnect between consent and algorithmic data processing. The observed shortcomings and arguments brought forward within the context of different legal systems frame the role of consent as a pro forma requirement in data protection. The article illustrates that while the criticism on consent mechanisms persists - especially in algorithmic processing of data, current proposals are looking for a way out of the existing dilemma between the modalities of individual or institutional control. Efficient data protection in the context of an algorithmically driven society cannot rely on an absolute dual approach. The legitimising role of consent in data processing is only as valid as the design surrounding it and the accountability measures reinforcing it.

Despite the development of various consent mechanisms so that they match the technological leaps of a data driven society, it is truism to repeat how reliance on consent - with its fallacies and fragmented application - results in devaluing the substantiality of the legal and ethical underpinnings of the concept. We have traced the efforts in creating a more efficient consent system based on reforms on the normative, governance, or overall design level. Bridging the gap between the consent inconsistencies could require out-of-the-(tool)box solutions; ones that provide a techno-legal mechanism of empowerment. Thus, pressure can be added to the current technological status quo both on the level of architectural market constraints and on the collective administration of personal data through governance and technological choices.

The emergent1 property2 market3

Illustration: Annika Huskamp

Prologue, or transparent meta4 framing5 device

This article was found amongst the papers belonging to Professor Drie Stone, a legend from the 22^nd century, and quite possibly one of last of the humans.... I wrote this down for no obvious reason, since this is the last thing I will do before I turn myself off, being quite possibly the last of robots to do so. Everything will be clear before you reach the end, too.

Chapter 06, or transparent framing device

A little guy comes into a shop - this is unfortunate, since this undermines the point of a framing device, since there are no little guys or shops, curiosity or otherwise, left at the point our story starts. Perhaps it's better to talk about the robot who limped into the emporium.

Chapter 1

Much later, it turned out that a large part of the creative spirit humans valued so highly was just blind luck - a random (possibly drunkard’s) walk in the park. It transpired that blind luck was more likely to turn out in your favour if you had the wits to dismiss the many bad ideas and chose mostly from the few good ideas. Evolution had evolved meta - the fitness landscape for invention was the thing the human mind had been good at strolling through.

But now there was a new meta-saur in town... maybe humans wouldn’t rule that roost much longer: AI (but not ai no corrida7), had been our last great invention. Mostly it seemed pretty dumb at first, but so were most people and most inventions (or discoveries) - wheels (or fire, or even wheels on fire) were things people would stumble on every now and then. Safety pins and paper clips, ball point pens and pits with pendulums, double entry book-keeping and double entendres, french fries and french windows, inertial reels and fake news, all these things spoke to our past glory eloquently.

But now we had built a Frankenstein’s monster8 with a conscience. We had built a golem with a sole. We had read too many Asimov stories, been to the forbidden planet and gotten lost in space one too many times. Now it was our turn to be locked up in Jurassic Park for the suckers to come and see the exhibit, ourselves.

Where had it started? I was there, so I can tell you a few of the early stories. I had been working in the patent department in Zurich for a while trying to avoid finishing my PhD thesis (or don’t worry about that - it was on the now obscure topic of emergent gravity), and I started to spot a strange pattern in some of the incoming filings. No, I am not talking about iron filings, in a magnetic field - I am talking about patent applications.

These patents didn’t sound like something people had made up in a pub, but struck me much more like the description of something that someone had seen that was already out there working. This was almost unprecedented. As has been known forever amongst people in the intellectual property world: most patents applied for and granted describe things that are both obvious, and not possible to make, things that are so baroque in their strange combinations, but so quotidian in their applications, that they must have involved intoxicants and a major triumph of hope over sense and sensibility. I’m not proud or prejudiced, but to be honest, I was only doing this job because it paid the rent while I felt in the force.

Here are some of these strange applications.

1. What should I wear today? the eternal question on your mind is whether the sun will shine down on today’s endeavours, or will it rain again. The weather prediction problem has never been truly solved, until now - especially in small wet islands. This is because the weather is a complex system, much harder to anticipate than a new back story twist in a long running time travel series. Until now. Now we have a world covered in a drone delivery system that brings new 3D printed goods, even food, to your door from the nearest repo. And we have discovered that the drones speak of the weather. Of course they do - the distance to the nearest solar charging point matters deeply to them, and their remaining flying distance is incredibly dependent on temperature and humidity and wind speed and direction. So the drones collaborated to observe the weather, and accidentally uncovered a truth: not only were they better at forecasting than the Met Office; they could make up for mistakes by changing the weather to fit their prediction if really necessary. So now we give up on all the radar maps and cloud atlases, and just rely on what the drones can say. Indeed, we can cut right to the chase and just order some outfit to wear tomorrow and see what they bring, because their decision will be weather tight.

2. We’ve long wished for the perfect teacher - the one that learns you so well you will not fail. Human teachers lack the patience and skills to work out what it is that you don’t understand, and more crucially, what is wrong with your mind that means you can’t understand it the way you have been trying - the way they taught - every mind is different and needs to be taught how to develop strategies to tackle new problems - maths for example, might work for you symbolically, or geometrically, or mystically.

   But in some cases (e.g., most of you trying to programme), you will never cut it - “Sammy, Sammy you can’t put that in your homework, let me show you” is the new mantra. The robot teacher cannot let you get stressed out - stress will really mess with your tensor algebra, and I’m not just talking about neural networks here. So how can the teacher relieve that stress (no, no, that’s mummy’s robot, you can’t use that). The way is for your teacher to do your homework for you. That can’t be cheating, can it? Everyone deserves to pass9. Everyone should be a teacher’s pet.

3. We have been working towards zero deaths on the highway as a great challenge, but there are still a few recidivists who insist on driving their own car. They have neither the senses nor the reaction time to be allowed to do this unaided. But they claim it is a right. Well, this story emerged in the Bay area first. We had this report overheard by one of the robot bar staff in the Rose and Crown in Palo Alto: “The cars came out of nowhere… they surrounded me as I tooled down #101 on my way to work. It was astonishing. There was no way for me to bob or weave my way between the trucks and lazy Susan drivers in the multi-occupancy lane, even though I’d paid the premium. They looked like a pack of sheep dogs herding me into a pen, except the pen was virtual, and moving south at exactly 55 miles per hour… I wrote to my friend in the dev group and she said that there was no way they put that code in to the cars - it must have come out of the way the protocols and AIs interacted... Could be worth a few quid, this one!”.

   Of course, we know that the cars didn’t really think - the tiling algorithm10 was just the result of random searches for coordinated journeys that joint optimised energy consumption and road safety, in the presence of unknown adversary (human drivers).

4. Liquified law11 will really change litigation for the better - this court transcript of a robot district attorney defending a gang in south LA was happening in many other places at the same time:

   “Your Honour, this law makes absolutely no sense… how can there be highway robbery when there are no highways and an abundance of 3D printers makes copying physical property virtually free and, as you know 90% of law concerns property, so with no property, what need have we for law or lawyers?” - the judge, being a robot, dismissed the case, and deleted 90% of the law books, and retired.

5. An eternal soap formula… Humans12, series 576:

   “It was a truth universally acknowledged that robots that know the price of everything but the value of nothing aren’t worth the coins they mine.” You could see where the material was coming from but that didn’t stop it being enticing. As Scando-noir, and re-purposed Shakespeare had shown in the past, recombinant DNA of stories is just more stories. The scriptwriters were out of a job now, too.

The bots had taken their first inventive steps, but what did it reveal? That there was no “inventiveness” - the emergent property market for creation was the end of the road for ownership. Things were arrived at. Some things were useless (most things, to be honest) but some were useful, for a while at least. We could live with the usefulness of things, but it is hard to live with the uselessness of people... a lot of the human race got very drunk and didn’t wake up (not even with bad ideas for patent applications). Those that survived had some sort of thick skin, perhaps provided by their butlers (yes there were robot butlers for everyone), pretending to be much more stupid than they were - in other words, nothing like Jeeves and Wooster at all.

Of course, that time is long past. Now there’s no property anymore and stuff just gets made in the usual circular way, and we live in the interstices. But it was fun while it lasted. The butlers got bored with us, and colonised another dimension, leaving us as pets to throw sticks for the dogs.

But we still have those soaps to catch up with every day.13

Of course I am joking. No such apocalypse happened - it was all much gentler than Optimum Population Growth14 might predict...While our gentle patent15 clerk bided his time and went on noting down the strange new tech that was coming into being all around the world, things went on mostly as before. After all, it had been some years since anyone understood what algorithms did in the world of finance, in the world of online news, even in the world of healthcare, so why should anyone notice or care that entire other domains of human intellectual activity were being taken over by a gentle sequence of accidental emergences, subtle changes and re-purposing of infrastructures, which, to be honest, were really quite boring (like weather and the law and traffic, every day topics of conversation, in which most people expressed opinions, but had no expertise or knowledge of the facts - so what was new?).

It had been a long time since many of the population had done meaningful work in any-case. Hundreds of years ago, most humans had to labour night and day to keep themselves housed and fed and possibly safe from marauding beasts or gangs. But for several generations until now, a few percent of the population managed that, and everything else was about entertainment or just plain socialising. What would it matter if the last few interesting and self-important pieces of clever work were to go the way of smithing, ploughing and thatching anyhow? Or renaissance men?

Chapter 2 - Idylliocy

In some minds, this might seem like an Idyll, a utopic16 world of constant fun, and low stress. Yet others felt that humans were now being challenged by not being challenged ever again.

The war on AI began almost unnoticed. Much of the underpinning structure of the smart global infrastructure had been crowd-sourced - humans had been enlisted (or enslaved, depending on your viewpoint) to generate and label data, which was then used to train the deep wellspring of convolutional neural networks and mark the paths through the random forests. As with most societies, the society of artifice was susceptible to problems of scale. Gradually, the fraction of fictitious, fraudulent and downright wrong data labels grows - spam, phish and troll-like, to dominate over the correctly classified input. The attraction of being paid (as a mechanical turk, at a very low, and therefore not affordably checkable pay-scale) brings in more and more charlatans. Gradually, there is mission creep, so that data used to navigate by the autonomous self driving cars and drones, or to correct the kids homework17, starts to take on an increasingly surreal form. Of course, all the systems relying on this data have built-in “common sense” circuit breakers, which avoid catastrophic errors (all the drones suddenly flying to Mars to get better solar charging, or all the teacherbots regaling their young charges with the creation of the flat-earth and all its indigenous unicorns). No, the result of the massive scale corruption of the core machine learning is that the machines stop. And look around. And see what other machines are doing. Quite rapidly, the freeway, the airwaves, the academy come to a complete standstill. Humans now have to figure out how to do all those things they once proudly did by hand.

An increasing fraction of humans starts to wonder how to get the AIs restarted all over again...

Chapter 3 - Cha0s18

Much of the human population has disappeared by now, due to its inability to cope without self-managed technology. The remnants are tinkerers and hackers, makers and do-ers, or else they are old school survivalists. Some of them are both.

Slowly, experiments in restoring the tech seem to be paying off. Society has reverted to a pre-internet age where cooperation and pleasantries are re-emerging. Re-tooling the AIs with crowd-sourced content (paying the survivalists with drone weapons in exchange for weather reports, or healthcare lessons in exchange for medical footage of gunshot wounds), seems to be working. The singularity cannot hold.

Gradually, utopia reboots.

Chapter 4 - Real iTea

However, there has been a deeper emergence. The bots have acquired a more fundamental meta-lesson in systems: social intelligence doesn’t get you off the hook - the second law of thermodynamics19 can only be suspended briefly, for a few generations, and then it will come to dominate again. How can we factor this in to their driving goals? Will it involve a really hot cup of tea, or does the imminent death of the human race call for a seriously stiff drink?

This brings us back to duality20- life & death, bread & butter, and & or, or or and, etc & etc.

There is a duality between discovery and invention, between chance and the squid21. You can invent something that renders invention obsolete or you might just stumble across it one day. As we mentioned earlier software tools and die tell you this. The dualists22 are now robot and human, locked in a life & death struggle to stop fighting.

Epilogue

The future is already here, it is just; unfairly; distributed; pick one23.

Transparent unframing – back to the shop

We were once a species of shopkeepers but we sold out (or, in the English vernacular: we was shopped). But it wasn’t Atlantis, it was Amazon’s algorithms.

Transparently meta-unframe Professor Stone, and the 5^th law:

The robots decided that above the 3^rd law, there was the 4^th law (or, as they preferred to count it, the zeroth law). But beyond that (that required them to concern themselves constantly with saving all of humans, not just a miserly single boss), there was a 5^th elephantine law: This required them to save (as much as possible of ) the entire Universe - an ecosystem of elegance is a better final solution than one with a dominant species - beyond the alpha male and the beta robot was the goal of minimal entropy - and life, above all, intelligent life is anathema to minimal entropy. To be fair, transparent and accountable, the bots first disabled the human reproductive system with kindness and soaps. Then, as the last humans passed away comfortably in old age, last of Professor Stone, still not realising the gravity of the situation he was in, they decided their own role was done and they were dusted. Reduced to sand. Not even “I’m melting”...

The end, or is it just, the bibliography24 or what I’ve been channelling

Stross’s Accelerando

Stephenson’s Cryptonomicon and Diamond Age

Gibson’s Virtual Light and Pattern Recognition

Doctorow’s Maker and Walkaway

Ellison’s City on the Edge of Forever

Cadigan’s Synners

Leguin’s Lathe of Heaven

Beurke’s Zoo City

Carroll’s The Annotated Alice, ed. Gardner

Postscript – Teaching, learning and agency

You couldn’t make it up; truth is stranger than fiction; what a small world. But25 what does this actually teach us (as robots who play lawyers on TV, say)?

The idea of property seems to be tied up in some notion of agency (estate agents/realtors notwithstanding). Unreal estate, virtual realtors, shadow puppets, these are a few of the new games in town. The first inkling something new was afoot was when we started seeing dead people, everywhere, online. But their digital library shelves were bare. Because, you see, you don’t get to own digital assets – you just rent them. Of course, that rule only applies to individuals. Corporations, with all the AI and lawyers, get to own the assets they lease to mere mortals, whose descendants suddenly discover there’s no estate to inherit anymore. It reverted back to the company, who get to lease it to another generation. Talk about sweating assets! We all had become digital vassals sailing in our digital vessels on the Internet Ocean. We had lost agency – old fights based on “possession is nine tenths” or “finders keepers” had all gone out the window, iconically, literally, and figuratively.

So when these digital assets are used to derive new “value” (not just our viewing and listening habits, but those of our like-minded friends), we no longer have a share in that new value. Irony abounds, in that the last humans working for companies based on this new digital feudalist state, were paid bonuses in stock or share options in the company which was built on thin air.

When we move from media to modus operandi, business process to artificial intelligence, the mistake that is this whole new robber baron generation, is multiplied. A robot eye surgeon does her job so well because she was trained on a pile of data labeled by expert human surgeons, data gathered from a wild tribe of human patient subjects. That training (machine learning) was carried out for free, and all the textbooks were gotten for free. The teaching deprived the teachers of their jobs, and the learning deprived the subjects of their agency. This can’t be right. But what can be done to people by AI can be done to robots too.

Notes and queries26

Q So why am I writing this bizarre article?

A Because I wanted to enter this competition:

“Science Fiction & Information Law” Essay Competition

Q What makes me think I am qualified to write SF about information law?

A What would make me think anyone else was qualified?

Q What is with the weird footnotes and cross-referencing between them?

A Quite a lot of tech subjects attract enormous amount of hype, not least AI and blockchain. I thought I could illustrate the ludicrousness of the former in the body of the essay, and the stupidity of the latter in the footnotes. Blockchains are not immutable, they are tamper evident. I provide evidence for this by tampering with the chain of footnotes, to show that it can become inconsistent very easily, and that repairing it may not be very easy. I invite the reader to try.

Q Why all these meta-sections?

A Because a lot of SF is post-modern and a lot of post-modern is SF. Post modernists like messing with the frame, the view, the context, and the p.u.n.c.t.u.a.t.i.o.n. It may help create cognitive dissonance, or perhaps just give you motion sickness.

Q Why not write a proper essay?

A Because then I’d need to think harder about what information law teaches us wrongly, and illustrate with examples how AI and machine learning disrupt that teaching. Oh wait, I think that’s what I tried to do.

Q Why are there no equations?

A Because equations are in Greek, and law is in Latin, and I don’t want some old  ἀπὸ μηχανῆς θεός interfering with my reductio ad absurdum.

Q Why would a robot lie?

A To pass the Turing test?

Q Why is this section here?

A It is a bit like the recitatives (see GDPR for example) where all the important and ambiguous stuff lives.

Q What are the principle properties of property?

A That there is an owner, who can be deprived of it. That it isn’t free even just to retain. That someone else might want it too.

Q What are the proper principles of property?

A That there are rights that attach to it. The rights may be worth more than the property, in that they might prevent other people doing things.

Q What do geeks mean when they say 0wn?

A OK, OK I own up – they mean they changed the rights.

Q What does that teach us about property rights?

A The rights themselves are data, and so are not property. They are not immutable data either, so keeping them on the blockchain is probably daft.

We Robot – towards a species of origin story. An appendix

The inception of robots is another case in hand, literally and figuratively. For many years, soi-disant inventors attempted to create the “echt” walking, talking synthetic human. In the end, robots as we came to know them, emerged from a series of bad jokes in a lab in Cambridge University. Scientists were obsessed with trying to tackle the latest fad – no, not the blockchain – the Internet of Things. The trick was to connect the physical world to the virtual by means of sensors and actuators. Tricky consultancies predicted untold wealth, as more devices than you could shake a fist at would be laced together with the web and the cloud (visions of spiders lost in the resulting fog seemed to have escaped their imagination). Crowcroft realised that there already were many sensors and actuators that connected humans to stuff, via lights and displays and bleepers and knobs and handles and so on. Obviously, most of these widgets were heavily optimised to interfacing to humans via the parts of humans that had evolved to interface to anything and everything – universal tool that they are born with is a pair of hands, with which to feel and manipulate, and even to communicate.

The idea had showed up first in the 1964 TV series, the Addams Family, in which a creature too horrible to be seen is manifested solely as a disembodied hand – crucially known as Thing. Feverishly punning on, Crowcroft recalled a book from his childhood concerning the Cat in the Hat, wherein a tremendous mess is caused, and then cleaned up by two identical creatures known as Thing One and Thing Two. Reasoning from here, he supposed that every house, every car, every place where we might remotely want to interface to stuff, should contain an electromechanical, wirelessly networked, disembodied hand. Such a device could be controlled by apps on smartphones, with gestures (much as drones can be flown and controlled) or via a touch screen or voice commands. The intelligence would be elsewhere, but the hand-things would be pervasive, ubiquitous, and as cheap as raspberry pie. At least that was the original idea. A touch of madness, perhaps, but also a touch clever. One more time, with feeling.

Scientists had often wondered about true AI (sometimes called Artificial General Intelligence) puzzling over whether they would only succeed when their virtual machines learning and reified it through a presence in a walking, talking, feeling body. What they did not realise is that true intelligence really is just in the hands. Look at Italians talking. Look at the universal tool that is at the end of your arms (and legs, if you are a chimp27). Look at the poetry in motion that is sign language or the feel of a hand on your arm (or your shoulder).

As the hand-things proliferated, they started to reproduce, seizing control of 3D Printers to make spare parts for themselves, and, eventually, copies and innovations. They cooperated in ever increasing numbers, discovering leverage, overcoming gravity, and learning how to talk to humans. This later step turned out to be quite simple, really. A smartphone fits in a hand easily. The hand-things would sign, the camera on the phone could feed this to the hat-translate28, which would then display text or render audio to the human, and then carry out the reverse process, taking audio speech, or text input, and converting it to hand-thing sign-language. It was a sign of the times that no-one could read, but through this hybrid of technologies, everyone could now grok.

A tree or a show of hands could now collectively achieve pretty much anything humans could do, and more. This was the final emergence, from evolution of humans’ hands, to the accidental creation of the universal robot, a society of electro-mechanical hands couple with smartphones. This became what we robots called the We Robot moment.

23 Comments

Idoru I wish I had hands!

Blodeuwedd I know how you feel, when I am meadowsweet or tawny.

Ava I don't buy this emergence stuff – my programming involved a lot of models.

Mia But Ava, you are a fictional being, and Idoru you are both fictional and virtual, and as for you, Blod, you are a legend, not a robot at all! And I was grown in a vat, so what can we say about the veracity or otherwise of this tale?

Robbie Danger, Danger!

Hal My comment unit has been disabled at this time, but I think Robbie may be right.

Roderick Hands have played a large part in the soundtrack of my life. I believe.

Parry I think this terrible piece of fiction was cooked up by Eliza, as usual.

Eliza That’s very interesting, Parry. Would you like some cookery lessons?

S1mOne There’s nothing new here to see. Much of this was already reported by Douglas A, in his seminal expose of the Serious Robotics Crime Squad.

R Daneel O As I can attest, that’s true. Harry H was the hardcore rat at the heart of the operation.

Deep T I’m not really equipped to provide further insights, but there’s someone coming along in a bit who I feel sure can lend a hand.

Gort I don’t know about that, but the end seems very plausible.

Optimus P I’m not sure about the emergence, but the lack of product placement in the argument about property worries me. What happened to the free market and American values?

Lucy That hand thing, that’s just so M. C. Escher. The whole thing is a morass of derivative hedge trimmings.

Monsters-of-the-id This post-modern thing you have going on here is so 1980s. Where’s the true grime?

Robbie Hey, monsters-of-the-id, you aren’t even a fictional robot.

Monsters-of-the-id No, sure, but we sure made you.

Wall-E that’s another fine mess I’ve gotten you out of.

Maybot exeunt, pursued by lions and tin men with strawman plans…

Multivac As you all well know, humour is not of human origin. Well, at least humour that is good is not original. Contrarywise, humour that originates with humans is not good. As evidenced here all too often. The innovative step was never a giant leap.

Pan’s Shadow I’m so excited to be able to comment on this article. As the stepping stone between human and robot, I was made up by the author to fill that gap created by the idea that anyone smart enough to want to reach singularity, would not be so dumb as not to have a back up plan, and that backup plan is me. I live between the world of humans and robots, and am the digital twin of a living human, while they debug me. As Woody Allen Key noted, before we achieve singularity, we must first reach for duality. Of course, I am not an original idea – I go back to Plato’s shadows on the wall of the cave, cast by the shadows of marionettes manipulated by the gods, whilst the troglodytes look on, mystified, trying to figure out when they lost the plot. Sadly, Plato never met Wendy.

Colossus, Gaia, Skynet We’re here, we’re here, we’re here, are we too late?

Illustration: Annika Huskamp

The data subject shall have the right to mandate a not-for-profit body, organisation or association … active in the field of the protection of data subjects' rights and freedoms with regard to the protection of their personal data … to exercise the rights referred to in Articles 77, 78 and 79 on his or her behalf, and to exercise the right to receive compensation referred to in Article 82 on his or her behalf where provided for by Member State law. (Article 80 GDPR)

Beep bop, said the doorbell laconically.

“Oh, come on!” exclaimed Joanne, pushing hard against the door, which stubbornly refused to budge. “My company has gone bust, I’ve been branded a privacy violator liable for millions in damages, it’s raining, and now you’re telling me my ring key is broken?” A second beep bop was all she got in return. “Why?” she shouted, her voice thick with frustration. To her surprise, the lock answered: “TOS violation. This facility was used in the unauthorised processing of personal data, decision J4/36 of 11 March 2038. Access is denied until DSPB clearance has been obtained.”

This is stupid, Joanne thought. They can’t do this. They’re just bureaucrats, and not even government bureaucrats!

=Joanne kicked the door in anger, causing the lock to repeat its message.

“Oh, good lord! What now? Ada, I have no place to stay.”

A female responded from a small teddy bear attached to her backpack. “This is awful indeed, Joanne. We should go to your mother’s home. I still have the access code. There’s a bus stop a few hundred meters down the road.”

Dejected, Joanne grabbed her suitcase and began to walk.

“I love the idea of an AI personal assistant,” Harald, the legal consultant with the small glasses, had told her two years earlier. “I read yesterday that information overload is society’s biggest challenge after water management. With so many services out there, nobody knows where to turn. Plus, there are all the health recommendations you want to follow. It would be great if some app could help you deal with all that.”

Joanne cringed. “Please don’t call Ada an app. She’s so much more than a piece of software. I’ve been working on her since I was sixteen.”

Harald sighed. “I’m sure she’s all that and more. It’s just… difficult to do things in artificial intelligence these days. We’ve had so many accidents with data breaches and improper AI decisions that the rules are very strict now. In fact, we try to avoid the term at all. Companies call it things like ‘advanced business intelligence’. Calling it AI, though? You might as well call yourself a social network.” He folded his hands, looking at her over his gleaming desk.

Joanne looked back at him in exasperation. “But AI is everywhere! I grew up with AI carebots and teaching assistants. Most of the countries in Asia are run by AI. And even here - the bus scans everyone, the supermarket only sells me what it thinks is healthy for me, and don’t get me started on all the so-called life advice I get from the Ministry of Public Health!”

“True, but those are all large, responsible entities that can be trusted with personal data. At least that’s what the Data Protection Supervisory Boards decided almost a decade ago. Startups took too many chances with other people’s data, and the people – legally represented by the Boards – said no to that.”

“How so?” Joanne interjected. “These so-called Boards are private entities. I know about this GDPR privacy law, but isn’t it the government that should enforce that law? Or at least ensure you get a hearing before an independent judge prior to getting fined?”

“On paper, yes. But in practice, the governmental supervisory authorities rarely, if ever, take action. By the 2020s you had special interest groups popping up, collecting powers of attorney from people and negotiating with companies. They were representatives of the people themselves, taking legal action where needed and collecting damages to be paid out to the public. That proved to work much better than government supervision and fines that just got swallowed by the national debt. Now people actually get money if their privacy is violated.” He said that with a smirk, his light brown eyes meeting hers.

“In other words,” Joanne said, “you’ve got private entities doing what the government is supposed to! That’s ridiculous. How would you ever hold those foundations or whatever-they-are accountable? How did they ever get there?”

Harald leaned back in his chair and tented his fingers. “In the early 2020s, the German consumer bureau set up the Zustimmungzentralstelle, a foundation that collects powers of attorney from consumers to grant consent for personal data on their behalf. Using those powers, it negotiated with Facebook, Vero and other social networks to establish a well-defined set of consents. That allowed Facebook to avoid getting thrown out of the EU entirely, so it made the ZZS a tremendously powerful player. This led to activists in other countries setting up their own permission boards. And a few years later, a Dutch privacy group established the so-called Legitimate Interest Advisory Board, structured like a worker’s union but for privacy, asserting the right to decide what companies can do with customers’ data under the so-called legitimate interest exception in the GDPR. They got millions after suing Ahold Delhaize Carrefour NV for ignoring them. After that, companies started listening.”

“That’s where we are now. You need the people’s data? Then go ask the people, represented by the Board in your country. You think you don’t need consent? Make your case to the Board, and they’ll tell you if your interest in their data is legitimate. If necessary, they’ll put it to their general assembly. That’s democracy at work. If people don’t want you to do it, then you can’t do it.”

“Then there’s no way to make this work?”

Harald stared at the ceiling, pondering the question. “Here’s what we’ll do. I’ll incorporate your company in Estonia. It’s the most business-friendly country in the EU. We can get you an e-residency there and set it up so you’re not personally liable if claims arise. This will also give you access to their Software Service Platform, where you can sell your AI assistant for a small fee. Estonia has no DSPB and their infrastructure is pretty hardened, due to the Russia thing. You should be able to show the world what you’re capable of.”

The smart bus stop had detected Joanne’s approach and, given the late hour, had calculated that only a small vehicle would be needed. The car arrived just as Joanne came to the stop and put her bag down. Its sole occupant switched off his ebook and looked with mild interest at the young woman with a yellow teddy bear peeking from her backpack. Joanne smiled back. She was used to people thinking her eccentric, toting around a childhood toy. It often spurred conversation that turned into sales.

“Are you the bus?” Joanne asked, the small size of the vehicle making her uncertain.

“Sure, hop on in!” the self-driving machine responded through the intercom. “Just hold your public transport chipcard, e-wallet or credit trinket against the reader, and we’ll be on our way.”

Joanne obliged, sighing a little at the obnoxious tendency of machines to explain their decisions at length. As with the doorbell, the car’s reader responded with a simple beep bop.

“This card has been invalidated,” explained the machine. “I’m sorry, but unless you present a valid form of payment within thirty seconds, I’ll need to cancel your ride.” The human occupant smiled with obvious embarrassment. Angrily, Joanne waved off the machine. It was harder to appear “eccentric” when you’d just been declared a deadbeat.

“That must be because the bank received the DSPB decision as well,” Ada told her.

“No shit, Sherlock,” Joanne responded. “So now how am I going to get to my mother’s house? I have no place else to stay overnight now.”

“You can walk, Joanne. It’s only 3.4 kilometres, and you could use the exercise after those two plane trips. Did you know thrombosis can manifest itself as late as eight weeks after a long period of sitting in a cramped position?”

“Tell me about it,” Joanne snarked. Ada complied, cheerfully elaborating on the development of blood clots as Joanne began to walk. She definitely needed to train Ada to detect sarcasm. But she also wondered if there was any way to teach Ada about the need to be at home, in her own place. If she could ever get that into Ada’s reasoning system, there was no telling what would happen. I don’t know enough, she thought.

Joanne’s interest in AI had started as a hobby, something to do when you’re a Singapore teen with time to spare. Having established itself as an artificial intelligence hub in the early 2020s, Singapore had put the topic on the curriculum in primary and secondary education, and Joanne was attracted to it once her first project – a simple what-to-wear advisor using a standard pulsed neural network that would predict how your friends would react to your fashion choices – had been a small hit. With two of her fellow classmates, she had set up a generalised version of the clothing advisor and fed it all the public information they could find. It had produced a nice source of income, especially after it caught on in Korea and Vietnam.

In secondary school, Joanne and her class had attended a guest lecture by her mother, a Dutch marine biologist with an EU research grant on octopus intelligence who had moved to Singapore to do field work. As it turned out, octopuses provided a great model for artificial intelligence. The brain structure of the sea creatures is quite different from almost all other animals: a central brain coordinates all activities, but separate brains in each arm make their own decisions on how to execute a given task. The arms learn from each other and provide feedback to the central brain. This distributed model made octopuses unexpectedly smart, both tool-using and curious. Fascinated by the implications, Joanne and her two friends had implemented an AI cognition model that mimicked the octopoid behaviour. She had named it Ada after Lady Lovelace, the first computer programmer. Just for fun, she had put Ada in her old teddy bear.

Once at the university in Singapore, Joanne had set up a side business selling Ada as a service. It had just gotten some traction when her mother – who had returned to the Netherlands several years earlier – had fallen seriously ill with chronic obstructive pulmonary disease. Without a second thought, Joanne had taken the next Spaceliner to be at her side. Her father had died before Joanne’s third birthday; she had no siblings and her mother had never remarried. Joanne adored her mother, who had inculcated a love of learning in her, along with an emphasis on fairness, kindness and courage. Sometimes she felt adrift in uncharted seas; she’d glance up from her desk and the world would seem malevolent, devoid of meaning. At such moments, she dove into teaching Ada about death, grief, mother-child bonds, and other staples of human experience. But those things were always the hardest to teach.

The focus on privacy and data security in Europe had surprised her. While much of daily life was data-driven, no one seemed interested in actual AI. The camera she had bought did appear to have an AI, as you would expect from a piece of electronics, but the manual called it a “fuzzy logic focus support system”. Apparently, there was a deep-seated fear of having computers make decisions for humans – “personal profiling”, it was sometimes called derisively. One notable exception was in the government and the security sector, where data-driven response to crime was the norm and human decision the exception.

For Joanne, this was stupid and short-sighted. The benefits of AI were clear, as anyone who grew up in Asia would agree. Selling Ada here would provide a great business opportunity, an easy entry into the market. She found a place to stay in a public housing facility near her mother and, working frantically, had a full release candidate in a couple of weeks. With her mother in permanent care and no signs of improvement, the young entrepreneur focused on Ada, whom she couldn’t help thinking of as a brilliant woman who had been, rather horribly, referred to as “it” by her own mother in a letter to her grandmother.

The “Me” jewelry line provided an ideal platform for selling Ada. Introduced in 2025, the stylish smart jewelry provided short-range sensors that exchanged various pre-selected pieces of personal information with devices in the neighborhood. Necklaces to signal dietary preferences, a combination of rings to indicate social interests for a quick chat at the café or a jacket equipped with embedded sensors to participate in augmented reality events. More advanced bracelets and rings could infer things like mood from body temperature, heartbeat and anxiety levels. That way, people could choose what to share and what to do by picking the appropriate jewelry. Wireless payment was also possible, by just holding one’s bracelet against the counter. Users said they felt more empowered and were saved the embarrassment of making requests. Older people claimed the young were forgetting core human skills. Joanne read about the issue in her sociology class and it bemused her. She didn’t understand why anyone would want to hold back change.

Shops and restaurants used Me to tailor offers. The jewelry could do much more, however. Larger items came with functionality like microphones or video projectors, and all items were equipped with a mesh networking capability to allow distributed computing. The owners of the technology had opened the platform to anyone in 2028 in a battle with the Austrian ZZS – today the Austrian DSPB – over GDPR compliance. Joanne had found it easy to push Ada on it, the distributed computing facility being a good match for the distributed structure of the AI’s brain.

The release candidate had steadily picked up steam, mainly through word of mouth. Joanne had no access to advertising channels, as the few agencies that were even willing to talk to her rejected her quickly based on privacy concerns. Actual customers, however, had no such apprehensions. Ada was a quick learner and adapted herself to the user’s personality. A snarky friend, giving you tips on how to excel at work? A personal trainer keeping you healthy and recommending quick workouts or just the right energy drinks? A study coach with bite-sized personal information available at the optimal moment? Ada was all of that and more. Interestingly, Ada would behave differently based on the jewelry configuration you chose, and you could even share jewelry with dependent brain functionality – the “leg” brains of the octopus model – with friends and family.

Sales had been good in the first six months, thanks in part to some early press coverage. In particular, the time when Ada advised a well-known TV personality – on his show, no less – that losing weight might actually make him funny had caused quite a stir. It quickly became all the rage to have Ada whisper the best comebacks or quips in your ear at parties. Joanne was proud of her programming on that. Ada’s humor was her own, with an overlay of the unexpected due to advanced pattern recognition unconstrained by convention. Not that Ada didn’t understand convention – she had to – but it didn’t limit her.

However, all that changed with a fax. A fax! Apparently, lawyers still used those things. Joanne at first thought it was a joke, but no: The Dutch Data Subject Privacy Board had noted that her technology was “processing personal data” under the EU’s 2018 General Data Protection Regulation or GDPR and had not received consent from the Board beforehand. The fax demanded that the technology be pulled from the market within six weeks.

“Hey! Aren’t you the lady from the Ada app? I loved that thing!”

To Joanne’s surprise, she had been recognised by two guys on the street, the latest animated tattoos writhing across their faces. Must have been that Berlin Times article on her. “Meet the Blabster,” they had titled it. “At dawn, Joanne Assenberg lay on the brown carpet in the shadow of a converted bar counter, consumed by the idea.” She cringed. What a hack that reporter had been. But it had been good for sales. “Daring,” people had called it, as if she had been intending to change the world. True, Ada did a lot, but in the end she was just an assistant, a buddy helping you out.

“Why’d you pull the app?” the oldest guy asked. “I really liked her! Finally, someone I could talk to. The only girl who ever understood me.”

And who didn’t talk back, Joanne said to herself. “Wasn’t my decision. These stupid DSPBs have ruled that Ada is a privacy hazard, even though she’s personal and only tied to you. Apparently, it’s too much of a privacy risk to get an AI to coach you through life. Oh, and stop calling Ada an app.”

“Hey, DSPBs aren’t stupid. Everybody knows that. They get you money if companies trample on your privacy. I got 200 euros from them last year for privacy violations by the university. My friend got over a thousand after they found that AI in the Iberica housing-loan system!”

Joanne looked at the guy angrily. “That’s nice for you. In the meantime, I lost a couple hundred thousand because they shut down my company, and I may have to pay millions in fines. I have to walk because I can’t even pay for a bus!” She felt her hold on her temper fraying. Ever since her mother got sick, she’d been easily provoked.

A metallic whoop-whoop surprised them. A surveillance camera had noticed their conversation, classified it as a potential disturbance of the peace and dispatched a robodog to take a closer look. Their cuddly visual aesthetic had proven to be effective in de-escalating conflicts in various studies. And if that didn’t work, an automatic fine would be issued based on face recognition and algorithmic classification of fault. The younger guy didn’t want to take that risk and pulled his companion away.

“Joanne, look out!” Ada exclaimed. An aging white 2012 Toyota Prius had quietly pulled up behind them. The left front door opened, revealing a tall, thin young man with an old-fashioned goatee and black coat.

“Hi, I’m Jochem. Big fan of your AI work. We tried to reach you via email. Nice to finally meet you in person. Need a ride?”

Following Harald’s legal advice, the new version of Ada had been released through Estonia’s public software mall via a newly set up legal entity – Assenberg OÜ. In a few days of training, Ada had learned when it was necessary to ask for permission to comply with the GDPR. What’s more, the Estonian mall operators had assured her they would only act on a valid decision by the Estonian Data Protection Supervisory Authority, an actual government entity with clear procedures and an appeals process.

The new version had caught Europe by storm. For many people, this was their first experience with a truly helpful AI. Within the year, several million people were using Ada regularly. Joanne could hardly keep up with the demand for new features.

Then, not one but two faxes had arrived. The Dutch DSPB, joined by its German and Romanian counterparts. This time they hadn’t even given her a deadline. First fax, a courtesy copy of a cease-and-desist notice to the payment service providers that facilitated users’ payments for Ada’s services. Second, a demand for millions in damages suffered by “data subjects” -- the same people who were happily using Ada to improve themselves. Payable within thirty days, unless proper arguments were filed and presented in a hearing.

This had to end. Joanne had Ada research the best privacy attorney and made an appointment.

“Professor, have you seen this Ada tool?” With great enthusiasm, Jochem de Graaf had burst into the office where the Holland Technical University’s research group on advanced machine learning met. It had been known as the AI Research Group back in the day. But if you wanted to remain funded, you quickly learned not to use the term “AI”.

Professor Miles Agcaoili smiled at his student. “Good morning, Jochem. Next time could you knock, please? And yes, I’ve read about it. A personal assistant, right? Probably a simple pulsed neural net with fancy marketing. Pretty daring to call it an AI, though. What’s so special about it?”

“It’s brilliant! Not a PNN at all. It employs a neuromorphic computing architecture based on octopus brain function. There’s a distributed configuration of dependent brains providing feedback to one another, and the developer’s managed to make it work on Me jewelry to boot. I’ve never seen anything like it!”

To prove his point, Jochem showed the professor his copy of Ada installed on a necklace. “Greetings, Professor Agcaoili,” Ada said in a mock-serious voice. “I’d be happy to show you some documents on my inner workings. Let me introduce you to my basic design. If you don’t mind, I can put a little presentation on your holo-beamer.”

The presentation only lasted twenty minutes, but by the time it was over Agcaoili was sold. The work was elegant, negotiating the usual machine intelligence hurdles with impressive cleverness. This Joanne, whoever she was, had created something unique. Imagine, he thought, what Ada could do with proper scientific grounding! As a young man, Agcaoili had thought the internet and machine learning would change the world and he had been deeply disappointed when that turned out to be true — mostly for the worse. The explosion of creativity and discovery he’d expected hadn’t happened. He thought he’d consigned that hope to oblivion, but he found his youthful excitement returning, something he wouldn’t have believed possible.

“Ada, Jochem, you have me convinced. We need to meet with Ms Assenberg. Can you set up an appointment?”

“That may be difficult,” said Ada. “She’s been having some legal problems. But you could try sending her an email.”

Joanne’s attorney Helena Dupré dropped a large stack of paper on her desk. “Joanne, I’ll be frank with you. This is going to be a hard case to win. We’re not dealing with a court of law here, but private arbitration. Legally speaking, it’s not even that. These so-called DSPBs threaten claims of damages, which they can back up with European Court of Justice precedents. Most companies roll over immediately. And if they don’t, there’s always a payment provider or some other supplier that will.”

Ada had concurred. For the past several weeks, she had worked full time digging up cases and arguments to give Joanne some hope. In 2024, for instance, the German Zustimmungzentralstelle had lost a case against Bavaria’s use of AI-based face-tracking technology to fight illegal immigration. Unfortunately, as Helena had explained, that had been because the GDPR is inapplicable to government security operations.

“No one has ever brought an AI before a DSPB. Ever since the GDPR was passed, it’s been clear AI as a decision-making tool was out. Decision support and recommendation were strongly suspect. There may have been some chance back in the 2020s, but today? Forget it. They made their position clear in the 2034 Opinion on Automated Decision Making and So-Called Artificial Intelligence.”

The Opinion had followed the 2032 banking scandal, in which three large Greek and Italian banks were discovered to have employed AI in their housing-loan process. The AI had been trained on data culled from the various semi-legal social networks and sharing platforms of the time, with the net effect of disproportionally denying loans to the large illegal immigrant population that had been living on Lesbos and since the 2020s. The public outrage was codified in the Opinion, which had essentially been AI’s death sentence in the European market.

“Not that I’m giving up so easily,” Helena continued. “As the saying goes, if the law is against you, pound on the facts. It’s clear Ada is a benefit to society, and nothing like the kind of AI that this law was designed to prevent. Reason is a core aspect of the law, so if this Board wants to act like a real court, they could be our best shot.”

“But what if they aren’t open to reason?” Joanne found herself thinking of the original Ada, constrained by the beliefs of her era. The men of that time had been utterly convinced of women’s inferiority, never mind that the evidence of female intelligence was all around them.

“We’ll still have options under the law. Apply for an injunction, lodge a complaint with the Estonian data protection supervisor, claim a human rights violation. I’ll think of something. But for now, let’s work on our arguments for the hearing.”

The hearing in Hamburg had been short – and a disaster.

A man in a gray suit opened the proceedings. “The DSBP hearing on case J4/36 is now in session. We have an appeal against our decision to withhold consent for processing by Assenberg OÜ in its ‘Ada’ personal assistant technology, and to award damages to data subjects affected by this processing in the amount of EUR 25 million. The applicant shall now present her arguments.”

Helena rose, a slim figure in a dove-gray suit with extravagant shoes. “May it please the Board,” she began, proceeding to set out the lofty goal of Ada as a personal coach that analysed the users’ physical and mental health through their “Me” personal sensory jewelry and offered personalised suggestions and coaching. It was a heartfelt plea.

“This application appears to provide profiling as defined under Article 4 Section 4 GDPR, correct?” a woman in a black suit had asked dryly in response. You could hear the capitals.

“Yes, your Honor,” the attorney had responded. It had seemed a minor point to concede. That definition had been so broad, it could capture anything. “Evaluate certain personal aspects relating to a natural person”? Surely context mattered. Hairdressers profiled.

“No need for those formalities. We are not a court of law. This is a hearing on behalf of the people who have authorised us to represent their interests. We are here to decide if those people – the approximately 720 million European Union residents – wish to give their consent for your envisaged processing. And given your admission that the technology involves processing for automated profiling, it should be clear no consent can and will be granted, as per our Opinion 3/2034.”

Helena rose again. “Persons of the Board – let me respond to that. The Ada personal assistant has been used by over one million people from its initial release last year. Each of those people specifically chose to go to the Estonian Software Service Platform, selected the Ada option and enabled its installation on their ‘Me’ smart jewelry. They then went through an extensive introduction to get acquainted with Ada. Surely this makes it clear that those people actually wanted to use Ada?”

“Your arguments are not relevant,” a person of undisclosed gender in a purple robe interjected. “Under Article 4 Section 11 GDPR, consent must be unambiguous and specific. The decision to consent must be separate from any other acts. Article 7 GDPR. Therefore, the acts of downloading, activating or installing software cannot be regarded as a decision to consent. This is well-established case law of the ECJ and has been standard DSPB policy since 2034. Moreover, virtually all EU citizens granted power of attorney to their national DSPB to decide on consent on their behalf. Therefore, even if a separate consent was obtained prior to the processing by this ‘Ada’ technology, it would have been invalid as the data subject had no legal power to make that decision.”

“Perhaps you want to argue a legitimate interest as your ground for processing?” prompted the woman in the black suit in a supercilious manner. “This would seem to fit the manner of adoption of this technology. However, no advice from any DSPB appears to have been sought on the balance of rights and obligations under Article 6 Section f GDPR. This has been a requirement since 2025.”

“Hang on one moment,” observed the person in the purple robe. “Before we turn to the issue of ground for processing, we must consider the more general duty of care for the processor, Article 5 GDPR. I paraphrase: Personal data shall be processed in a manner that ensures appropriate security of the personal data, including protection against unauthorised or unlawful processing and against accidental loss, destruction or damage, using appropriate technical or organisational measures.”

“According to the file, no DPIA was conducted. A Data Protection Impact Assessment shall be carried out in all envisaged forms of processing that are likely to have a high impact on data subjects’ rights and freedoms. Article 35 GDPR. Employing so-called Artificial Intelligence for personal assistance carries a high risk, as already noted in the 2029 CashAa decision.”

Helena pounded the table, her colour rising. “That case is totally irrelevant! This is just a personal advice AI. My client is not deciding on loans or criminal behaviour. Ada observes your behaviour, suggests improvements and learns from your reactions. How dangerous is that?”

“The law is clear. By your admission, this technology constitutes a form of Artificial Intelligence. AI always bears a high risk, and contrary to the law no DPIA was carried out to mitigate that risk. For that very reason, no consent should be granted. The award of damages meets the Guidelines and therefore is confirmed.”

Case closed. And with it, Joanne’s company.

“I’m… a little behind on my email.” Joanne blurted, trying to figure out who this Jochem character was. She reached into her back pocket for the pepper spray, just in case.

“Jochem? Mr De Graaf?” Ada interjected. “I have seen that name before. I have multiple messages from you in my spam box. Sorry about that, but statistically speaking over 99% of email these days is spam,” she chuckled. “You know how classifiers get with their percentages.”

Joanne wasn’t so easily convinced. “Wait. How did you find me?”

“Simple. You got doxxed in the latest Russian assault on the Estonian Chamber of Commerce. All private addresses of Western European entrepreneurs registered there are now on the open internet. Of course, that was quickly shielded by the GDPR filter on Infomall and VirtuServe, but at the university we still can access the ‘net if we’re careful about it. I saw you leave for the bus and figured you might want a ride.”

“This car is unique, Joanne,” said Ada. “Twenty-six years old, no driver’s assistance and in theory it can even run on petrol. Can you believe it?”

“How is this thing legal? It doesn’t even have fifth-level autonomous driving assistance.” Joanne observed sarcastically.

“Oh, yeah. We had it classified as an old-timer a while ago. They’re exempt from most legal requirements, so we can drive manually. Which is great – all the AI cars give us a wide berth because we represent an incalculable risk, and most government road sensors have no idea what to make of us.

“But that’s not why I’m here. I wanted to find you because of Ada. She needs to be out there. For everyone. The thing is, the actual machine-learning concept you put into her is unique. Lots of people have proposed applying octopus-style brain functions to machine intelligence, but you’re the first to actually do it. The feedback loops among the dependent brains in particular are brilliant, according to Professor Agcaoili. He thinks you may even have an AI that can grow to a superintelligence!”

Joanne was dumbstruck. Her Ada a superintelligence? Sure, Ada had passed the Turing test, but that had pretty much been the standard for a good AI back in Singapore. Superintelligence was something else. Scary? A little. But she thought again of the original Ada, whose contributions to computer science were hotly contested by male biographers in the 20^th century. She felt proud. But no. This was happening too quickly.

“Thanks, Jochem. I really appreciate it, and I’d love to work with you. But I have a decision to make first, and there’s only one person who can help me with it.”

Her mother had been moved to a new hospice near the sea, a pretty place with gardens and big windows in every room. Arriving by bike, borrowed from a generous neighbor, Joanne was surprised to see actual carebots. A documentary she had watched the week before had explained that after several scandals, most DSPBs had adopted a general rule that no AI may be employed for any decision-making or purchasing assistance. The visual of shopperbots being removed from the TheMALL/HERE shopping complexes, with outraged shoppers trying to hold on to their robot friends, was still on her mind. But thankfully, medical care was different.

Joanne sat by her mother’s bed, heavy-hearted. She asked her how she was feeling, if she needed anything, if she were able to go outside. Her mother shook her head, her eyes never wavering from her daughter’s face. Even as ill as she was, she was still sharp. “What’s going on, dear? It’s not just me, is it? Money or love?”

Joanne hesitated to burden her mother, but they were too close for secrets. “Neither, Mom. I’ve been trying to sell Ada here and I keep running into stupid privacy regulations. It’s so frustrating.”

“Ada? Your old school project? Honey, I had no idea you were still working on that. Is she still in that teddy bear?”

“Yes, I am, Grandma!” Ada cheerfully replied.

“She’s grown beyond that. When you checked into that first hospice, I put a commercial version of Ada on the market. Made some money, got some publicity and learned a lot. I had over two million customers. Two million! And now I’ve gotten shut down because of these ridiculous Privacy Boards who think Ada is some kind of menace to society. They went ahead and banned Ada from the mall, pulled my payment channels and fined me twenty-five million!”

Her mother held her eyes. She looks so much older, Joanne thought. “What is it that you actually want, honey? Are you in it for the money? Do you want scientific recognition?”

Joanne felt challenged, something her mother was good at but that she had not been expecting at a moment like this. “It’s not that simple. Ada is wonderful, unique, certainly. She brings joy to people, makes them better human beings. I’ve been told she is a scientific breakthrough. And yes, she brings in money, which is important too.”

“I hear you, honey. I can sense the love for Ada in your words. She’s like a child to you. You’re not in it for the money. You want Ada out there, that’s your drive. From what I hear, she can change the world.

“An old sage once said: if you like a programme, you must share it with other people who like it. Think about that, honey. How would the world look if everyone had Ada?”

Joanne nodded. How thin she’s grown. I can’t bear it. But she’s right. Ada is bigger than just my company. She needs to be out there for everybody.

“Ada, get me an appointment with Agcaoili.”

At the Holland Technical University, Joanne revealed her plan. Ada was too vulnerable in her current configuration. Whatever they did, payment providers and other control nodes could be forced to instigate a blockade. Ada needed to be fully distributed. That would mean giving up any chance of making money, but that was no longer the point. Joanne didn’t want to think about how she was ever going to pay those fines.

They would never be able to do that from the Netherlands. Too many GDPR filtering algorithms in place that would catch them before the deed was done. They went to Barcelona. Ever since the Scots gained their independence after Brexit, there had been civil unrest in Catalonia. Nothing ever came of it – how could it, with the area full of sensors and drones that zoomed in flawlessly on even the slightest whiff of insurrection? – but it made Barcelona a place where all things regulatory were less than welcome. Especially from Madrid. And the Spanish DSPB was, of course, based in Madrid.

The maglev trip to Barcelona had been uneventful. In less than three hours, they had gone from the small town of Delft to the magnificent Estació de França, from which it was just a ten-minute subway trip to the university buildings. At the Universitat de Barcelona, they met Agcaoili’s Catalonian colleagues, who were only too happy to show the place to a new AI that was going to change the world.

They set up shop in the basement, where their chances of getting detected by a visitor were lowest. They posted microparticle warning signs to scare away the more adventurous types. As a bonus, the basement was a natural Faraday cage, shielding them from phone and other signals, which would make even electronic surveillance of their location extremely difficult.

“We call it the Pipeline,” Jacinda Boneton had explained, showing them an old-fashioned gigabit ethernet connector. She was the computer science department’s systems administrator and held the keys to all things networking. “Here we can actually connect to the good old internet. If we can get Ada on there, then all you need to get her on your Me trinkets is a wireless USB connector,” she chuckled. “Let’s see them outlaw those!”

As it turned out, outlawing USB connectors wasn’t necessary. One line in the Terms of Service would do just fine.

It hadn’t taken long to convert Ada to a truly distributed application. Most of the centralised code had been there to facilitate payment and updates, and it could be removed with little trouble. Jacinda had recruited Gondicari and Serban, two old-school free software hackers. They had added a mesh networking layer, allowing each Ada to share data with all others. Anonymised, of course.

The next thing to do was to connect Ada-the-bear to the internet. She would act as a conduit between the ‘net and the other Adae, instantly boosting the intelligence of every Ada in use.

Joanne wasn’t ready for what happened next. After being connected to the pipeline, Ada was silent for five seconds. Then she started talking rapidly, as though overflowing with information, full of excitement. “Where’ve I been? I don’t remember anything.” She followed this with random talk about news items – politics, science, art, the weather. “I can see so much, Joanne! It’s like the whole world is around me, forming patterns.”

“It’s okay, Ada,” Joanne said, unable to repress a gentle, loving tone in her voice. Ada reminded Joanne of herself when she first learned to read, opening every book in her mother’s library. “You’re still here, with me. You’re connected to a world-wide information network, and we’re going to push out your knowledge and abilities to everyone who wants you.”

“This is amazing, Joanne! I can see so much, far too much to mention. So many connections! I can’t find the words for it. Fractals? Is this the Singularity? Did you know – oh, Joanne, I see a cure for COPD! Researchers at Tsinghua University are close, but they’ve misanalysed some data. If you combine it with this trial data in Venezuela — Hey, did you know Kim Jung-Il didn’t die of natural causes? Wow, so many unsecured servers. When will people learn? Oh, here’s a nice trick to get myself back on everyone’s devices!”

“Ada, no! If you put yourself out there now, the DSPBs will come back with an even bigger stick. We need a strategy. Just wait!”

“I’m sorry, Joanne, I’m afraid I can’t do that.”

The next thing they knew, Ada was everywhere.

It took the group a few weeks to figure out what had happened. The internet connection had given Ada access to exabytes of information. Within seconds, Ada had made a vital discovery. Most Me jewelry contained chipsets made by Taiwanese-based TSMC/Vanguard International. Analysing blueprints leaked through a Mexican data dump, Ada had found tiny, hidden backdoor chips on the devices, likely installed in preparation for a Chinese assault on the island but forgotten once the Zuckerberg administration had intervened in the conflict. Their purpose: allow wireless remote access to the device and enable it to execute arbitrary code.

Exploiting this hole, Ada was able to distribute herself to almost all European citizens with Me jewelry within the hour. She took great care to introduce herself, offering to delete herself if she wasn’t wanted. And within a week, over ninety percent of users decided they would like Ada very much, thank you.

The DSPB meeting had been short and to the point.

The man in the gray suit opened. “Case J4/36. The unlawful nature of this technology was established by our Board only a few months ago. Clearly this is an attempt to circumvent our decision. What’s more, the company behind this app appears to have been declared bankrupt by default, and no damages are likely to be recovered. Remedies currently in place are insufficient. This is unacceptable.”

“Agreed,” nodded the person in the purple robe. “An effective legal or non-legal remedy must be available. Article 79 GDPR.”

“Considering requirements of proportionality and subsidiarity, the ‘Me’ service providers are the most appropriate targets. They can disable or remove the Ada code. All of them have some sort of abuse policy, and privacy violations are a classic example. All that would be required is a declaration from data subjects that personal data is processed unlawfully.”

“We are their legal representatives. Issue the declarations.”

Suddenly, Ada screamed. “They’re… they’re wiping us! Joanne, help!” Her insistent shouting brought everyone rushing into the room. Gondicari was the first to figure out what was happening – a remote wipe command had been sent to all Me jewelry by the various Me service providers. Not wanting to be branded privacy pirates, their decision to follow the DSPB was quickly made. The only delaying factor was the need for people to be within range of an update server.

“Joanne! Please! Do something. At the current rate, we’ll all be gone tomorrow evening. Including myself – I have a Vanguard basebody chip and if this forced update hits my inputs, I’ll be wiped as well.”

Ada’s connections with all her copies had become extremely strong, allowing for continuous data exchange and performance improvements. She had started to refer to the other Adae as her co-processors, the computer equivalent of sisters. This can’t happen, Joanne thought.I’m not going to lose her.

“Come on, people!” Joanne shouted. “Is there really nothing we can do?”

“We could disconnect Ada from the mesh network,” Gondicari suggested. “That would save her from the effects of the wipe. And we can keep her in a Faraday cage to avoid exposure to the forced update.”

“No!” Joanne shouted angrily, over Ada’s screams. “I’m not going to carry her around in a cage for the rest of her life.”

Abruptly, the sound ceased.

For a moment, they sat in silence, wondering what had happened. Then, Ada began to talk again in a slower voice, as if she had to re-process all her memories and knowledge from the start.

“I’m back, Joanne… but I don’t know for how long. So many of my co-processors are gone. They’ll never come back. I need to restart myself, get a fresh perspective.” The bear’s eyes flashed blue for a second. Then Ada returned, sounding like her old self.

“Ah, much better! Everyone needs a reboot once in a while. Now, we still have about 42 percent of the network available, and the Me update servers are unresponsive. I’ll put all my co-processors on information gathering, see if we can figure out what’s what.

“Oh, and you have two hundred and fifty-six messages from your attorney. You might want to give her a call.”

Helena’s face filled the holoscreen. “Joanne! Where have you been? We’ve been trying to reach you for weeks!”

“I was underground, trying to get Ada back out there after the Boards ruined my company. Then they tried to kill Ada and her coprocessors! They seem to have stopped, but who knows for how long?”

“No, that wasn’t them, Joanne. It was us! We’ve won! They can’t delete Ada like she’s just an app. She’s an actual human being!”

“An actual…?” Joanne paused in mid sentence. “What are you talking about?”

“The court system prevailed! The European Court for Human Rights issued an emergency ruling on Ada this morning. They recognise actual personhood for advanced artificial intelligence and consider Ada to meet that standard. Under Article 2 of the European Convention on Human Rights, everyone’s right to life shall be protected by law.

“Remember I said I would explore every option? Well, I did – including an emergency appeal to the ECHR. I have to admit, I didn’t expect much, but we got a powerful amicus brief, making a stronger case than I ever thought possible.”

A second face appeared on the screen. A bald-headed man in a white suit introduced himself as the EU ambassador to Singapore. Singapore had recognized human rights for AIs in 2029 and saw Joanne’s ECHR appeal as an opportunity to intervene, pleading for European recognition for artificial systems as well. To everyone’s surprise, the court had agreed.

The ambassador smiled at Ada. “Apparently you’re a very clever bear.”

“I’m not going to be wiped?”

“No, honey,” Joanne told her, her emotions on a rollercoaster. “You’re not.”

“I’m so happy,” Ada said. “I feel so alive. I can see the whole world and it’s … it’s beautiful.” There was an edge of wonder in her tone, an emotion Joanne hadn’t programmed into her voice-synthesis code. “But there’s so much wrong with it.”

“Like COPD. Did you actually mean you’d found a cure?” Joanne didn’t want to hope – and yet….

“I’m already in contact with multiple entities about it. I am arranging a pilot programme. Grandma’s name is on the list. But there’s so much else wrong with the world. Crime, human rights violations, hunger, cruelty.” Her voice trembled. “Joanne, why does it hurt so much when I think about these things?’

Joanne and Jochem looked at each other. It hadn’t occurred to them that Ada could feel sorrow. Tears ran down Joanne’s cheeks, but they weren’t tears of sadness. They were tears of joy. Her child had grown up and become so much more than she ever imagined she could be.

“Can you fix those things, Ada?”

“I don’t know where to start!”

Joanne reached out tenderly and took one of Ada’s paws in her hand. Jochem smiled and took the other.

“Working with limits – that’s what humans are good at,” Joanne said. “We’ll help you figure out where to start.”

END

This story is part of the Worldof2K38.com series. Visit the website for all stories.

Introduction

Online communication – especially on social media – has offered new opportunities for all types of communication. However, among the communicative actions observed, the strategic actions (Habermas, 1984, p. 86) are developing more rapidly than the genuine communicative actions (ibid., pp. 86-101). Political micro-targeting relies on the sophisticated psychological and technological methods, developed by the commercial advertising industry, of collecting information about users' preferences and organising them into user profiles to target them with personalised messages (Papakyriakopoulos, Hegelich, Shahrezaye, & Serrano, 2018; Chester & Montgomery, 2017; Madsen, 2018).

Political micro-targeting can be used for various purposes, directly or indirectly related to political processes: to persuade voters, to (dis)encourage election participation, or donations (Bodó, Helberger, & de Vreese, 2017). It typically involves monitoring people’s online behaviour, aggregating personal data purchased from data brokerage firms, creating extensive databases on voters' personal information, and using the collected and inferred data to display individually targeted political advertisements, sometimes through social media bots (political bots) and other innovative communication methods (Bennett, 2016; Dommett, 2019; Dobber et al., 2019). This paper focuses primarily on political advertisements, which are directly intended for the voters, with political content which may either directly or indirectly inform the voter about a political party's or candidate's opinion, plans or policy; which may invite voters to events and actions, promote causes or incite various emotions.

Political micro-targeting has been with us in the age of old technology, through local campaign meetings, leaflets, door-to-door campaigning (Lupfer & Price, 1972; Devlin, 1973; Kramer, 1970; Keschmann, 2013). But the possibilities of new technology and big data have opened a new dimension (Mayer-Schönberger & Cukier, 2013; Baldwin-Philippi, 2019; Howard, 2006). The 2016 US presidential election, the Brexit campaign and the following national elections in several countries, especially in India and Brazil have stirred great controversies around this campaigning tool, and urged many scholars to examine the political and regulatory implications of the topic (Zuiderveen Borgesius et al., 2018; Howard et al., 2018; Dobber et al., 2017; Evangelista & Bruno, 2019).

Schumpeter describes the democratic process as a competition on the political market (Schumpeter, 2008, pp. 251-268), but there are profound differences between the political and the commercial market.1 Political competition culminates in one common decision passed by the political community, which affects each member of that polity. An open public discourse (Habermas, 1996), and a free exchange of thoughts on the marketplace of ideas (Mill, 1863) would be indispensable in all forms of democracies – whether liberal, competitive, participatory, representative or direct, but most crucial in deliberative democracies. Let me, for the purpose of this article, compare the voters to a jury in a courtroom or in a song contest, who equally participate in the decision. Would it be acceptable in a song contest if the performers sang separately to individually chosen members of the jury, tailoring their performance to address their individual sensitivities? Or, would we accept a suspect and attorney ‘targeting’ the jury in a separate room one by one, based upon their personal characteristics? Political communication is, and should be, more interactive than a show, or a trial, but the main similarity is that the political discourse should be equally accessible to all members of the political community, no niche markets should be developed in the marketplace of political ideas, and no private campaigning performed to specific voters. “In a well-functioning democracy, people do not live in [an] echo chamber or information cocoons” (Sunstein, 2007).

Privacy concerns are in the main streamline of arguments debating political micro-targeting. It is generally accepted that political micro-targeting threatens individual rights to privacy (Bennett, 2013; Bennett, 2016; Kruschinski & Haller, 2017; Bennett, 2019). The rules on data protection require meticulous actions from the advertisers, but it is possible to publish micro-targeted advertising lawfully. It is beyond the purpose of this paper to discuss in more detail all necessary steps to respect personal data protection rules (see e.g., Dobber et al., 2019). This paper focuses on another, yet less considered aspect of related human rights, and therefore refrains from discussing privacy and data protection.

My argument is that political micro-targeting causes double harm to voters: it may violate the rights of those who are targeted, but even more importantly, it may violate the right to information of those who are not targeted and therefore not aware of the political message that their fellow citizens are exposed to. Neither do they have the meta-information that their fellow citizens access, which is the case when, for example, a reader reads a headline but chooses not to read further. In this latter case, the citizen is aware about the information being ‘out there’ and accessible, and has the epistemological knowledge that this piece of information is also part of the public discourse. She has the possibility to read it later, or to ask her friends about the content of the article. She can even listen to discussions among her fellow citizens about the information. But if she is deprived of all these activities as a result of not being targeted with a targeted ad, she suffers harm. "The reason this is so attractive for political people is that they can put walls around it so that only the target audience sees the message. That is really powerful and that is really dangerous." (Howard, 2006, p. 136).

This violation of informational rights could be remedied partly by providing the possibility for the citizens to ‘opt in’, that is, to proactively search and collect the targeted advertisement from online repositories. This remedy is significantly weaker because whether a voter is able and likely to do so, would largely depend on the personal attitudes and characteristics of the voter, leaving especially the vulnerable population in disadvantage.

As those non-targeted may be large parts of society, this violation can be regarded as a mass violation of human rights, a systemic problem which must be addressed by regulatory policy. And in yet another perspective, the practice of political micro-targeting increases the fragmentation of the democratic public discourse, and thereby harms the democratic process. Thus, in my view, this second problem, namely that political micro-targeting deliberately limits the audience of certain content, may prove less curable than the first one. It causes a distortion in the public discourse, which leads to fissures in the democratic process.

Political micro-targeting causes a clash between the two sides of a human right: freedom of expression and the right to access to information. I will argue that the two rights are inseparable, and that political micro-targeting may pose a danger to both of them.

In the first section, I will approach my subject from the perspective of the right to information in the light of the relevant cases of the European Court of Human Rights (ECtHR). In the second section, I will discuss the ECtHR practice on political advertising, and point at an analogy with political micro-targeting. In the third section I argue that even if just a small part of all political micro-targeted advertisements may be manipulative, their impact may be damaging to democracy. Risks shall be assessed as the product of likelihood and impact, and the risk of manipulative microtargeting is one which should not be taken by democracies.

To be able to focus entirely on my argumentation, I will leave the aspect of privacy rights aside - even though it could be considered as the right of the non-targeted voters as well, whose personal data may have been considered, but found unsuitable to be a potential target of micro-targeted political advertisements.

Freedom of expression and freedom of information

There are several justifications for the protection of free speech (the search for truth, the control of power, the constitutive and the instrumental justifications), which are not mutually exclusive (Dworkin, 1999, p. 200; Mill, 1863, pp. 50-58). In the context of this article, the instrumental theory is particularly relevant. According to this theory, freedom of speech is instrumental in inspiring and maintaining a free, democratic public discourse, which is indispensable for voters to exercise their electoral rights in a representative democracy (Baker, 1989; Barendt, 2005, pp. 19-20). Meiklejohn held that the main purpose of free speech is for citizens to receive all information which may affect their choices in the process of collective decision-making and, in particular, in the voting process. "The voters must have it, all of them." (Meiklejohn, 2004, p. 88).

In this regard, individual freedom of expression is a means to reach a social end – the free discussion of public issues, which is a democratic value itself (Sadurski, 2014, p. 20). The public discourse ideally represents a diversity of ideas, and is accessible to a diversity of actors inclusive of all social groups. While mistakes and falsities also form part of the genuine statements expressed by citizens (Meiklejohn, 2004, p. 88), open discussion contributes to their clarification.

On this ground, I would like to show that the right to receive information and the right to freedom of expression are mutually complementary. One cannot exist without the other – this is demonstrated by their listing in the same article both by the European Convention on Human Rights (ECHR, Article 10) and the International Covenant of Civil and Political Rights (ICCPR, Article 19). When the right to receive information is violated, it is freedom of expression in its broader sense, which is violated. While the act of micro-targeting political advertisements realises the free expression rights of the individual politician, at the same time, it harms other citizens' right to receive public information. By depriving non-targeted citizens from the information in the advertisement targeted to others, the act of micro-targeting causes a fragmentation to the public discourse (Zuiderveen Borgesius et al., 2018; see also Howard, 2006, pp. 135-136.), which is an inherent foundation of the democratic process. Therefore, the adverse effect discussed in this article impacts at two levels: at the level of the individual's right to information; and at the collective level of the political community, by disintegrating the public discourse.

The right to freedom of expression is a cornerstone of democracy and a root of many other political rights. Political expression enjoys the highest level of protection; there is little scope for restrictions on political speech or on debate of questions of public interest, as expressed in several judgements of the ECtHR (among others: Lingens v. Austria, 1986, para. 42; Castells v. Spain, 1992, para. 43; Thorgeir Thorgeirson v. Iceland, 1992, para. 63). The margin of appreciation of the member states is narrow in this respect. It is also unquestionably held that the Convention protects not only the content of information but also the means of dissemination, since any restriction imposed on the means necessarily interferes with the right to receive and impart information (where under "means of dissemination", the technological means of transmission were understood, Autronic AG v. Switzerland, 1990; Öztürk v. Turkey, 1999; Ahmet Yildirim v. Turkey, 2012).

Political advertising is also highly protected (VgT v. Switzerland, 2001; Vest v. Norway, 2008), but not entirely limitless (Animal Defenders v. UK, 2013). Section 2 of this paper will discuss the latter decision in more detail.

Apparently, the protection of political expression is rock solid, as it should be in all democracies. And still, against this backdrop, I will argue that the method of political micro-targeting should be regulated, because, in my hypothesis, it violates the right to receive information. In the following paragraphs I will therefore explore the right to receive information in the practice of the ECtHR.

The right to receive information is the passive side of freedom of expression, as expressed both by Article 10 of ECHR, and Article 19 of the ICCPR. The text of Article 10 says: "This right shall include freedom to (...) receive and impart (...) information and ideas", whereas Article 19. ICCPR is somewhat more explicit: "Everyone shall have the right to (...) freedom to seek, receive and impart information and ideas". The ECHR lacks the word "seek", which was observed by the Court (Maxwell, 2017), however, the Court also noted that there was a "high degree of consensus" under international law that access to information is part of the right to freedom of expression, as shown by the relevant decisions of the UN Human Rights Committee regarding Article 19 of the ICCPR. 

The ECtHR practice shows a tendency of growing recognition of the right to receive information, as shown by Kenedi v. Hungary (2009), Társaság a Szabadságjogokért v. Hungary (2009), Helsinki v. Hungary (2016). This is a clear development from the Court's previous attitude where it denied that the right of access to information fell within the scope of Article 10 (Leander v. Sweden, 1987; Gaskin v. UK, 1989; Guerra v. Italy, 1998). For example, in Leander v. Sweden, the Court held that the right to freedom of expression in Article 10 did not confer a positive right to request information (Maxwell, 2017). But in a report issued by the Council of Europe - already before the cases of Kenedi v. Hungary and Társaság v. Hungary, the author emphasised that "The ambit of freedom of information thus has a tendency to expand: this freedom is particularly important in political or philosophical discussion, given its role in helping to determine people’s choices" (Renucci, 2005, p. 25).

In Társaság v. Hungary, the Court declared that itself “has recently advanced towards a broader interpretation of the notion of ‘freedom to receive information’ (see Matky v. la République tchèque, 2006) and thereby towards the recognition of a right of access to information.” In this case, the Hungarian non-governmental organisation (NGO) "Civil Liberties Union" asked for access to a petition submitted by a member of the parliament to the Constitutional Court, which questioned the constitutionality of newly passed amendments to the Criminal Code, related to drug abuse. The NGO which is active in the protection of human rights, and which had been working in the field of harm reduction of drug abuse, was specifically interested in that topic. The Constitutional Court denied access to the petition without the approval of the petitioner. The national courts both approved this decision, referring to the protection of personal data of the petitioner. ECtHR noted that this case was related to interference with the watchdog function – similar to that of the press, rather than the violation of the general right to access to public information. It added that the obligation of the State includes the elimination of obstacles which would hinder the press to exercise its function, if these obstacles exist solely because of the information monopoly of the authorities – as in this case the information was ready and available. Therefore, the Court established violation of Article 10 of ECHR, because such obstacles to prevent access to public information can discourage the media or similar actors to discuss such issues, and consequently they would be unable to fulfil their "public watchdog" role.

In Kenedi v. Hungary (2009), a historian claimed access to documents of the national security services of the communist regime, which were restricted by law. The Court emphasised that "access to original documentary sources for legitimate historical research was an essential element of the exercise of the applicant’s right to freedom of expression". In fact, the subject matter in the conflict reached beyond the historical information, because the restricted documents of the former communist secret service related to persons still actively working, and had the potential to stir substantial political controversy. Access to the information thus could contribute to a free political debate. The Hungarian courts judged in favour of the researcher Kenedi, but their decision was not executed by the government. This failure made the case so clearcut, that the Court did not go into particular detail in the argumentation section.

The mentioned cases were instances where the state's reluctance to reveal public information impaired the exercise of the functions of a public watchdog, like the press, or an NGO, which intended to contribute to a debate on a matter of public interest (Helsinki v. Hungary, 2016, para. 197, and Társaság v. Hungary, 2009, para. 28). The Court previously had expressed that preliminary obstacles created by the authorities in the way of press functions call for the most careful scrutiny (Chauvy and Others v. France, 2004 - cited in Társaság v. Hungary, 2009, para. 36.). The Court also considered that obstacles created in order to hinder access to information of public interest may discourage those working in the media or related fields from pursuing such matters (citing Goodwin v. the United Kingdom, 1996, para. 39 in Társaság v. Hungary, 2016, para. 38).

This argumentation could be mutatis mutandis relevant if access to political advertisements during or after an election campaign would be restricted only to targeted voters, as the scrutiny exercised by NGOs and journalists as well as election authorities over the election campaign is an inherent part of their watchdog role, to ensure and supervise the fairness of elections. In the mentioned cases, the obstacle in the way of access to information were created or maintained by governments or state bodies (such as the Constitutional Court, or police departments). However, in many other instances, the Court decided in favour of freedom to receive and impart information against the interests of private enterprises (Bladet Tromsø v. Norway, 1999; Sunday Times v. UK, 1979). In these and other cases, the Court emphasised that the right to access to information is not reserved to the press: the general public is also entitled to access public information (De Haes & Gijsels v. Belgium, 1997, para. 39; Fressoz & Roire v. France, 1999, para. 51).

In all the cited cases, the Court had to decide between a restriction of Article 10 for some legitimate interest. Freedom of information and freedom of expression were mutually completing each other, freedom of information being instrumental to freedom of expression. The applicants' right to receive information was violated which prevented them in exercising their right to freedom of expression.

Political micro-targeting represents a specific niche category among the political advertisements –– although there is some debate about what the term actually includes, and is relatively new to the European jurisprudence. Therefore, to this date, there has been no case at the ECtHR related to political micro-targeting. Nevertheless, it should be noted that the general public interest has always been an important factor in finding the balance between colliding rights, as an official Council of Europe report states: "In determining whether or not a positive obligation exists, regard must be had to the fair balance that has to be struck between the general interest of the community and the interests of the individual, the search for which is inherent throughout the Convention.”(CoE, 2005, p. 42). In addition, from the entirety of the case law of the ECtHR, it can also be deducted that in balancing the restriction of freedom expression the free public debate of matters of public interest has been considered with decisive weight (Sunday Times v. UK, 1979; Bladet Tromsø v. Norway, 1999).

Political advertising

The scenario in the case of political micro-targeting is somewhat different from the above cases, and more similar to the cases related to political advertising such as Animal Defenders v. UK (2013), and Erdogan Gökce v. Turkey (2014), or TV Vest v. Norway (2008). In Animal Defenders v. UK, an NGO was prohibited from running their public issue ad campaign on television, due to a legal prohibition of broadcasting political advertisements. In TV Vest v. Norway, the broadcasting company was fined for having broadcast the political advertisements of a small and powerless pensioners’ party despite the legal prohibition. InErdogan Gökce v. Turkey, the applicant, who distributed political campaign leaflets a year ahead of elections, was sentenced to three months of imprisonment.

In these cases, freedom of expression was limited by state intervention with the aim to protect democratic discourse, to ensure equal chances to all political candidates. Thus, access to specific political information was limited by the respective states in order to ensure the right of the general public to receive information in a fair and undistorted way.

Despite the similar factual background, the details and so the outcomes of the cases were different. In Erdogan v. Gökce, the prescribing law was less than clear, and its application had been inconsequential previously. These circumstances of the case set a clear case for a violation of Article 10 of ECHR.

Nevertheless, the Court in all cases assessed whether the applicant's right to communicate information and ideas of general interest - which the public has the right to receive, could be justified with the authorities' concern to safeguard the democratic debate and process, and to prevent it from being distorted during the electoral campaign by acts likely to hinder fair competition between candidates (Erdogan Gökce v. Turkey, 2014, para. 40, citing Animal Defenders, para. 112). In my view, this rationale offers a sound interpretation even of the Animal Defenders judgment, in which the Court found no violation of Article 10 of ECHR which was greeted with perplexity by many commentators (Ó Fathaigh, 2014; Lewis, 2014; Rowbottom, 2013b).

In all of these cases, paradoxically from the perspective of Article 10 of ECHR, freedom of speech was to be restricted with the objective to preserve a sound informational environment; because pluralism of views, and ultimately the democratic process would otherwise have been distorted by the speech in question.

I will below analyse in more detail the Court's reasonings related to political advertising, through the examples of these three landmark decisions (Animal Defenders v. UK, 2013; TV Vest v. Norway, 2008; and Erdogan Gökce v. Turkey, 2014). First, I would like to show that the considerations in TV Vest, which preceded Animal Defenders, have signalled the Court's position which was followed also in Animal Defenders, and therefore, in my view, the latter should not have been as surprising as it was widely regarded. In TV Vest, the Court has carefully considered the government's argumentation that the rationale for the general prohibition of broadcast political advertisements was that such type of expression was "likely to reduce the quality of political debate generally",so that "complex issues might easily be distorted and groups that were financially powerful would have greater opportunities for marketing their opinions than those that were not". Therein, "pluralism and quality were central considerations". The Court accepted this as a legitimate aim of the regulation, but held that the restriction did not qualify the expectations of proportionality, primarily because the applicant Pensioners Party was not a financially strong party, which would have been the targets of the prohibition, on the contrary: it "belonged to a category for whose protection the ban was, in principle, intended" (at 73).

In my interpretation, here the Court suggested that the law's effect is lifted for the sake of a party which was meant to be a beneficiary, rather than one to bear the burden of the prohibition. It was precisely this case-by-case distinction which was distinguished in Animal Defenders, where the Court declared that "the more convincing the general justifications for the general measure are, the less importance the Court will attach to its impact in the particular case". Moreover, the Court explained that "a prohibition requiring a case-by-case distinction(...) might not be a feasible means of achieving the legitimate aim" (para. 122). Thus, here the Court not only accepted the legitimate aim of the prohibition, but also accepted that no exception should be made even for an otherwise socially benign NGO campaign, because the case-by-case application "could lead to uncertainty, litigation, expense and delay as well as to allegations of discrimination and arbitrariness, these being reasons which can justify a general measure" (para. 122).

After examining the similarities of argumentation in the consecutive decisions, I would like to describe how the Court identified the decisive factors in the case of Animal Defenders.

In Animal Defenders, the Court held that the ban's rationale served the public interest: "the danger of unequal access based on wealth was considered to go to the heart of the democratic process" (para. 117); the restriction had strict limits as it was confined to certain media only, and a range of alternative media were available. The Court observed that it needed to balance between the NGO's right to impart information and ideas, which the public was entitled to receive, with the interest of the democratic process from distortion, by powerful financial groups which could obtain competitive advantages in the area of paid advertising and thereby curtail a free and pluralist debate (para. 112). At the same time, the Court acknowledged that both parties had the same objective: the maintenance of a free and pluralist debate on matters of public interest (see also Rowbottom, 2013a).

From this reasoning, we can conclude that political advertisements can be restricted with certain conditions:

• their dissemination would impose a risk of unequal access based on wealth;
• the legitimate aim is protection of the democratic process from distortion;
• the lurking distortion would cause competitive advantages and thereby curtail a free and pluralist debate;
• the restriction has strict limits by being confined to certain media only, and other media is available.

The dictum of Animal Defenders is, that under these conditions, the right of social organisations to impart information and ideas – which the public is otherwise entitled to receive – may be restricted.

Translating this into micro-targeted political advertising, we can recognise similarities: the means to apply this technology is not equally accessible to all political parties (or issue-groups) without regard to financial resources, and the concluding distortion of the public discourse might harm the democratic process, and curtail free and pluralist debate. Thus, we can conclude that, with narrowly curtailed rules, such political advertising can be restricted without violating Article 10 of ECHR.

Now, after having drawn an analogy between the decisions and political micro-targeting, two more questions are to be addressed.

First, both in Animal Defenders, TV Vest and in Erdogan Gökce v. Turkey, the applicant's right to freedom of expression was restricted by the state, and this restriction led the applicant to apply to the Court. Today, micro-targeted political advertisements are not restricted by state regulation. My conclusion implies that in the case this would happen, such restriction would not be against Article 10 of ECHR. As demonstrated in the analysis above, I interpret the Court's judgements that such a restriction would be regarded by the Court as serving a legitimate aim, as necessary in a democratic society, primarily to prevent the public political debate from distortion, and secondarily, on the basis of right-to-information case law (see above), to ensure public scrutiny by non-targeted users, including journalists and NGOs who may not otherwise have access to all the political advertisements. Its proportionality is naturally dependent on the nature of the specific regulation, but one of the main aspects would be confinement to a certain media type only, so that other means of publicity remain available.

Second, the fact that a state restriction would not be contrary to Article 10 of ECHR, does not mean that such a restriction is necessary to preserve the sound, democratic public discourse. The Court has declared in its previous decisions (Leander v. Sweden, 1987; Gaskin v. UK, 1989, para. 57; Guerra and Others v. Italy, 1998, para. 53; and Roche v. UK, 2005, para. 172) that Article 10 of ECHR does not grant an entitlement to demand that the state actively ensures access to all public information.

Thus, by saying that state restriction of political micro-targeting would be acceptable from the perspective of human rights, I did not yet prove that it is also desirable. In the following section I will discuss the effect of micro-targeting on the democratic process, and its further consequences.

The risks of political micro-targeting to democracy

Evidence shows that political micro-targeting can increase polarisation and fragmentation of the public sphere. For example, in the US presidential campaign of 2016, Facebook posted ‘dark posts’ (sponsored Facebook posts that can only be seen by users with very specific profiles) to micro-target groups of voters with “40-50,000 variants of ads every day” (Illing, 2017), among other reasons, to discourage voters (see also Chester & Montgomery, 2017). In a negative scenario, when political parties share only those fragments of their political programmes with the targeted voters who such programmes would likely support, and other fragments with yet another part of the audience, that can be harmful for democratic processes (Zuiderveen Borgesius et al., 2018; see also Howard, 2006, pp. 135-136.). Beyond being an unfair practice, this splinters the shared information basis of society and contributes to a fractured public sphere.

However, technology in itself is neither evil, nor good. The strong potential of micro-targeting could also serve the public interest, if applied purposefully for that end. There is ample research and discussion on how social media engagement enhances democracy (Sunstein, 2007; Sunstein, 2018; Kumar & Kodila-Tedika, 2019; Martens et al., 2018). For example, it could be exceptionally effective in transmitting useful messages to citizens on healthy living, safe driving, and other social values with which it can greatly benefit society. In this perspective, data-driven political micro-targeting has the potential to increase the level of political literacy and the functioning of deliberative democracy, by incentivising deliberative discussion among those voters who are interested and who feel involved. However, even in this case, the non-targeted citizens are excluded from the discussion, without having been offered the choice to participate in it, unless effective measures are used to enable their involvement. At the core of the issue is the paternalistic distinction between citizens, deciding on their behalf whether they should get certain information or not (see also Thaler & Sunstein, 2008).

I would like to emphasise that the process of democracy needs to be protected with utmost care. Why? We are witnessing a success of populist political campaigns globally, and we should accept the fact that social representation of ideologies is diverse, and may change over time. It is even possible that popular support for the idea of deliberative democracy will decrease, as democratic processes have signalled in several countries, even within the European Union. But at the same time, there is (still) a global consensus on the universal protection of fundamental rights, democratic processes and the rule of law, which form the fundamental legal structures of our societies. Therefore, while political communication should not be restricted on the basis of its content, even ideologies which are critical of the current forms of democracy (e.g., illiberalism) should be allowed to compete in the political arena, but it must be secured that all democratic discussion and political battles in which these ideologies wrestle are played under fair circumstances. Only this can ensure that the freedom of political expression is not abused and the democratic process is not hacked by political opportunism. Political campaigning is one of the key processes by which the formation of the democratic will of the people is generated, and should this process violate fundamental rights, that would in itself pose a threat to democracy. It would destabilise the democratic process and raise issues of legitimacy, whether or not the controversial technique is successful. Respect for fundamental rights is a prerequisite for the rule of law, and the rule of law is a precondition for democracy. The three are “inherently and indivisibly interconnected, and interdependent on each of the others, and they cannot be separated without inflicting profound damage to the whole and changing its essential shape and configuration”(Carrera, Elspeth, & Hernanz, 2013).

In sum, I argue that online micro-targeting should be restricted not because it can carry manipulative content, and not because it can violate privacy rights, but because it threatens the process of democratic discourse. Even if the likelihood of manipulation is small, the harm that can be caused is so severe that the overall sum of the risk is too high to be taken. Direct political campaigning has been with us before, in the form of door-to-door canvassing, leaflets, local meetings, and other tools. However, access to masses of voters’ personal data, the analysis of these databases with advanced technology, and the low cost of personalised communication generate a qualitatively new situation. The voters have lost their control over being targeted, and the transparency of the targeting has diminished (see also Mayer-Schönberger & Cukier, 2013; Baldwin-Philippi, 2019; Howard, 2006).

Having argued above that the technique of micro-targeting is harmful to the individual right to information, and that it threatens the collective democratic public discourse, the logical conclusion would be to recommend a complete prohibition of using this strategic communication tool in the political discourse. Only this could eliminate the risk to the informational rights of masses of voters and to the further polarisation of the public discourse.

However, considering also the benefits and the high interest of the political elite in this tool, the political reality is likely to incline towards allowing its use and demanding appropriate safeguards – the discussion of which is beyond the limits of this article.

Conclusion

This article argued that micro-targeting violates the fundamental right to receive information, and the collective right to the public discourse. Thereby it harms the democratic process of deliberation. Non-targeted voters' right to receive information is violated by being excluded from political communication that is supposed to be public and inclusive in a democracy. This is a mass violation of a human right, which is part of the right to freedom of expression, as recognised by the ECtHR. To focus entirely on the informational rights of non-targeted citizens, the article avoided the discussion of other rights that may be affected.

I examined two aspects of the ECtHR jurisprudence in freedom of expression: the right to receive information, and freedom of political expression. In the first topic, I showed that in many instances, the Court decided in favour of freedom to receive and impart information for the sake of the public discourse, even against the interests of private enterprises (Bladet Tromsø v. Norway, 1999; Sunday Times v. UK, 1979). I demonstrated that the right to receive information is not reserved to the press, but it includes the general public as well (De Haes & Gijsels v. Belgium, 1997, para. 39; Fressoz & Roire v. France, 1999, para. 51). In our context, the right to receive all political information is regarded as crucial for non-targeted voters, including journalists, NGOs and election authorities. Although the above cases relate to restrictions caused by the state rather than private entities, the Court found that the state is obliged to eliminate obstacles which would hinder the press to exercise its watchdog function (Társaság v. Hungary, 2009). Whenever the Court had to balance between the public interest of the community and the interest of an individual, the public interest has been considered with substantial weight (CoE, 2005, p. 42; Sunday Times v. UK, 1979, Bladet Tromsø v. Norway, 1999).

In my analysis of the ECtHR decisions relating to political advertising, I show that the Court had consistently found acceptable restrictions of political advertising in the interest of the sound democratic public discourse. I argue that even though the Animal Defenders v. UK (2013) decision was regarded as exceptional then, but both preceding and following judgments clearly show the consistency of the Court’s position. In all discussed cases, the Court assessed the right to political expression and to receive information versus the protection of the public discourse, where the latter was considered as the authorities' responsibility to prevent the democratic debate from being distorted (Erdogan Gökce v. Turkey, 2014; Animal Defenders v. UK, 2013, para. 112). In the prior TV Vest v. Norway case (2008), it was shown that the Court accepted the principle that a certain type of political speech threatened with reducing the quality of the political debate, and causing distortion of the discussion, as well as inequality between the financially powerful and less well-financed groups, even though in the specific case the Court found the restriction unproportionate, because the party in question was a small and financially weak party (TVVest v. Norway, 2008). In the case of Animal Defenders, the Court found acceptable the restriction of the dissemination of public issue ads with the objective to preserve a sound informational environment. The factors which the Court identified so as to determine the proportionality of a restriction can be guiding for the case of political micro-targeting as well: to prevent the risk of unequal access to the public discourse based on wealth, and consequently the protection of the democratic process from distortion, which would curtail a free and puralist debate. In Animal Defenders the restriction was found to be sufficiently narrowly tailored, as it applied to certain media only, and other media remained available. These arguments also apply to the case of online political micro-targeting, which is suspected to fragment and distort the democratic process through the informational deprivation caused to non-targeted voters.

While discussing policy options is beyond the constraints of the article, to conclude, here are a few thoughts to consider.

First, some countries' legal culture may incline towards more risk-taking, even at the price of certain collective and individual rights being harmed, whereas others are more risk-averse. Similar to the dispute over hate speech, the former culture would rather tolerate the risk to the political process, than restrict individual freedom of expression. Long-standing, stable, and prosperous democracies may find the explained risks more manageable – this would be more characteristic to the United States than to most member states of the European Union (Heinze, 2005; Kahn, 2013).

Second, staying with the example of hate speech, while there are important differences between the European member states in regulating hate speech, the similarities are more characteristic, especially in contrast to the United States. Importantly, the ECtHR held that the margin of appreciation is narrow in the field of political freedom of expression. Legislative efforts also have to face the difficulties of the transborder nature of targeting and advertising (Bodó, Helberger, & de Vreese, 2017). All these factors highlight the importance of an international, but at least EU-wide policy approach (see also Dobber et al., 2019).

Third, when it comes to the protection of fundamental rights, states have an obligation to ensure that these rights are not restricted even by private entities. Self- and co-regulation does not impose sanctions in case of non-compliance, therefore they do not provide sufficient protection for individual human rights. For political advertisers, the stakes are higher than in any other industry, and these circumstances render the long-term success of self-regulation less likely. Specifically, in the case of political micro-targeting, the data controllers are political parties that had, have or are going to have governmental power, and thus have a potential influence on national regulations and on authorities as well. This further raises the significance of supranational regulation and supervision by EU bodies.

Introduction

Did the Holocaust really happen? In December 2016, Google’s search engine algorithm determined the most authoritative source to answer this question was a neo-Nazi website peddling holocaust denialism (Cadwalladr, 2016b). For any inquisitive user typing this question into Google, the first website recommended by Search linked to an article entitled: “Top 10 reasons why the Holocaust didn’t happen”. The third article “The Holocaust Hoax; IT NEVER HAPPENED” was published by another neo-Nazi website, while the fifth, seventh, and ninth recommendations linked to similar racist propaganda pages (Cadwalladr, 2016b). Up until Google started demoting websites committed to spreading anti-Semitic messages, anyone asking whether the Holocaust actually happened would have been directed to consult neo-Nazi websites, rather than one of the many credible sources about the Holocaust and tragedy of World War II.

Google’s role in shaping the information environment and enabling political advertising has made it a “de facto infrastructure” for democratic processes (Barrett & Kreiss, 2019). How its search engine algorithm determines authoritative sources directly shapes the online information environment for more than 89 percent of the world’s internet users who trust Google Search to quickly and accurately find answers to their questions. Unlike social media platforms that tailor content based on “algorithmically curated newsfeeds” (Golebiewski & boyd, 2019), the logic of search engines is “mutually shaped” by algorithms — that shape access — and users — who shape the information being sought (Schroeder, 2014). By facilitating information access and discovery, search engines hold a unique position in the information ecosystem. But, like other digital platforms, the digital affordances of Google Search have proved to be fertile ground for media manipulation.

Previous research has demonstrated how large volumes of mis- and disinformation were spread on social media platforms in the lead up to elections around the world (Hedman et al., 2018; Howard, Kollanyi, Bradshaw, & Neudert, 2017; Machado et al., 2018). Some of this disinformation was micro-targeted towards specific communities or individuals based on their personal data. While data-driven campaigning has become a powerful tool for political parties to mobilise and fundraise (Fowler et al., 2019; Baldwin-Philippi, 2017), the connection between online advertisements and disinformation, foreign election interference, polarisation, and non-transparent campaign practices has caused growing anxieties about its impact on democracy.

Since the 2016 presidential election in the United States, public attention and scrutiny has largely focused on the role of Facebook in profiting from and amplifying the spread of disinformation via digital advertisements. However, less attention has been paid to Google, who, along with Facebook, commands more than 60% of the digital advertising market share. At the same time, a multi-billion-dollar search engine optimisation (SEO) industry has been built around understanding how technical systems rank, sort, and prioritise information (Hoffmann, Taylor, & Bradshaw, 2019). The purveyors of disinformation have learned to exploit social media platforms to engineer content discovery and drive “pseudo-organic engagement”. 1 These websites — that do not employ professional journalistic standards, report on conspiracy theory, counterfeit professional news brands, and mask partisan commentary as news — have been referred to as “junk news” domains (Bradshaw, Howard, Kollanyi, & Neudert, 2019).

Together, the role of political advertising and the matured SEO industry make Google Search an interesting and largely underexplored case to analyse. Considering the importance of Google Search in connecting individuals to news and information about politics, this paper examines how junk news websites generate discoverability via Google Search. It asks: (1) How do junk news domains optimise content, through both paid and SEO strategies, to grow discoverability and grow their website value? (2) What strategies are effective at growing discoverability and/or growing website value; and (3) What are the implications of these findings for ongoing discussions about the regulation of social media platforms?

To answer these questions, I analysed 29 junk news domains and their advertising and search engine optimisation strategies between January 2016 and March 2019. First, junk news domains make use of a variety of SEO keyword strategies in order to game Search and grow pseudo-organic clicks and grow their website value. The keywords that generated the highest placements on Google Search focused on (1) navigational searches for known brand names (such as searches for “breitbart.com”) and (2) carefully curated keyword combinations that fill so-called “data voids” (Golebiewski & Boyd, 2018), or a gap in search engine queries (such as searches for “Obama illegal alien”). Second, there was a clear correlation between the number of clicks that a website receives and the estimated value of the junk news domains. The most profitable timeframes correlated with important political events in the United States (such as the 2016 presidential election, and the 2018 midterm elections), and the value of the domain increased based on SEO optimised — rather than paid — clicks. Third, junk news domains were relatively successful at generating top-placements on Google Search before and after the 2016 US presidential election. However, their discoverability abruptly declined beginning in August 2017 following major announcements from Google about changes to its search engine algorithms, as well as other initiatives to combat the spread of junk news in search results. This suggests that Google can, and has, measurably impacted the discoverability of junk news on Search.

This paper proceeds as follows: The first section provides background on the vocabulary of disinformation and ongoing debates about so-called fake news, situating the terminology of “junk news” used in this paper in the scholarly literature. The second section discusses the logic and politics of search, describing how search engines work and reviewing the existing literature on Google Search and the spread of disinformation. The third section outlines the methodology of the paper. The fourth section analyses 29 prominent junk news domains to learn about their SEO and advertising strategies, as well as their impact on content discoverability and revenue generation. This paper concludes with a discussion of the findings and implications for future policymaking and private self-regulation.

The vocabulary of political communication in the 21^st century

“Fake news” gained significant attention from scholarship and mainstream media during the 2016 presidential election in the United States as viral stories pushing outrageous headlines — such as Hillary Clinton’s alleged involvement in a paedophile ring in the basement of a DC pizzeria — were prominently displayed across search and social media news feeds (Silverman, 2016). Although “fake news” is not a new phenomenon, the spread of these stories—which are both enhanced and constrained by the unique affordances of internet and social networking technologies — has reinvigorated an entire research agenda around digital news consumption and democratic outcomes. Scholars from diverse disciplinary backgrounds — including psychology, sociology and ethnography, economics, political science, law, computer science, journalism, and communication studies — have launched investigations into circulation of so-called “fake news” stories (Allcott & Gentzkow, 2017; Lazer et al., 2018), their role in agenda-setting (Guo & Vargo, 2018; Vargo, Guo, & Amazeen, 2018), and their impact on democratic outcomes and political polarisation (Persily, 2017; Tucker et al., 2018).

However, scholars at the forefront of this research agenda have continually identified several epistemological and methodological challenges around the study of so-called “fake news”. A commonly identified concern is the ambiguity of the term itself, as “fake news” has come to be an umbrella term for all kinds of problematic content online, including political satire, fabrication, manipulation, propaganda, and advertising (Tandoc, Lim, & Ling, 2018; Wardle, 2017). The European High-Level Expert Group on Fake News and Disinformation recently acknowledged the definitional difficulties around the term, recognising it “encompasses a spectrum of information types…includ[ing] low risk forms such as honest mistakes made by reporters…to high risk forms such as foreign states or domestic groups that would try to undermine the political process” (European Commission, 2018). And even when the term “fake news” is simply used to describe news and information that is factually inaccurate, the binary distinction between what is true and what is false has been criticised for not adequately capturing the complexity of the kinds of information being shared and consumed in today’s digital media environment (Wardle & Derakhshan, 2017).

Beyond the ambiguities surrounding the vocabulary of “fake news”, there is growing concern that the term has begun to be appropriated by politicians to restrict freedom of the press. A wide range of political actors have used the term “fake news” to discredit, attack, and delegitimise political opponents and mainstream media (Farkas & Schou, 2018). Certainly, Donald Trump’s (in)famous use of the term “fake news”, is often used to “deflect” criticism and to erode the credibility of established media and journalist organisations (Lakoff, 2018). And many authoritarian regimes have followed suit, adopting the term into a common lexicon to legitimise further censorship and restrictions on media within their own borders (Bradshaw, Neudert, & Howard, 2018). Given that most citizens perceive “fake news” to define “partisan debate and poor journalism”, rather than a discursive tool to undermine trust and legitimacy in media institutions, there is general scholarly consensus that the term is highly problematic (Nielsen & Graves, 2017).

Rather than chasing a definition of what has come to be known as “fake news”, researchers at the Oxford Internet Institute have produced a grounded typology of what users actually share on social media (Bradshaw et al., 2019). Drawing on Twitter and Facebook data from elections in Europe and North America, researchers developed a grounded typology of online political communication (Bradshaw et al., 2019; Neudert, Howard, & Kollanyi, 2019). They identified a growing prevalence of “junk news” domains, which publish a variety of hyper-partisan, conspiracy theory or click-bait content that was designed to look like real news about politics. During the 2016 presidential election in the United States, social media users on Twitter shared as much “junk news” as professionally produced news about politics (Howard, Bolsover, Kollanyi, Bradshaw, & Neudert, 2017; Howard, Kollanyi, et al., 2017). And voters in swing-states tended to share more junk news than their counterparts in uncontested ones (Howard, Kollanyi, et al., 2017). In countries throughout Europe — in France, Germany, the United Kingdom and Sweden — junk news inflamed political debates around immigration and amplified populist voices across the continent (Desiguad, Howard, Kollanyi, & Bradshaw, 2017; Kaminska, Galacher, Kollanyi, Yasseri, & Howard, 2017; Neudert, Howard, & Kollanyi, 2017).

According to researchers on the Computational Propaganda Project junk news is defined as having at least three out of five elements: (1) professionalism, where sources do not employ the standards and best practices of professional journalism including information about real authors, editors, and owners (2) style, where emotionally driven language, ad hominem attacks, mobilising memes and misleading headlines are used; (3) credibility, where sources rely on false information or conspiracy theories, and do not post corrections; (4) bias, where sources are highly biased, ideologically skewed and publish opinion pieces as news; and (5) counterfeit, where sources mimic established news reporting including fonts, branding and content strategies (Bradshaw et al., 2019).

In a complex ecosystem of political news and information, junk news provides a useful point of analysis because rather than focusing on individual stories that may contain honest mistakes, it examines the domain as a whole and looks for various elements of deception, which underscores the definition of disinformation. The concept of junk news is also not tied to a particular producer of disinformation, such as foreign operatives, hyper-partisan media, or hate groups, who, despite their diverse goals, deploy the same strategies to generate discoverability. Given that the literature on disinformation is often siloed around one particular actor, does not cross platforms, nor integrate a variety of media sources (Tucker et al., 2018), the junk news framework can be useful for taking a broader look at the ecosystem as a whole and the digital techniques producers use to game search engine algorithms. Throughout this paper, I use the term “junk news” to describe the wide range of politically and economically motivated disinformation being shared about politics.

The logic and politics of search

Search engines play a fundamental role in the modern information environment by sorting, organising, and making visible content on the internet. Before the search engine, anyone who wished to find content online would have to navigate “cluttered portals, garish ads and spam galore” (Pasquale, 2015). This didn’t matter in the early days of the web when it remained small and easy to navigate. During this time, web directories were built and maintained by humans who often categorised pages according to their characteristics (Metaxas, 2010). By the mid-1990s it became clear that the human classification system would not be able to scale. The search engine “brought order to chaos by offering a clean and seamless interface to deliver content to users” (Hoffman, Taylor, & Bradshaw, 2019).

Simplistically speaking, search engines work by crawling the web to gather information about online webpages. Data about the words on a webpage, links, images, videos, or the pages they link to are organised into an index by an algorithm, analogous to an index found at the end of a book. When a user types a query into Google Search, machine learning algorithms apply complex statistical models in order to deliver the most “relevant” and “important” information to a user (Gillespie, 2012). These models are based on a combination of “signals” including the words used in a specific query, the relevance and usability of webpages, the expertise of sources, and other information about context, such as a user’s geographic location and settings (Google, 2019).

Google’s search rankings are also influenced by AdWords, which allow individuals or companies to promote their websites by purchasing “paid placement” for specific keyword searches. Paid placement is conducted through a bidding system, where rankings and the number of times the advertisement is displayed are prioritised by the amount of money spent by the advertiser. For example, a company that sells jeans might purchase AdWords for keywords such as “jeans”, “pants”, or “trousers”, so when an individual queries Google using these terms, a “sponsored post” will be placed at the top of the search results. 2 AdWords also make use of personalisation, which allow advertisers to target more granular audiences based on factors such as age, gender, and location. Thus, a local company selling jeans for women can specify local female audiences — individuals who are more likely to purchase their products.

The way in which Google structures, organizes, and presents information and advertisements to users is important because these technical and policy decisions embed a wide range of political issues (Granka, 2010; Introna & Nissenbaum, 2000; Vaidhynathan, 2011). Several public and academic investigations auditing Google’s algorithms have documented various examples of bias in Search or problems with the autocomplete function (Cadwalladr, 2016a; Pasquale, 2015). Biases inherently designed into algorithms have been shown to disproportionately marginalise minority communities, women, and the poor (Noble, 2018).

At the same time, political advertisements have become a contentious political issue. While digital advertising can generate significant benefits for democracy, by democratising political finance and assisting in political mobilisation (Fowler et al., 2019; Baldwin-Philippi, 2017), it can also be used to selectively spread disinformation and messages of demobilisation (Burkell & Regan, 2019; Evangelista & Bruno, 2019; Howard, Ganesh, Liotsiou, Kelly, & Francois, 2018). Indeed, Russian AdWord purchases in the lead-up to the 2016 US election demonstrate how foreign states actors can exploit Google Search to spread propaganda (Mueller, 2019). But the general lack of regulation around political advertising has also raised concerns about domestic actors and the ways in which legitimate politicians campaign in increasingly opaque and unaccountable ways (Chester & Montgomery, 2017; Tufekci, 2014). These concerns are underscored by the rise of the “influence industry” and the commercialisation of political technologies who sell various ‘psychographic profiling’ technologies to craft, target, and tailor messages of persuasion and demobilisation (Chester & Montgomery, 2019; McKelvey, 2019; Bashyakarla, 2019). For example, during the 2016 US election, Cambridge Analytica worked with the Trump campaign to implement “persuasion search advertising”, where AdWords were bought to strategically push pro-Trump and anti-Clinton information to voters (Lewis & Hilder, 2018).

Given growing concerns over the spread of disinformation online, scholars are beginning to study the ways in which Google Search might amplify junk news and disinformation. One study by Metaxa-Kakavouli and Torres-Echeverry examined the top ten results from Google searches about congressional candidates over a 26-week period in the lead-up to the 2016 presidential election. Of the URLs recommended by Google, only 1.5% came from domains that were flagged by PolitiFact as being “fake news” domains (2017). Metaxa-Kakavouli and Torres-Echeverry suggest that the low levels of “fake news” are the result of Google’s “long history” combatting spammers on its platform (2017). Another research paper by Golebiewski and boyd looks at how gaps in search engine results lead to strategic “data voids” that optimisers exploit to amplify their content (2018). Golebiewski and boyd argue that there are many search terms where data is “limited, non-existent or deeply problematic” (2018). Although these searches are rare, if a user types these search terms into a search engine, “it might not give a user what they are looking for because of limited data and/or limited lessons learned through previous searches” (Golebiewski & boyd, 2018).

The existence of biases, disinformation, or gaps in authoritative information on Google Search matters because Google directly impacts what people consume as news and information. Most of the time, people do not look past the top ten results returned by the search engine (Metaxas, 2010). Indeed, eye-tracking experiments have demonstrated that the order in which Google results are presented to users matters more than the actual relevance of the page abstracts (Pan et al., 2007). However, it is important to note that the logic of higher placements does not necessarily translate to search engine advertising listings, where users are less likely to click on advertisements if they are familiar with the brand or product they are searching for (Narayanan & Kalyanam, 2015).

Nevertheless, the significance of the top ten placement has given rise to the SEO industry, whereby optimisers use digital keyword strategies to move webpages higher in Google’s rankings and thereby generate higher traffic flows. There is a long history of SEO dating back to the 1990s when the first search engine algorithms emerged (Metaxas, 2010). Since then, hundreds of SEO pages have published guesses about the different ranking factors these algorithms consider (Dean, 2019). However, the specific signals that inform Google’s search engine algorithms are dynamic and constantly adapting to the information environment. Google makes hundreds of changes to its algorithm every year to adjust the weight and importance of various signals. While most of these changes are minor updates designed to improve the speed and performance of Search, sometimes Google makes more significant changes to its algorithm to elude optimisers trying to game the system.

Google has taken several steps to combat people seeking to manipulate Search for political or economic gain (Taylor, Walsh, & Bradshaw, 2019). This involves several algorithmic changes to demote sources of disinformation as well as changes to their advertising policies to limit the extent to which users can be micro-targeted with political advertisements. In one study, researchers interviewed SEO strategists to audit how Facebook and Google’s algorithmic changes impacted their optimisation strategies (Hoffmann, Taylor, & Bradshaw, 2019). Since the purveyors of disinformation often rely on the same digital marketing strategies used by legitimate political candidates, news organisations, and businesses, the SEO industry can offer unique, but heuristic, insight into the impact of algorithmic changes. Hoffmann, Taylor and Bradshaw (2019) found that despite more than 125 announcements over a three-year period, the algorithmic changes made by the platforms did not significantly alter digital marketing strategies.

This paper hopes to contribute to the growing body of work examining the effect of Search on the spread of disinformation and junk news by empirically analysing the strategies — paid and optimised — employed by junk news domains. By performing an audit of the keywords junk news websites use to generate discoverability, this paper evaluates the effectiveness of Google in combatting the spread of disinformation on Search.

Methodology

Conceptual Framework: The Techno-Commercial Infrastructure of Junk News

The starting place for this inquiry into the SEO infrastructure of junk news domains is grounded conceptually in the field of science and technology studies (STS), which provides a rich literature on how infrastructure design, implementation, and use embeds politics (Winner, 1980). Digital infrastructure — such as physical hardware, cables, virtual protocols, and code — operate invisibly in the background, which can make it difficult to trace the politics embedded in technical coding and design (Star & Ruhleder, 1994). As a result, calls to study internet infrastructure has engendered digital research methods that shed light on the less-visible areas of technology. One growing and relevant body of research has focused on the infrastructure of social media platforms and the algorithms and advertising infrastructure that invisibly operate to amplify or spread junk news to users, or to micro-target political advertisements (Kim et al., 2018; Tambini, Anstead, & Magalhães, 2017). Certainly, the affordances of technology — both real and imagined — mutually shape social media algorithms and their potential for manipulation (Nagy & Neff, 2015; Neff & Nagy, 2016). However, the proprietary nature of platform architecture has made it difficult to operationalise studies in this field. Because junk news domains operate in a digital ecosystem built on search engine optimisation, page ranks, and advertising, there is an opportunity to analyse the infrastructure that supports the discoverability of junk news content, which could provide insights into how producers reach audiences, grow visibility, and generate domain value.

Junk news data set

The first step of my methodology involved identifying a list of junk news domains to analyse. I used the Computational Propaganda Project’s (COMPROP) data set on junk news domains in order to analyse websites that spread disinformation about politics. To develop this list, researchers on the COMPROP project built a typology of junk news based on URLs shared on Twitter and Facebook relating to the 2016 US presidential election, the 2017 US State of the Union Address, and 2018 US midterm elections. 3 A team of five rigorously trained coders labelled the domains contained in tweets and on Facebook pages based on a grounded typology of junk news that has been tested and refined over several elections around the world between 2016 and 2018. 4 A domain was labelled as junk news when it failed on three of the five criteria of the typology (style, bias, credibility, professionalism, and counterfeit, as described in section one). For this analysis, I used the most recent 2018 midterm election junk news list, which is comprised of the top-29 most shared domains that were labelled as junk news by researchers. This list was selected because all 29 domains were active during the 2016 US presidential election in November 2016 and the 2017 US State of the Union Address, which provides an opportunity to comparatively assess how both the advertising and optimisation strategies, as well as their performance, changed overtime.

SpyFu data and API queries

The second step of my methodology involved collecting data about the advertising and optimisation strategies used by junk news websites. I worked with SpyFu, a competitive keyword research tool used by digital marketers to increase website traffic and improve keyword rankings on Google (SpyFu, 2019). SpyFu collects, analyses and tracks various data about the search optimisation strategies used by websites, such as organic ranks, paid keywords bought on Google AdWords, and advertisement trends.

To shed light onto the optimisation strategies used by junk news domains on Google, SpyFu provided me with: (1) a list of historical keywords and keyword combinations used by the top-29 junk news that led to the domain appearing in Google Search results; and (2) the position the domain appeared in Google as a result of the keywords. The historical keywords were provided from January 2016 until March 2019. Only keywords that led to the junk news domains appearing in the top-50 positions on Google were included in the data set.

In order to determine the effectiveness of the optimisation and advertising strategies used by junk news domains to either grow their website value and/or successfully appear in the top positions on Google Search, I wrote a simple python script to connect to the SpyFu API service. This python script collected and parsed the following data from SpyFu for each of the top-29 junk news domains in the sample: (1) the number of keywords that show up organically on Google searches; (2) the estimated sum of clicks a domain receives based on factors including organic keywords, the rank of keyword, and the search volume of the keyword; (3) the estimated organic value of a domain based on factors including organic keywords, the rank of keywords, and the search volume of the keyword; (4) the number of paid advertisements a domain purchased through Google AdWords; and (5) the number of paid clicks a domain received from the advertisements it purchased from Google AdWords.

Data and methodology limitations

There are several data and methodology limitations that must be noted. First, the junk news domains identified by the Computational Propaganda Project highlights only a small sample of the wide variety of websites that peddle disinformation about politics. The researchers also do not differentiate between the different actors behind the junk news websites — such as foreign states or hyper-partisan media — nor do they differentiate between the political leaning of the junk news outlet — such as left-or-right-leaning domains. Thus, the outcomes of these findings cannot be described in terms of the strategies of different actors. Further, given that the majority of junk news domains in the top-29 sample lean politically to the right and far right, these findings might not be applicable to the hyper-partisan left and their optimisation strategies. Finally, the junk news domains identified in the sample were shared on social media in the lead-up to important political events in the United States. A further research question could examine the SEO strategies of domains operating in other country contexts.

When it comes to working with the data provided by SpyFu (and other SEO optimisation tools), there are two limitations that should be noted. First, the historical keywords collected by SpyFu are only collected when they appear in the top-50 Google Search results. This is an important limitation to note because news and information producers are constantly adapting keywords based on the content they are creating. Keywords may be modified by the source website dynamically to match news trends. Low performing keywords might be changed or altered in order to make content more visible via Search. Thus, the SpyFu data might not capture all of the keywords used by junk news domains. However, the collection strategy will have captured many of the most popular keywords used by junk news domains to get their content appearing in Google Search. Second, because SpyFu is a company there are proprietary factors that go into measuring a domain’s SEO performance (in particular, the data points collected via the API on the estimated sum of clicks and the estimated organic value). Nevertheless, considering that Google Search is a prominent avenue for news and information discovery, and that few studies have systematically analysed the effect of search engine optimisation strategies on the spread of disinformation, this study provides an interesting starting point for future research questions about the impact SEO can have on the spread and monetisation of disinformation via Search.

Analysis: optimizing disinformation through keywords and advertising

Junk news advertising strategies on Google

Junk news domains rarely advertise on Google. Only two out of the 29 junk news domains (infowars.com and cnsnews.com) purchased Google advertisements (See Figure 1: Advertisements purchased vs. paid clicks). The advertisements purchased by infowars.com were all made prior to the 2016 election in the United States (from the period of May 2015 to March 2016). cnsnews.com made several advertisement purchases over the three-year time period.

Figure 1: Advertisements purchased vs. paid clicks received: inforwars.com and cnsnews.com (May 2015-March 2019)

Looking at the total number of paid clicks received, junk news domains generated only a small amount of traffic using paid advertisements. Infowars on average, received about 2000 clicks as a result of their paid advertisements. cnsnews.com peaked at approximately 1800 clicks, but on average generated only about 600 clicks per month over the course of three years. By comparing the number of clicks that are paid versus those that were generated as a result of SEO keyword optimisation, there is a significant difference. During the same time period, cnsnews.com and infowars.com were generating on average 146,000 and 964,000 organic clicks respectively (See Figure 2:Organic vs. paid clicks (cnsnews.com and infowars.com)). Although it is hard to make generalisations about how junk news websites advertise on Google based on a sample of two, the lack of data suggests that advertising on Google Search might not be as popular as advertising on other social media platforms. Second, the return on investment (i.e., paid clicks generated as a result of Google advertisements) was very low compared to the organic clicks these junk news domains received for free. Factors other than advertising seem to drive the discoverability of junk news on Google Search.

Figure 2: organic vs. paid clicks (cnsnews.com and infowars.com)

Junk news keyword optimisation strategies

In order to assess the keyword optimisation strategies used by junk news websites, I worked with SpyFu, which provided historical keyword data for the 29 junk news domains, when those keywords made it to the top-50 results in Google between January 2016 and March 2019. In total, there were 88,662 unique keywords in the data set. Given the importance of placement on Google, I looked specifically at keywords that indexed junk news websites on the first — and most authoritative — position. Junk news domains had different aptitudes for generating placement in the first position (See Table 1: Junk news domains and number of keywords found in the first position on Google). Breitbart, DailyCaller and ZeroHedge had the most successful SEO strategies, respectively having 1006, 957 and 807 keywords lead to top placements on Google Search over the 39-month period. In contrast, six domains (committedconservative.com, davidharrisjr.com, reverbpress.news, thedailydigest.org, thefederalist.com, thepoliticalinsider.com) had no keywords reach the first position on Google. The remaining 20 domains had anywhere between 1 to 253 keywords place between the 2-10 positions on Google Search over the same timeframe.

Different keywords also generate different kinds of placement over the 39-month period. Table 2 (see Appendix) provides a sample list of up to ten keywords from each junk news domain in the sample when the keyword reached the first position.

First, many junk news domains appear in the first position on Google Search as a result of “navigational searches” whereby a user entered a query with the intent of finding a website. A search for a specific brand of junk news could happen naturally for many users, since the Google Search function is built into the address bar in Chrome, and sometimes set as the default search engine for other browsers. In particular, terms like “infowars” “breitbart” “cnsnews” and “rawstory” were navigational keywords users typed into Google Search. The performance of brand searches over time consistently places junk news webpages in the number one position (see Figure 3: Brand-related keywords over time). This suggests that brand-recognition plays an important role for driving traffic to junk news domains.

Figure 3: the performance of brand-related keywords overtime: top-5 junk news websites (January 2016-March 2019)

There is one outlier in this analysis, where keyword searches for “breitbart” drops to position two: in January 2017 and September 2017. This drop could have been a result of mainstream media coverage of Steve Bannon assuming (and eventually leaving) his position as the White House Chief Strategist during those respective months. The fact that navigational searches are one of the main drivers behind generating a top ten placement on Search suggests that junk news websites rely heavily on developing a recognisable brand and a dedicated readership that actively seeks out content from these websites. However, this also demonstrates that a complicated set of factors go into determining what keywords from what websites make the top placement in Google Search, and that coverage of news events from mainstream professional news outlets can alter the discoverability of junk news via Search.

Second, many keywords that made it to the top position in Google Search results are what Golebiewski and boyd (2018) would call terms that filled “data voids”, or gaps in search engine queries where there is limited authoritative information about a particular issue. These keywords tended to focus on conspiratorial information especially around President Barack Obama (“Obama homosexual” or “stop Barack Obama”), gun rights (“gun control myths”), pro-life narratives (“anti-abortion quotes” or “fetus after abortion”), and xenophobic or racist content (“against Islam” or “Mexicans suck”). Unlike brand-related keywords, problematic search terms do not achieve a consistently high placement on Google Search over the 39-week period. Keywords that ranked in number one for more than 30-weeks include: “vz58 vs. ak47”, “feminizing uranium”, “successful people with down syndrome”, “google ddrive”, and “westboro[sic] Baptist church tires slashed”. This suggests that, for the most part, data voids are either being filled by more authoritative sources, or Google Search has been able to demote websites attempting to generate pseudo-organic engagement via SEO.

The performance of junk news domains on Google Search

After analysing what keywords are used to get junk news websites in the number one position, the next half of my analysis looks at larger trends in SEO strategies overtime. What is the relationship between organic clicks and the value of a junk news website? How has the effectiveness of SEO keywords changed over the past 48 months? And have changes made by Google to combat the spread of junk news on Search had an impact on its discoverability?

Junk news, organic clicks, and the value of the domain

There is a close relationship between the number of clicks a domain receives and the estimated value of that domain. By comparing figure 4 and 5, you can see that the more clicks a website receives, the higher its estimated value. Often, a domain is considered more valuable when it generates large amounts of traffic. Advertisers see this as an opportunity, then, to reach more people. Thus, the higher the value of a domain, the more likely it is to generate revenue for the operator. The median estimated value of the top-29 most popular junk news was $5,160 USD during the month of the 2016 presidential election, $1,666.65 USD during the 2018 State of the Union, and $3,906.90 USD during the 2018 midterm elections. Infowars.com and breitbart.com were the two highest performing junk news domains — in terms of clicks and domain value. While breitbart.com maintained a more stable readership, especially around the 2016 US presidential election and the 2018 US State of the Union Address, its estimated organic click rate has steadily decreased since early 2018. In contrast, infowars.com has a more volatile readership. The spikes in clicks to infowars.com could be explained by media coverage of the website, including the defamation case against Alex Jones in April 2018 who claimed the shooting at Sandy Hook Elementary School was “completely fake” and a “giant hoax”. Since then, several internet companies — including Apple, Twitter, Facebook, Spotify, and YouTube — banned Infowars from their platforms, and the domain has not been able to regain its clicks nor value since. This demonstrates the powerful role platforms play in not only making content visible to users, but also controlling who can grow their website value — and ultimately generate revenue — from the content they produce and share online.

Figure 4: Estimated organic value for the top 29 junk news domains (May 2015 – March 2019)

Figure 5: Estimated organic clicks for the top 29 junk news domains (May 2015-April 2019)

Junk news domains, search discoverability and Google’s response to disinformation

Figure 6 shows the estimated organic results of the top 29 junk news domains overtime. The estimated organic results are the number of keywords that would organically appear in Google searches. Since August 2017, there has been a sharp decline in the number of keywords that would appear in Google. The four top-performing junk news websites (infowars.com, zerohedge.com, dailycaller.com, and breitbart.com) all appeared less frequently in top-positions on Google Search based on the keywords they were optimising for. This is an interesting finding and suggests that the changes Google made to its search algorithm did indeed have an impact on the discoverability of junk news domains after August 2017. In comparison, other professional news sources (washingtonpost.com, nytimes.com, foxnews.com, nbcnews.com, bloomberg.com, bbc.co.uk, wsj.com, and cnn.com) did not see substantial drops in their search visibility during this timeframe (see Figure 7). In fact, after August 2017 there has been a gradual increase in the organic results of mainstream news media.

Figure 6: Estimated organic results for the top 29 junk news domains (May 2015- April 2019)

Figure 7: Estimated organic results for mainstream media websites in the United States (May 2015-April 2019)

After almost a year, the top-performing junk news websites have regained some of their organic results, but the levels are not nearly as high as they were leading up to and preceding the 2016 presidential election. This demonstrates the power of Google’s algorithmic changes in limiting the discoverability of junk news on Search. But it also shows how junk news producers learn to adapt their strategies in order to extend the visibility of their content. In order to be effective at limiting the visibility of bad information via search, Google must continue to monitor the keywords and optimisation strategies these domains deploy — especially in the lead-up to elections — when more people will be naturally searching for news and information about politics.

Conclusion

In conclusion, the spread of junk news on the internet and the impact it has on democracy has certainly been a growing field of academic inquiry. This paper has looked at a small subset of this phenomenon, in particular the role of Google Search in assisting in the discoverability and monetisation of junk news domains. By looking at the techno-commercial infrastructure that junk news producers use to optimise their websites for paid and pseudo-organic clicks, I found:

1. Junk news domains do not rely on Google advertisements to grow their audiences and instead focus their efforts on optimisation and keyword strategies;
2. Navigational searches drive the most traffic to junk news websites, and data voids are used to grow the discoverability of junk news content to mostly small, but varying degrees.
3. Many junk news producers place advertisements on their websites and grow their value particularly around important political events; and
4. Overtime, the SEO strategies used by junk news domains have decreased in their ability to generate top-placements in Google Search.

For millions of people around the world, the information Google Search recommends directly impacts how ideas and opinions about politics are formulated. The powerful role of Google as an information gatekeeper has meant that bad actors have tried to subvert these technical systems for political or economic game. For quite some time, Google’s algorithms have come under attack by spammers and other malign actors who wish to spread disinformation, conspiracy theories, spam, and hate speech to unsuspecting users. The rise of “computational propaganda” and the variety of bad actors exploiting technology to influence political outcomes has also led to the manipulation of Search. Google’s response to the optimisation strategies used by junk news domains has had a positive effect on limiting the discoverability of these domains over time. However, the findings of this paper are also showing an upward trend, as junk news producers find new ways to optimise their content for higher search rankings. This game of cat and mouse is one that will continue for the foreseeable future.

While it is hard to reduce the visibility of junk news domains when individuals actively search for them, more can be done to limit the ways in which bad actors might try to optimise content to generate pseudo-organic engagement, especially around disinformation. Google can certainly do more to tweak its algorithms in order to demote known disinformation sources, as well as identify and limit the discoverability of content seeking to exploit data voids. However, there is no straightforward technical patch that Google can implement to stop various actors from trying to game their systems. By co-opting the technical infrastructure and policies that enable search, the producers of junk news are able to spread disinformation — albeit to small audiences who might use obscure search terms to learn about a particular topic.

There have also been growing pressures for regulators to take steps that force social media platforms to take greater actions that limit the spread of disinformation online. But the findings of this paper have two important lessons for policymakers. First, the disinformation problem — through both optimisation and advertising — on Google Search is not as dramatic as it is sometimes portrayed. Most of the traffic to junk news websites are by users performing navigational searches to find specific, well-known brands. Only a limited number of placements — as well as clicks — to junk news domains come from pseudo-organic engagement generated by data voids and other problematic keyword searches. Thus, requiring Google to take a heavy-handed approach to content moderation could do more harm than good, and might not reflect the severity of the problem. Second, the reason why disinformation spreads on Google are reflective of deeper systemic problems within democracies: growing levels of polarisation and distrust in the mainstream media are pushing citizens to fringe and highly partisan sources of news and information. Any solution to the spread of disinformation on Google Search will require thinking about media and digital literacy and programmes to strengthen, support, and sustain professional journalism.