The escalating risks of mass biometric surveillance in the age of AI | Part 1
What’s worse than being constantly watched? Not knowing you’re being watched, or that your unique physical and biological information is being fed into a vast surveillance infrastructure that you have no control over. From ICE’s use of facial recognition on the streets in the US to Hungary identifying and tracking Pride event attendees, China’s monitoring of ethnic minorities, France normalizing mass surveillance during the Paris Olympics, and the monitoring of political dissenters in Africa, these security systems are rapidly blurring the line between targeted and mass surveillance.
In part 1 of this two-part series, we examine the power imbalance of mass biometric surveillance beyond the justification of public safety: how this rapidly expanding infrastructure impacts the “watched.” In part 2, we explore how artificial intelligence (AI) is amplifying an already dangerous erosion of privacy, democracy, and freedom across the globe, concluding with the current legal frameworks, accountability gaps, and how transparency can empower citizens, build trust, and help bridge that gap.
Written by Lindsay Langenhoven and co-authored by Rachel Fagen, in collaboration with Olivia Mora.
What are biometric, targeted, and mass surveillance?
Biometric technology identifies a person based on their unique physical and behavioral characteristics, such as their fingerprint or face. Iris recognition, voice recognition, DNA matching, a person’s walking gait, their online engagement patterns, and now even their brain morphology are also forms of biometrics. When security or law enforcement agencies use these characteristics to identify or monitor a person, it’s referred to as biometric surveillance.
The government and private sector use surveillance technologies to perform targeted surveillance or mass surveillance. Targeted surveillance identifies a particular individual in a specific investigation. Mass surveillance, on the other hand, monitors the public indiscriminately. With mass surveillance, there is no reasonable suspicion that law enforcement is acting on, and the surveillance is often carried out without the public’s knowledge.
Mass biometric surveillance, like Facial Recognition Technology (FRT), gathers and processes anyone’s biometric data in public spaces such as parks, squares, sports stadiums, transport hubs, or online public spaces. It’s very different from targeted or personal use, like unlocking your phone, which doesn’t infringe on a person’s ability to enjoy their rights in public spaces.
Today, the pervasiveness of AI in business and government infrastructure increases the stakes for the public. AI has essentially collapsed the difference between targeted and mass surveillance: what used to require targeted deployment now happens by default. This change has prompted many scholars and civil rights advocates to question whether public safety is the actual purpose of mass surveillance.
Image credit: Comuzi / https://betterimagesofai.org / © BBC / https://creativecommons.org/licenses/by/4.0/
Public safety or pretext?
While the goal of mass surveillance is to improve public safety—protecting citizens from threats ranging from everyday crime to terrorist attacks—we may need to think more critically about this premise. Is that the true purpose of this type of technology? Who actually benefits from its use?
Mass surveillance systems are allegedly built for public safety. But, in many cases, safety is the justification used to legitimize the deployment of infrastructure built for something else entirely.
The use of police body cameras, for instance, demonstrates this incongruence. American civil rights lawyer and author Alec Karakatsanis highlights how police body cams are often hyped up by vendors, duping the public into believing the cameras improve transparency and accountability. In reality, the number of civilians killed by police annually “has only increased each year since the widespread adoption of body camera equipment.”
In a growing number of cases, the public safety narrative is little more than a pretext. In China, one of the most surveilled nations in the world, extensive surveillance purports to promote public safety, while one of its core functions is ethno-religious control of Uyghur and other minority populations.
The Electronic Frontier Foundation (EFF) highlights a similar misrepresentation in the US. Texas deputies utilized Flock Safety’s surveillance data in an abortion investigation, claiming the woman was “being searched for as a missing person” and that the surveillance operation was about her “safety.”
In reality, this “safety” veneer frequently cracks, exposing the shortcomings of surveillance as more systemic in nature. In the research paper, “The Surveillance AI Pipeline,” Dr. Abeba Birhane and fellow researchers uncover the real motives behind these systems. They document a fivefold increase from the 1990s to the 2010s in computer vision papers linked to surveillance-enabling patents, highlighting how the “institutions authoring the most papers with downstream surveillance patents align with well-established historical legacies of the military-industrial-academic complex.”
Critically, the evidence in support of the “public safety” motive is largely self-reported by vendors and agencies that benefit from its continued use, while the harms remain systematically undercounted. In Chicago, “where police can tap into a network of over 10,000 cameras, the arrest rate attributed to the camera system is less than 1% of the total police arrests in a recent three-year period,” the American Civil Liberties Union (ACLU) stresses. As a result, these agencies are compelled to collect more and more data to improve their success rate.
Ultimately, surveillance can serve legitimate safety goals, such as rapid incident response, but that’s increasingly the exception, not the rule. Today, the use of “public safety” as a justification has expanded to cover protest monitoring, immigration enforcement, and school attendance management, often well outside what citizens understood they were consenting to.
Data collection, participatory surveillance, and consent
Since the advent of social media, our data has formed the backbone of the surveillance economy, monetizing our personal lives. Today, tech companies like Meta and Clearview AI (who secretly scraped approximately 10 billion images from sites like Facebook and Twitter) actively acquire our personal data by scraping it from the web. Or via surveillance apparatus when we pass through a surveilled public space like an airport or shopping center.
These institutions can also acquire our data more passively, when the public takes the lead, sharing their personal data on social media, for instance. This is called participatory surveillance and occurs when citizens actively share or contribute their personal data through mechanisms they often don’t recognize as surveillance, such as social media, chatbots, or fitness trackers. It’s important to note that while sharing their information is voluntary, their participation is engineered by manipulative design and not freely chosen.
Devices like Meta’s smart glasses and smart doorbells extend these privacy concerns, surveilling not just their owners but everyone who passes within range. The privacy and misuse risks are even greater when citizen-level surveillance feeds into larger infrastructure. For instance, Amazon Ring footage feeding into Flock camera networks. Under these circumstances, individual participation is quietly enabling systemic harm.
Typically, mass biometric surveillance infrastructure gathers citizens’ biometric information without their knowledge or consent. These systems offer little (if any) transparency. Citizens don’t know what data is collected, how, or why. Or even that they are being monitored. That convenient “oversight” makes these systems more deployable and commercially valuable, because their “subjects” don’t even know they’re enrolled.
Ultimately, Big Tech companies gather more and more of our data, which amplifies their power and profit. But their approach ignores the fundamental principle at play:
Transparency is the precondition for consent, and consent is the precondition for legitimacy.
In the case of mass biometric surveillance, if people don’t know they’re being surveilled, they can’t agree to it. And if they can’t agree to it, the system has no legitimate democratic basis. Rather, it is exercising unchecked power.
Image credit: Comuzi / https://betterimagesofai.org / © BBC / https://creativecommons.org/licenses/by/4.0/
The impact of mass biometric surveillance
The mass extraction and processing of human data affects every citizen—vulnerable groups in particular. Yet, surveillance infrastructure is becoming more pervasive. According to Privacy International, roughly 75% of governments and police forces across the globe deploy (or have access to) FRT on a large scale.
Increasing bias and discrimination
Biometric surveillance tools demonstrate a consistent pattern of inaccuracy. Yet, US Immigration and Customs Enforcement (ICE) is currently using unvetted FRT apps like Mobile Fortify on the public. In field conditions, poor lighting or off-angle images degrade image quality, and with it, accuracy. In one case, this led to the app returning “two different names after scanning a woman’s face during an immigration raid,” 404 Media reports.
Research into algorithmic discrimination by Joy Buolamwini and Timnit Gebru exposed how false arrests from facial recognition have overwhelmingly involved Black individuals: error rates for darker-skinned women reached 34.7% in commercial systems, compared to 0.8% for lighter-skinned men.
According to Birhane, this outcome is far from accidental. In fact, accuracy likely decreases as these systems scale. The audit she and fellow researcher, Vinay Uday Prabhu, conducted in 2020 of MIT’s “80 million Tiny Images” dataset—one of the most widely used foundations for training computer vision systems—found it riddled with racist and misogynistic labels applied to people’s images without their knowledge or consent.
In addition, Birhane’s subsequent research, “The Dark Side of Dataset Scaling,” found that as multimodal training datasets increase in size, bias gets worse, not better. When datasets scaled from 400 million to two billion samples, the probability of a model misclassifying a Black man as “criminal” increased, reaching close to 100% frequency in one model configuration.
These findings undermine one of the industry’s most common defenses of surveillance AI: that more data and larger models will eventually solve the bias problem. But as scale further degrades datasets and the accuracy of mass biometric surveillance tools, the harms become more real-world and dehumanizing.
Eroding privacy and human rights
Privacy in a public space is not an oxymoron. At least, it shouldn’t be. The legal fiction that “you have no expectation of privacy in public” was developed in an era when being seen in public meant only being recorded by the fallible human memory of the person that happened to be present. The situation is very different today.
According to Privacy International, Moscow has more than 200,000 FRT cameras covering more than 90% of its public spaces and residential areas. Similarly, China has implemented a national surveillance architecture that ties directly into its social credit system for every citizen.
These human rights violations are also entrenched in heterogeneous societies. Amnesty International documents the Israeli authorities’ use of an extensive FRT infrastructure in the Occupied Palestinian Territories (OPT), from FRT surveillance at multiple checkpoints to city-wide surveillance. Mass biometric surveillance not only “entrenches restrictions on freedom of movement,” but it also becomes “a weapon of discrimination, segregation and oppression,” Amnesty reports.
One interviewee shared her experience: “I’m being watched the whole time…[it] gives me a really bad feeling everywhere in the street. Every time I see a camera, I feel anxious. Like you are always being treated as if you are a target.”
Across the globe, the danger of mass surveillance, Tech Global warns, lies “not only in unchecked overreach, but in the quiet normalization of unaccountable governance in the digital sphere.”
Increasing data security risks and function creep
Biometric data is uniquely dangerous to lose in a cybersecurity breach. Unlike a password, biometrics cannot be changed. You can issue a new password; you cannot issue a new face.
Security and privacy risks escalate with mass biometric surveillance. “Facial recognition technology is often deployed as part of a broader ecosystem of surveillance tools,” the Electronic Privacy Information Center (EPIC) warns. As a result, law enforcement agencies can “quickly connect persons identified through facial recognition to a vast amount of information in government or commercially available databases.”
Rapidly growing datasets are prone to trigger function creep. This phenomenon is not an edge case in biometric surveillance; it is the operating logic. Essentially, algorithmic systems are built to be repurposed, and the possibility of expanding their functions is baked into the technology.
There are many examples of surveillance function creep worldwide, including:
- Pandemic monitoring → general population surveillance (China)
- Doorbell cameras → normalized home surveillance (US)
- Traffic management → crime profiling (India)
- School canteen payments → attendance and movement tracking (UK)
EPIC warns that “as this mission creep continues, the risks that agencies will use these systems to target minority communities and constitutionally protected activities only increases.” And those risks grow exponentially with AI in the mix. We’ll explore the impact of AI in surveillance infrastructure in Part 2 of this article series.
The road ahead
Building on our understanding of what mass biometric surveillance is, the true intentions behind its use, and its impact on every continent, we explore how AI is scaling surveillance, the chilling effect, accountability gaps, and the role of transparency in reducing harms in part 2 of this two-part series.
If any organizations want to chat about transparent and accountable surveillance solutions, such as Open Ethics’ Public Surveillance Transparency Project, feel free to reach out.