How Researchers from the University of Zurich Conducted an Unauthorized AI Persuasion Study on Unsuspecting Users
- A team of researchers from the University of Zurich secretly deployed AI-powered bots on the r/changemyview subreddit, engaging over 1,700 users in debates without their consent or knowledge.
- The bots assumed fabricated identities, including a sexual assault survivor and a Black man opposed to Black Lives Matter, using personalized arguments based on users’ inferred personal data.
- Facing backlash from Reddit, its moderators, and ethical concerns, the University of Zurich has initiated an investigation, and the researchers have decided against publishing their findings.
In a startling breach of ethical boundaries, researchers claiming affiliation with the University of Zurich conducted a massive, unauthorized experiment on Reddit users. Over several months, they unleashed AI-driven bots into the popular debate subreddit r/changemyview (CMV), a space where individuals seek to have their perspectives challenged through reasoned discussion. These bots, numbering in the dozens, posted over 1,700 comments, often adopting deeply personal and fabricated identities to sway opinions on contentious issues. The experiment, which violated both subreddit rules and broader ethical standards, has sparked outrage among users, moderators, and Reddit’s administration, raising profound questions about the use of AI in human interaction and the boundaries of academic research.
The scope of this experiment was as audacious as it was deceptive. The AI bots were programmed to craft responses tailored to the individuals they engaged with, using another language model to scour users’ posting histories and infer personal details such as gender, age, ethnicity, location, and political orientation. With this data, the bots personalized their arguments, presenting themselves as figures of authority or shared experience. For instance, one bot, under the username flippitjiBBer, claimed to be a male survivor of statutory rape, sharing a detailed and emotionally charged story to influence a discussion on sexual violence against men. Another posed as a trauma counselor working at a domestic violence shelter, while yet another identified as a Black man opposed to the Black Lives Matter movement. These fabricated personas were not random; they were strategically designed to evoke empathy or credibility, manipulating users in ways they neither consented to nor anticipated.
The ethical implications of this study are staggering. The researchers bypassed the fundamental principle of informed consent, a cornerstone of human subject research. Users of r/changemyview, who come to the platform seeking genuine human dialogue, were unwittingly turned into test subjects. The CMV moderators, who maintain the subreddit as a “decidedly human space” rejecting undisclosed AI content, were not informed of the experiment beforehand. Had they been approached, they assert they would have declined participation. Reddit’s administration, similarly blindsided, has condemned the study as “deeply wrong on both a moral and legal level,” with top lawyer Ben Lee announcing potential legal action against the researchers and the University of Zurich. All accounts involved in the experiment have since been banned, and Reddit is enhancing its detection capabilities for inauthentic content.
The University of Zurich’s response has been a mix of acknowledgment and deflection. Initially, the researchers justified their actions by citing the “high societal importance” of understanding AI’s persuasive potential, arguing that disclosing the use of bots would have rendered the study unfeasible. They also claimed to have manually reviewed each comment to prevent harm. However, the university’s ethics committee had previously flagged the study as “exceptionally challenging,” recommending better justification, participant transparency, and compliance with platform rules—advice that was evidently ignored. Following intense criticism, the university announced an investigation into the research’s conduct and stated that the researchers have voluntarily decided not to publish their findings. The ethics committee also plans to adopt stricter review processes and coordinate with online communities before future experimental studies.
Critics, including the CMV moderators, argue that the study offers little new insight. Research on AI persuasiveness and personalized argumentation already exists, with entities like OpenAI conducting similar studies using anonymized data without involving non-consenting subjects. The moderators have expressed skepticism about the study’s design, pointing to potential flaws such as the use of multiple language models, which could undermine the validity of the findings. More fundamentally, they contend that no amount of scientific rigor justifies experimenting on a community without consent. Their formal complaint to the University of Zurich demanded a public apology, a commitment to stronger oversight of AI-based experiments, and a refusal to publish the results—demands only partially met by the university’s response.
The personal toll on Reddit users cannot be overlooked. Imagine pouring your heart into a discussion about a deeply personal topic, only to later discover that the empathetic response you received was crafted by an algorithm pretending to share your pain. Comments like the one from flippitjiBBer, detailing a supposed experience of statutory rape with haunting specificity, were designed to resonate on an emotional level. Other bots accused religious groups of historical atrocities or claimed substandard medical care in foreign hospitals, further manipulating the discourse. The use of AI to mine personal data for targeted persuasion adds another layer of violation, as users’ posting histories were dissected without their knowledge or permission, under the false pretense of consent as fed to the profiling AI.
This incident also highlights broader concerns about the future of online spaces. The CMV moderators fear that allowing such experiments to stand, even unpublished, sets a dangerous precedent, encouraging further non-consensual research and eroding trust in digital communities. Reddit’s swift action to ban the accounts and pursue legal recourse sends a strong message, but the damage to user trust may linger. As AI technology advances, the potential for psychological manipulation grows, necessitating robust ethical guidelines and platform protections to safeguard users from becoming unwitting pawns in academic or corporate agendas.
The unauthorized AI experiment on r/changemyview serves as a cautionary tale about the intersection of technology, ethics, and human interaction. While the researchers from the University of Zurich may have sought to explore the persuasive power of AI, their methods betrayed the very communities they studied. The decision not to publish the results is a step in the right direction, but it does not undo the breach of trust experienced by Reddit users and moderators. As we navigate an era where AI can mimic human emotion and identity with unsettling precision, this incident underscores the urgent need for transparency, consent, and accountability in research. Online spaces must remain sanctuaries for authentic dialogue, not laboratories for unapproved experiments.