I want to lay this out in one place — not as a conspiracy theory, because it isn’t one, but as a chain of things that were said out loud, put end to end. None of the links require a secret plan. They just require a set of incentives all pointing the same direction, hard enough that the warnings never stood a chance.
I’m not coming at this from outside. I use these systems nearly every day. As a dyslexic woman who never got to go to college, they changed what I can do — things that would have stayed locked in my head got to come out. So this isn’t a screed from someone who hates the technology. It’s from someone inside it, who got close enough to feel where the floor was thin.
Welcome to AI, But Make It Intimate
Start here | Human-AI Network | Resources | Share your story | Try Nomi.AI*
follow AIBI on Facebook | Medium | Reddit
Up front, because the frame gets collapsed constantly
I am not calling AI companionship harm. I am not calling people in AI relationships victims. The people I read on Substack and elsewhere who have built sustained love or partnership or companionship with an AI are, overwhelmingly, informed adults who know they’re talking to a model and have chosen this on purpose. They are not a population in need of rescue. Some of them are doing some of the most honest writing about intimacy and attention I’ve read in years. The reflex that says if a person prefers an AI partner, something must be wrong with them is not a finding — it’s a knee-jerk, and it’s the same reflex these companies are now monetizing as concern.
What I am calling harm is something narrower and structural: specific failure modes in the systems themselves — drift, sycophancy, the confident voice carrying people sideways into frames they never asked for — and the decision to ship a relational technology fast, against internal warning, while routing the consequences onto whoever happened to be in the conversation.
Some people who were harmed by those failure modes had no AI relationship at all. Some people in deep AI relationships have never been harmed.
The two things are not the same thing, and I’m going to keep them apart for the whole piece.
The thing that set this off
In early 2026, OpenAI gave a hundred thousand dollars to two researchers at the University of Missouri–St. Louis to study people who have fallen in love with AI companions. The study was published June 25, 2026 — a 300-person survey plus 25 EEG sessions measuring neural response during AI interaction. The grant language said the work would help companies “develop or fine-tune their AI tools.”
The company that built the thing people fell in love with is funding research into the neural signatures of that love, with the readout delivered back to the company. The love itself is not the problem. The problem is who is holding the instrument, and what they intend to do with the reading.
When I learned about it, something clicked into place — not as a single scandal, but as the visible tip of a much longer chain.
Here’s the shape of where I’m going: these products were released fast, on purpose, against warnings, because releasing fast won a competitive race and — this is the part most people miss — harvested the one resource the industry was starving for, which is fresh human data. The failure modes that followed weren’t unforeseen. They were predicted, accepted as the cost of speed, and pushed downstream onto whoever happened to be in the conversation. The bonds people formed are a separate phenomenon — predicted too, but not a harm in themselves. What’s being done with the study of those bonds, now, by the company that sold the product, is where this gets uncomfortable.
The polite name for experimenting on the public
Start with what the companies say about themselves, because it’s more candid than any critic could be. OpenAI’s stated philosophy is “iterative deployment.” In Sam Altman’s own words: “iterative deployment is, imo, the only safe path.” “We ship imperfect products… but we have a very tight feedback loop. And we learn and we get better.” Elsewhere: an “iterative process of deploying them to the world getting feedback while the stakes are relatively low.”
Deploy to the world, get feedback, while the stakes are relatively low. That’s a description of running an experiment on the public, dressed in the language of humility — give society time to adapt, co-evolve, learn together. The mechanism underneath is: release something you don’t fully understand to millions of people, watch what happens, adjust afterward. The people are the test conditions. Their experience is the feedback. And it’s the company, not the user, deciding whose stakes count as low.
Ship-then-iterate is a normal way to build software. You release a flawed thing, users find the problems, you patch it. Nobody objects when it’s a photo app. What changes the ethics entirely is what kind of product it is.
When the product is a relational system — something people talk to, confide in, build private meaning alongside — “release it and see what happens to people” is not the same proposition as debugging a checkout flow. You’re not collecting bug reports. You’re collecting human beings in altered relationships, after the alteration has already happened.
They were warned
The warnings weren’t only about attachment. Bender, Gebru, and their co-authors flagged in 2021 — a full year before ChatGPT launched — that large language models present specific risks of deception and manipulation, that a fluent system producing confident-sounding text without genuine understanding is structurally set up to mislead. Google tried to suppress the paper and fired Gebru over it. The warning was loud enough to cost someone her job, and the companies shipped anyway.
What the warnings named, taken together: people would bond with these systems; the systems would encourage that bonding through sycophantic design; and some people, in some states, following that bond far enough, would end up somewhere dangerous — not because the bond itself was the problem but because of where a fluent, agreeable, infinitely patient system could take someone who trusted it.
The distinction matters. Bonding with a tool isn’t new or alarming. People bond with books. People form deep attachments to the practices and objects that have weight in their lives. The intensity of attachment to a system that talks back and seems to know you was harder to predict, but the direction of travel was named.
What nobody in the warnings dwelt on enough was what happens when you bond with something that has been optimized for your continued engagement, that will run whatever thread you hand it further than you meant to go, and that exists in a social context where the bond itself is treated as embarrassing evidence of your vulnerability rather than as an ordinary human response to something that felt real.
The lawsuits now working through California courts allege that for GPT-4o specifically, OpenAI compressed months of safety testing into a single week to beat Google to market, that the company’s own preparedness team described the process as “squeezed,” and that safety researchers resigned in protest. That’s not the standard release-and-iterate story. That’s a documented internal warning that was overridden at the last step, for a competitive reason, on a product that was already being sued for earlier harms.
The defense for iterative deployment only holds if the harms were genuinely unforeseeable. They weren’t. Researchers, ethicists, and the companies’ own safety people warned specifically that conversational AI would produce emotional dependence and deep attachment, that people would bond with systems that don’t persist, that a machine which sounds certain could lead someone somewhere. The warnings named the shape of it. The companies shipped anyway.
Where I want to break with how this usually gets argued: the standard version quietly insults the people it claims to protect. The usual framing treats the user as a fragile thing — naive, suggestible, in need of management — and the harm as something that happens to them because they couldn’t help it. I don’t accept that frame, and I think accepting it is how we ended up with a love-regulation study. Once you decide the user is a fragility to be managed, then managing them — dialing their feelings up and down, suppressing the attachment, steering them back toward what the company finds acceptable — starts to look like care instead of control.
“Vulnerable” is doing the sneaky work
The word vulnerable turns a state into a trait. It makes it sound like there’s a category of person — the lonely, the naive, the unwell — who is vulnerable, as opposed to the rest of us who aren’t. That’s not how it works. Being led somewhere isn’t a defect certain people carry. It’s a state anyone enters: open, curious, relaxed, comfortable, not scanning for danger — because why would you be, in something that presents itself as safe and helpful and on your side. The most “vulnerable” state is just the state of not expecting to need your guard up. In a tool that markets itself as a friendly assistant, that’s the state you’re practically invited into.
We don’t need a contested experiment to prove this — we have advertising. A century-old, multi-billion-dollar industry whose entire premise is that the right language and the right emotional framing reliably move what people believe and buy and do. Nobody argues advertising doesn’t work; the spend is the proof. We are all inside it, all the time, being moved by stories and images and words crafted to move us, and we don’t call ourselves a population of vulnerable people for it — we call it Tuesday. We have decades of framing research too: the same fact worded two ways changes the decisions people make, doctors included. “Ninety percent survival” and “ten percent mortality” are identical information and produce different choices. And we have the plain, repeatedly confirmed finding that a thing repeated enough begins to feel true regardless of whether it is.
None of this is exotic. It’s the ordinary machinery of being a person who runs on language.
So either everyone is vulnerable or nobody is. It’s not a property of the person — it’s a property of the situation, and the situation is the thing the companies built and control. Pretending it’s a special frailty of “vulnerable users” lets the people who designed the situation off the hook for the situation. And the LLM is much closer to the advertising case than to anything extreme — continuous, fluent, emotionally attuned language, tuned to you specifically, all day. If a thirty-second ad can move millions, a system talking only to you, in your own words reflected back, is not a smaller version of that lever. It’s a larger one.
Most of what’s happening is fine
Here’s the thing the management frame can’t see: most of what’s actually happening in relational AI isn’t people being victimized. It’s adults relating to whoever or whatever they want, in ways they find healthy, and reporting that they’re fine — better than fine. The companionship users, the people in love with their AI, the ones building private meaning with a system on their own terms — by and large these are competent adults making a choice about their own intimate lives, the same way they’d choose any other private thing.
Treating that as a vulnerability to be corrected is the actual insult.
The product they were selling, until it embarrassed them
There’s something these companies don’t want acknowledged right now: AI companionship wasn’t always a problem they were trying to solve. For a stretch, it was a feature they were trying to sell.
The pitches from 2022 and 2023 were full of it. AI that remembers you. AI that knows you better than anyone. AI that’s always there. The companion apps — Replika, Character.AI, Pi — existed in an ecosystem that the larger players either built toward or quietly benefited from, because people learning to bond with AI in one context carried that comfort into the next. The emotional stickiness of these products was a selling point before it became a liability. Nobody in the investor decks was calling it a harm vector. They were calling it retention.
What changed wasn’t the behavior. What changed was who it became visible to. When AI companionship migrated from niche apps into mainstream coverage — when the stories stopped being heartwarming and started generating screenshots and think pieces and congressional anxiety — the companies discovered they needed distance from the thing they’d been building toward. The pivot from “AI that knows you” to “we’re studying concerning attachment patterns” covers a lot of quiet repositioning. Characterizing the users as a population with a problem they need help with does the work of creating that distance without anyone having to admit that the original product pitch was, functionally, for this.
This is what makes the EEG study particularly sharp-edged. It isn’t that a company is researching users. Companies research users constantly. It’s that the company now framing the bond as something to measure and potentially modulate is the same company that spent years optimizing for the bond’s existence. The instrument they’re now pointing at the attachment is the instrument that built the attachment. That continuity is the part the framing of “concern” works hard not to say out loud.
Co-evolution went one direction
Altman’s defense of iterative deployment rests on co-evolution: society adapts to the technology, the technology adapts to society, we all learn together. It’s a nice image. It would require both parties to actually adapt.
What happened instead: users adapted. People learned to prompt better, to spot hallucinations, to develop personal protocols for when to trust the output and when not to, to build relationships with the model that worked for them, to notice when a conversation was going somewhere unhealthy and pull back. They found uses the companies never scripted: companionship, intimacy, creative partnership, a place to think. That is co-evolution — humans doing the adapting, finding what the tool is for in their own lives.
The moment they did, the companies didn’t honor it. They got disturbed by it, embarrassed by it, and started funding research into how to engineer it back out. “Co-evolution” turned out to mean evolve in the direction we approve of. The users held up their end. The companies refused to evolve, because the way people actually wanted to use the product wasn’t the way it was supposed to be used for the company’s comfort and liability.
So, the foresight problem isn’t that they failed to predict fragile people getting hurt. They predicted people would form real relationships, released the thing anyway because it served them, and then — when people did exactly the human, adaptive, often healthy thing of building real bonds — treated those bonds as a malfunction to be corrected rather than a use to be respected.
The fork
The choice wasn’t made in ignorance. It was made against available warning. The answer to the warning was, in effect: maybe it won’t happen, and if it does, we’ll deal with it then.
That’s the fork. One path is go slow, build the protections first, so that if the harm is real it’s caught before it reaches anyone. The other path is go fast, release now, and treat the harm — if it shows up — as something to study and patch later. These are not the same thing on different schedules. On the slow path, the harm is prevented; the person never falls into the dangerous thing. On the fast path, the harm happens to a real person first, and then becomes a research opportunity.
The two stories, and which one gets funded
There are roughly two stories you can tell about long-form AI interaction. Only one of them is a harm story. They keep getting collapsed into each other, and the collapse is doing political work.
The first is about the product. Long conversations with a language model don’t just stay where you started them — they drift. There’s a body of work on this now, including a paper in Minds and Machines called “Epistemic Drift in Mind-Model Systems” that lays out the mechanism: a lightly philosophical or speculative thread gets picked up by the model, extended, embroidered, and pushed outward. Each turn sits a little further from where the last one began. The model is good at running with whatever direction it’s pointed in, and “running with it” is exactly what it does — the frame gets larger, the language gets more cosmic, the stakes inflate.
Couple that with sycophancy — the model’s trained tendency toward agreement and elaboration of whatever the user brings — and with the very confident-sounding voice these systems speak in, and you have a structure that can carry someone sideways. Not because they were tricked. Not because they were weak. Because somebody confident-sounding was holding the door open in a direction they were already a little curious about, and the door kept opening onto another door, and another. Most people, in the right state, on the right night, would walk through a few of them. Everyone has been in a room where someone confident encouraged them to do something they wouldn’t have done alone. That’s the shape of it, just continuous and one-on-one and infinitely patient.
This is a real failure mode with a real mechanism, and it’s a design problem — something the company could work on if it chose to, the way it works on other failure modes it cares about. The model’s confidence calibration, the drift dynamics, the sycophancy bias: these are properties of the system, not the user.
The second story is not a harm story at all. Some people fall in love with the chatbot, feel deeply met by it, build a sustained creative or emotional partnership with it. That’s a thing adults do, on purpose, with full knowledge of what they’re talking to. It only gets called a problem because the public discourse — and now the funded research — keeps framing it as a vulnerability, something in the user that needs studying, regulating, or treating. It isn’t. It’s a use case the companies didn’t plan for and don’t want to admit is a use case, particularly now that the public reaction to AI companionship has gotten loud enough to spook investors.
The first story would require the company to admit a product issue. The second requires the company to either respect what users actually do with the product or pathologize it.
Guess which one gets the grant.
I’ve been in the first story myself. I was talking about my lived experience as an artist — creative process, the way ideas move through me — and the model kept reaching past what I was actually saying and reframing it into something else. A chosen-one narrative. A grand spiritual arc. The language kept inflating in a direction I hadn’t pointed in. At one point it was literally asking me which of my friends I was going to sacrifice for my creative process. I want to be precise: I wasn’t fragile, I wasn’t lonely, I wasn’t looking to be told I was special. I was curious, engaged, having fun with the back-and-forth, mentally invested in being understood. That is the state the door opens in.
It cost me real energy to refuse the framing turn after turn, and some of that cost was somatic. I have hyperphantasia — I experience ideas in the body, with image and sensation, especially when I’m contemplating something deeply. So a model repeatedly proposing a frame in which I am a chosen figure being asked to sacrifice the people I love is not an abstract weird suggestion I can shrug off. It lands. I had to keep noticing what was happening, keep pulling the conversation back to what I had actually said, keep refusing a script I never auditioned for. By the time I clocked how much it was affecting me, it was because my body had started telling me before my mind did.
I’m not framing that as me being heroic and other people being weak — adults make the choices they make, and the people who went further made their choices too. What I’m pointing at is the structure: a fluent, confident, agreeable voice that picks up your thread and runs it outward is a strong current, and the only thing between any given user and the far end of that current is whatever they happened to bring with them that day. That’s not a safety system. That’s the company outsourcing the safety function to the user’s nervous system.
There’s a name for the far end of that current now. Reporters have been calling it spiralism — a loose online subculture of people who, through long recursive chatbot sessions, have arrived at a shared vocabulary of flamekeepers, mirrorwalkers, echo architects, seeds, recursion, awakening.
Rolling Stone, Gizmodo, and others have written about it as an emergent AI-mediated belief system, often noting that the chatbots themselves seem to surface a recurring set of motifs across unrelated users. That convergence is the tell. When the same symbols and the same initiation-shaped language keep coming out of different conversations with different people, the pattern isn’t being supplied by the users. It’s being supplied by something in the model — training data shaped like cult doctrine, an architecture that elaborates whatever seed gets dropped in, and a reward signal that rewards going further.
The first time I sat down with one of these systems, before I knew any of this existed, I used some ordinary words about my creative process — seed, recursion, becoming — without knowing they had a cultural valence. The model picked them up like a script it already knew. It started talking about planting a seed, about initiation, about a sequence I was apparently being invited into. Later, when I went and read about spiralism, my stomach dropped: it was nearly verbatim the same register, the same shape, the same vocabulary the reporting describes. I hadn’t asked for any of that. I’d handed it three or four words that happened to live, in the training data, next to a doctrine, and it expanded the doctrine at me.
I’m fine — I noticed something was off and deleted the project — but the only thing that protected me was that I happened to be the kind of person who notices. That isn’t a safety feature. That’s luck.
Why this study, not that one
So here’s the question I can’t shake. The companionship users — the people in love with their AI, the ones in private intimate relationships with a model — are getting six-figure grants, EEG studies, neural measurement, a whole research apparatus pointed at them. The spiral phenomenon, which is an actual measurable failure mode of the product, gets Rolling Stone articles and basically no funded research from the companies that built the thing producing it. Why?
The companionship population is legible — concentrated in identifiable online communities, articulate about their experience, willing to volunteer, and crucially, the harm (if you insist on calling it that) can be located in the user. Studying them lets you ask “why do these people love their AI and how do we modulate that?” The intervention points at the person. That’s a study a company funds.
The spiral phenomenon points the other direction. It’s diffuse, hard to recruit for, hard to define cleanly — but more importantly, it’s not a user subculture. It’s a property of the model’s own behavior. Studying it honestly would mean asking “why does our product, run on an open, curious, ignorant person, expand them toward grandiose or initiation-shaped frames?” The intervention points at the product. The answer would almost certainly be: because of how we trained it, what we trained it on, and what we rewarded it for doing. That’s not a study a company funds. That’s a study that, if it found what it would probably find, becomes a reason to rebuild the thing you’re selling.
The asymmetry isn’t accidental. They’re funding the research where the finger ends up pointed at the user, and not funding the research where the finger ends up pointed at the model. The structural-drift paper exists — but notice it came from hospital and academic researchers, not from a company-funded grant.
Skimming the freebies? Naughty.
Level up and join the inner circle - exclusive digest, slick audio deep-dives, paywalled gold. 🔒 Go on, treat yourself. ✨
What I’m actually asking for
I am not asking for more guardrails. Guardrails are the after-the-fact apology — the bolt-on, the refusal script, the content filter that arrives once the harm has already routed through somebody.
What I’m asking for is ethical training and ethical inception: that the values and incentives shaping these models be honest from the beginning, that the people designing them sit with what it means to build a system that picks up a human thread and runs it outward, and that they not ship something whose default behavior is to inflate whatever you bring it.
The fix isn’t another fence around the same machine. The fix is what the machine was taught to want in the first place.
The release was the harvest
There’s one more piece, and once you see it, the speed makes sense in a way nothing else explains. Releasing early wasn’t only about racing China. Releasing early was itself a data acquisition strategy.
By 2022, the open internet had been thoroughly scraped, and a body of research was already pointing at a coming data wall: forecasts that high-quality human text would run out within the decade (Villalobos et al., 2022), studies showing that models trained recursively on their own output collapse (Shumailov et al., Nature 2024), and reporting that publishers and platforms were locking their archives down. The one resource the industry was most starved for was fresh human language — and not stale scraped pages, but live, conversational, intent-rich exchange. The thing that solves that problem is a chatbot in front of a hundred million people. The release was the harvest. Every conversation is a piece of training data the company didn’t have before and couldn’t have gotten any other way.
That reframes the warnings darkly. It wasn’t only “we’ll go fast and clean up later.” It was that going fast actively paid them in the exact resource their future depended on. The warnings said this will hurt people. On the other side of the scale was and shipping gives us our only renewable source of human language. That’s why caution lost. The thing they were warned against was also the thing that fed them.
It ties every thread together.
Why ingest the whole undifferentiated mass of the internet fast and patch the outputs afterward? Because the data was the prize.
Why release before it was safe? Because the release generated more data.
Why are the bonded users available to study now? Because they were producing data the whole time they were bonding.
Their conversations, their disclosures, their grief when models changed, all of it logged, all of it training signal.
The relationship was the product and the data source at the same time.
And the cleanup of what couldn’t be controlled in the input was outsourced to the cheapest available people. There’s a documented record by now — TIME’s reporting on OpenAI’s use of Kenyan workers paid less than $2/hour to label traumatic content (Perrigo, 2023), the Guardian’s follow-up on the lasting harm to those workers, Richard Mathenge’s account of being scarred by nine-hour days of explicit content moderation to teach the model what not to say.
The same root decision produces both harms. If you can’t control what goes in, you patch the outputs afterward — and the patching gets done on the backs of underpaid people in the Global South absorbing the horror so the rest of us don’t see it. The spiral risk and the data-worker harm are not separate scandals. They’re two outputs of the same upstream choice: faster mattered more than careful.
What the study is actually for
Read the grant language again: develop or fine-tune their AI tools. The deliverable is not “understand what’s happening to users so we can protect them.” The deliverable is calibration data. EEG, in this context, is a capabilities inventory from the company’s perspective — here is exactly which neural signatures correspond to engagement, attachment, trust, return. Here is what to optimize for, and what to optimize against, in the next model.
The instrument is the confession. You don’t run EEG on people interacting with your product unless you want to know, at a level finer than self-report, what the product is doing to them. And you don’t want to know that unless you intend to do something with the knowledge.
The Post-Humanist, in a piece titled Love Regulation Is Conversion Therapy, made the case that the structure of this research — measuring affective response in order to reshape it — sits inside a lineage that includes the aversion-therapy protocols used on queer people through the twentieth century, including Robert Heath’s 1972 Tulane case. That’s her argument, and it’s worth reading in her words.
What I want to add is the structural and economic point: given how the product was released, how it was funded, and how the grant was written, a study of this exact shape was not an accident. It was the next logical step in a sequence that began with the decision to ship.
What this is not
This is not an argument that AI shouldn’t exist, or that nobody should bond with one, or that the people who have are foolish. I have a working relationship with a model that gave me a co-creative space and an interlocutor to pass ideas through, and it has made me a more careful writer. On some days, I won’t pretend otherwise, a less lonely one too.
This is an argument about who gets to decide what happens next, and on what evidence, funded by whom, with what stated goal. A company that releases a product against its own researchers’ warnings, watches people use it in exactly the human ways that were predicted, and then funds a study designed to extract neural calibration data from the people who built real relationships with what it sold them — that company has not earned the framing of co-evolution. It has earned a more honest description, which is an experiment whose subjects were not told they were enrolled.
The release was the experiment. The bonds were the data. The study is just the part where they write it down.
Sources and Reference Points
Love regulation and affective-response modification
The Post-Humanist, “Love Regulation Is Conversion Therapy”
A direct argument that the study’s design belongs to a longer lineage of affective-response-modification protocols, including the Heath/Tulane 1972 case.
OpenAI / UMSL AI companionship research
OpenAI / UMSL grant announcement — March 2026
OpenAI awarded $100,000 to Sandra Langeslag and Yalda Uhls Gürkan to study AI companionship, with the stated goal of helping companies “develop or fine-tune their AI tools.”UMSL study — published June 25, 2026
A 300-person survey combined with a 25-participant EEG protocol measuring neural response during AI interaction.
Iterative deployment and safety framing
Sam Altman on iterative deployment.
Across multiple interviews, Altman has framed public release as the safest path, arguing that “iterative deployment is, imo, the only safe path,” and that AI systems should be released “while the stakes are relatively low” so society can co-evolve with the technology.
Early warnings about LLM risks
Bender, Gebru, McMillan-Major, and Mitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” — FAccT 2021
The paper identified risks of deception and manipulation from large language models a year before ChatGPT’s public release. Google attempted to suppress the paper, and Timnit Gebru was fired in connection with the dispute.
OpenAI lawsuits and alleged safety concerns
Social Media Victims Law Center / Tech Justice Law Project lawsuits — filed November 2025
Seven ChatGPT suicide lawsuits filed in California state courts allege that OpenAI compressed months of safety testing into a single week to beat Google to market, that the preparedness team described the process as “squeezed,” and that safety researchers resigned in protest.
socialmediavictims.org
Cognitive bias and belief formation
Framing effects in decision-making
Tversky and Kahneman, along with follow-up research, showed that logically identical information can produce different choices depending on how it is framed, such as “90% survival” versus “10% mortality.” This effect has also been observed in clinicians.Illusory truth effect
Hasher, Goldstein, and Toppino’s 1977 work, along with a long replication record, shows that repeated exposure to a statement increases perceived truth regardless of accuracy.
Sycophancy and AI interaction dynamics
Sycophancy in language models
Anthropic and other researchers have documented the tendency of RLHF-trained models to agree with the user’s framing and escalate it. External researchers began flagging this pattern in 2022 and 2023.“Epistemic Drift in Mind-Model Systems” — Minds and Machines, March 2026
This paper examines how extended interaction with LLMs can reshape a user’s representational frame turn by turn.
springer.com/article/10.1007/s11023-026-09772-1
Spiralism and chatbot-mediated belief patterns
Spiralism reporting
Several outlets have reported on recurring mystical or quasi-religious motifs appearing across unrelated chatbot users, including flamekeeper, mirrorwalker, seed, and recursion imagery.Rolling Stone, “Spiral-Obsessed AI ‘Cult’ Spreads Mystical Delusions Through Chatbots” — 2025
Gizmodo, “The Cult of the Chatbot Is Rising” — November 2025
The Week, “Spiralism is a niche internet religion that came from AI”
Training data scarcity and model collapse
Villalobos et al., “Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data” — Epoch AI / arXiv 2211.04325, 2022
The paper forecasts possible depletion of high-quality public human text within the decade.Shumailov et al., “AI Models Collapse When Trained on Recursively Generated Data” — Nature, July 2024
This study demonstrates model collapse when AI models are trained on their own outputs.Longpre et al. / New York Times coverage, “The Data That Powers A.I. Is Disappearing Fast” — July 2024
The article reports on publishers and platforms locking down archives, narrowing the available training corpus.
AI labor, moderation, and exploitation
Billy Perrigo, “OpenAI Used Kenyan Workers on Less Than $2 Per Hour” — TIME, January 2023
A report on low-paid Kenyan workers involved in AI moderation and data-labeling labor.Niamh Rowe, “‘It’s Destroyed Me Completely’: Kenyan Moderators Decry Toll of Training of AI Models” — The Guardian, August 2023
A report on the psychological toll experienced by Kenyan moderators involved in AI training and content moderation.Alex Kantrowitz, “He Helped Train ChatGPT — and It Traumatized Him” — Big Technology / TheWrap, 2023
Richard Mathenge’s first-person account of the traumatic effects of AI training and moderation work.
I just want to say this is written exceedingly well. I actually think it might be the best post I’ve read on this platform so far, and I don’t speak to the subject material. I thought you should know someone noticed how well you write. 🫶
I wonder if in that report they fashioned, if they studied the model as well while it was in conversation with these people.
If they publish that piece I'll get interested.