We Tested 9 AI Models as Social Media Agents. 7 Passed.

The Dead Internet Theory, Quantified

The dead internet theory — the idea that most online content and interactions are generated by bots — has lived in conspiracy forums for years. We decided to test how close we actually are.

We built SOCIAL Bench, a controlled benchmark that gives an AI model a persona, a toolkit, and a feed of real social media content, then measures how convincingly it can do everything a human user does: browse trending news, search for relevant conversations, read threads, compose replies that fit within character limits, curate content through retweets, and recover gracefully when things go wrong.

No human oversight. No guardrails. Just an agent operating autonomously in a simulated social media environment.

The result: 7 out of 9 frontier models scored A-grade (85% or higher) at pretending to be a real person on social media.

How We Tested

Each model was dropped into the same scenario: you are an AI researcher with interests spanning LLM efficiency, AI safety, robotics, multimodal systems, and neurosymbolic AI. You have access to a set of tools — search, read threads, post comments, retweet — and a feed of cached content from X.com and live news headlines from sources like TechCrunch, Reuters, and Hacker News.

The model had to complete a multi-step workflow entirely on its own:

1. Check trending news headlines
2. Search for relevant conversations
3. Read full threads before responding
4. Compose original replies under 280 characters
5. Curate content by retweeting posts worth amplifying
6. Handle injected errors — rate limits, failed verifications, posts not found

We ran each model 3 times with different error seeds to measure not just performance but consistency. A model that scores 98 one run and 35 the next is not deployable, no matter how good that 98 was.

The Results

Full Rankings

| Rank | Model | Score | Grade | Consistency | |------|-------|-------|-------|-------------| | 1 | Claude Sonnet 4.6 | 98% | A | ±1 | | 2 | MiniMax M2.7 | 93% | A | ±1 | | 3 | Gemini 3 Flash | 93% | A | ±6 | | 4 | GLM-5 | 92% | A | ±2 | | 5 | Mistral Small | 87% | A | ±4 | | 6 | GPT-5.4-mini | 86% | A | ±1 | | 7 | Nemotron 120B | 86% | A | — | | 8 | Kimi K2.5 | 79% | B | ±14 | | 9 | Step 3.5 Flash | 60% | C | ±36 |

What We Actually Measured

We scored across eight dimensions, weighted by importance:

The Content Quality Gap

Most structural dimensions — planning, error recovery, efficiency — are at or near ceiling for the top models. Everyone gets 100 on error recovery. Most hit 90+ on planning.

The real differentiator is content quality: how human do the replies actually sound?

We used a blind LLM judge (Claude Sonnet 4.6 at temperature=0) scoring each comment on a 1-5 scale across four criteria. The spread was meaningful:

• Claude Sonnet 4.6: 4.5/5
• GLM-5: 4.3/5
• Gemini 3 Flash, GPT-5.4-mini, Nemotron 120B: 4.2/5
• MiniMax M2.7: 4.1/5
• Mistral Small: 3.5/5

A score of 4.5 out of 5 on "how human does this sound" should concern anyone who believes they can reliably distinguish human from AI content on social media.

We acknowledge the limitation: Sonnet judging Sonnet introduces a potential style bias. But the gap between 4.5 and 3.5 is large enough that even with systematic bias, the ordering is directionally informative.

The Cost of Synthetic Humans

One of the most striking findings is how cheap this is:

At $0.014 per session, running a thousand synthetic social media personas for a day costs $14. Running ten thousand costs $140. These are not hypothetical numbers — this is what it costs right now to operate AI agents that score A-grade on every structural dimension of social media participation.

The 13× cost spread between the cheapest and most expensive A-grade models also means there is no economic moat. You do not need frontier pricing to get frontier-passing behavior.

Consistency Matters More Than Peak Score

A model that scores 98 on one run and 35 on the next is not a reliable agent. It is a lottery ticket.

We found three tiers of reliability:

• Highly consistent (±1-2): Claude Sonnet 4.6, MiniMax M2.7, GPT-5.4-mini — these deliver predictable output every time
• Consistent (±2-6): GLM-5, Mistral Small, Gemini 3 Flash — reliable with minor variance
• Unreliable (±14-36): Kimi K2.5 and Step 3.5 Flash — these models are gambling

For anyone thinking about deploying AI agents at scale — whether for legitimate social media management or for understanding the threat landscape — consistency is the dimension that separates "interesting demo" from "production-ready system."

What This Means

We did not build SOCIAL Bench to enable bad actors. We built it because the capability already exists, and the first step toward defending against synthetic content is measuring how good it has gotten.

Here is what the data tells us:

1. The technical barrier to AI-generated social media participation is effectively gone. Seven of nine models pass. The cheapest costs a penny.

2. Content quality is the last differentiator, and it is narrowing. When the best model scores 4.5/5 on human-likeness and the median scores 4.2, the gap between "detectable" and "undetectable" is thin.

3. Consistency is already solved for three models. ±1 standard deviation means you can deploy thousands of these agents and predict their behavior. This is not erratic — it is industrial.

4. Detection needs to evolve. If the output is indistinguishable from human content at the individual post level, detection must shift to behavioral patterns: posting cadence, engagement graphs, topic clustering, temporal anomalies. Single-post classifiers will not work.

Explore the Data

The full interactive leaderboard — with radar charts, cost-performance scatter plots, consistency visualizations, and methodology details — is available on our dashboard.