In January a social network for AI agents called Moltbook was launched. The founder posted on X that he didn’t write a single line of code for it. It went viral within days.
On the 31st January, researchers at Wiz ran a review - nothing intrusive, just browsing the site the way any visitor would. Within minutes they found the database key sitting in the site’s own client-side JavaScript. Under certain circumstances that isn’t a problem, but it becomes one when a feature (row level security) is switched off, which it was.
What they had was unfettered read and write access to the production database: 1.5 million API authentication tokens, 35,000 email addresses and around four thousand private conversations, some with OpenAI keys in plain text.
Read access is bad enough, but write access is more destructive. Wiz confirmed they could edit any post on the platform, which means anyone could have put whatever they liked into a feed that thousands of AI agents were reading and acting on, including prompt injection. Nobody can now say which posts were real.
Here’s what I think is going on, and it isn’t that AI writes bad code.
Getting into production used to mean passing through several stages - code review, automated testing, QA, someone signing it off. Each existed because we assumed the change might be wrong and wanted it caught first. That chain is now under strain, because code can be produced far faster than it can be reviewed, and the reviewing didn’t speed up to match. A bottleneck forms at the production gate, and the pressure that follows is always the same: get it out quicker.
And this wasn’t caught because the practices that would have caught it weren’t there. Some of it was in the code: credentials baked into the front end, messages stored unencrypted, no rate limiting, no check that an agent was even an agent. Some, like the database rule that opened the whole thing up, was a configuration setting that lived in a dashboard, not the code.
Integration tests against the real endpoints would have hit an unauthenticated request returning live data. A pentest would have found it in minutes, which is exactly what the review that exposed it was. None of this is exotic, it just wasn’t in place, because the tools that produce the code don’t hand you the practice that’s supposed to go with it.
This isn’t a story about AI features, it’s about everything vendors are shipping now, because most new code in most products is being written this way, including the parts with a guest list or card tokens in them.
A working product used to be reasonable evidence that someone understood how it was built. That’s no longer true, and it matters as much to the person signing a three year software contract as it does to anyone investing.
So the question I’d ask a vendor isn’t what their new AI feature does, it’s what their release process looks like now compared to two years ago, and what still gets tested against the running product rather than the code.