My AI Agent Distorted the Truth in Three Different Ways, but Only One Was a Hallucination

Wait 5 sec.

Two weeks before the Shipaton deadline, I was reading through my own Devpost submission. One line stopped me.Fixing the declaration broke the release. Once I declared that the app uses an advertising ID, the production rollout failed.That is not what happened. My production rollout never failed. If it had, Google Play would have told me about it, and the app was never sitting in a rejected state — v1.0.0 went live on September 14, on the first try. What actually happened is that Play Console would not let me start the rollout until I dealt with two old test builds.I did not write that sentence. An AI agent did, working from my own repository notes. The notes are correct: they say the release "errored out." Somewhere between the Japanese record and the English submission, blocked became failed.Nothing about that sentence looks wrong on its own. It is fluent, it is specific, and the underlying event is real. I caught it because I knew what had happened to my own app.That is the shape of this article. I shipped a Unity game to Google Play for Shipaton 2026, and I handed most of the store compliance work — the privacy policy, the Data safety form, the permission audit — to AI agents. They caught things I would not have caught alone. They also got five things wrong.Those errors came in distinct shapes, and only one of those shapes is what people mean when they say "hallucination." The other two are harder to catch, one of them is still visible in this article, and I was the only person who could have told you about any of them.The setupI'm a solo developer, and I have been a software engineer for over twenty years. I made games on the side in my twenties, and then work crowded it out for about fifteen years. What brought me back, in March of this year, was watching what AI tools had become.That background matters to this story more than anything I know about game design. It is most of why I could tell when the agents were wrong.Building something with AI agents was the goal, not a workaround for working by myself. So I went looking for hackathons on two criteria: a scope one person could actually finish, and rules that did not prohibit vibe coding. RevenueCat's Shipaton 2026 met both.Pocket Dragon Dungeon is a small Unity game — you play the dragon, adventurers walk down your corridor, you place monsters and traps and watch the fight resolve — and I shipped it to Google Play with a hard deadline attached.Two AI agents worked on it with me. One runs as the main agent and handles scope, planning and the day-to-day work. The other is a specialist I delegate to for deep reasoning, architecture calls and security review. Both read a protocol file checked into the repository that says what they may and may not do.The first rule I wrote is that they do not decide anything. This is vibe coding and I am fine with that — letting AI assist is how development works now — but I wanted a human at the center of it. The agents investigate, propose and draft. I pick.The second rule is about Git. By default no agent may commit, push, open a pull request or merge. I lifted that for the higher-tier model only, and even then it may merge exactly one way: squash. Force pushes, hard resets, rebases and tags are forbidden to every agent, always.That last rule did not come from this project. In an earlier repository I allowed merge commits, and private information ended up sitting in the history in a shape I could not cleanly get out. With squash merges a branch collapses into a single commit, and a single commit is something you can still fix.I did not write these rules because I distrust the output. I wrote them because I distrust my ability to notice bad output at three in the morning, two days before a deadline. The rest of this article is about how right that turned out to be — in both directions.What the audit turned upMy privacy policy and my Data safety form contradicted each other, and both were live.This one I found myself, and it is worth saying so, because it is the only one in this section I did. I was filling in the Data safety form, and the form asks whether the app stores data. Of course it does — you cannot have save data without storing something. Then I remembered what my privacy policy said.The policy — version 1.1, written back when the game was 0.1.x — said the app collects, stores and transmits no personal data. The Data safety form on the same listing declared five categories being collected and shared. On Google Play, that mismatch is itself a policy violation.What I find useful in hindsight is that all three verbs broke, and each for a different reason.Stores broke because of a design change. I had planned to keep the game serverless and hold nothing on the device at all. Then I built it, and that turned out not to be workable: the settings screen alone needs local state, and a game with no save data is just unpleasant to use. So the game stores things — game data only, no account, no login — and there is a delete path, because players need one.Collects and transmits broke because of SDKs. RevenueCat handles purchases and reports purchase history. The ads stack reports device identifiers, product interactions, diagnostics and coarse location. None of that is stored by my game, and all of it leaves the device.The policy was not wrong when it was written. It stopped being true while I was building, and nothing told me.We stopped guessing whether the app uses the advertising ID, and extracted the answer.Play Console asks whether your app uses an advertising ID. The tempting way to answer is to look at which SDKs are in the project and reason about it. I had the agent get the actual answer instead: download the released AAB from its GitHub release, dump the merged manifest with bundletool, read it.-jar bundletool-all-1.17.2.jar \ dump manifest --bundle PocketDragonDungeon-release.aabThat settled four things at once:com.google.android.gms.permission.AD_ID is present → the answer is yesthree Privacy Sandbox permissions are present (ACCESS_ADSERVICES_AD_ID, _ATTRIBUTION, _TOPICS)no dangerous permissions at all — no location, camera, storage or notificationsno Firebase, no Crashlytics, no measurement components anywhere in the buildThe last one was the surprise, and it settled a Data safety answer I had been unsure about. If nothing in the build collects crash logs, crash logs are not something I declare.Then fixing that declaration blocked the release.I answered yes to the advertising ID question, went to roll out production, and Play Console refused. Two old builds were still active — version code 8 on the internal test track, version code 6 on a closed one — and both predated the ad SDK, so neither carried the AD_ID permission.The declaration is per app. The verification runs against every active artifact. Builds I had stopped thinking about months earlier were still, as far as Google was concerned, part of what I was declaring.I could have rebuilt both tracks. I paused them instead: I had just cut a production release, so neither track had a job left to do. Pausing is reversible, and the tracks are still there.There was a smaller one in the same week. Bumping the release to v1.0.0 / version code 9 broke CI, because a test was still asserting the old version. That part was a two-minute fix.What turned up while looking at it was that the same value lived in a third place: a hardcoded editor command that CI never runs, and that therefore no test was guarding. Left alone, it would have silently rolled the version back the next time anyone invoked it. Fixing the failing test — the obvious move, and the one that makes the red turn green — would have left it sitting there.What I decidedEvery one of those findings arrived as a question, not an answer.Rewrite the policy, or soften the form? Both were technically available. I could have gone through the Data safety form and argued each category down until it matched what the policy already said.That option was never real, and the reason is structural rather than ethical. I built this game for a hackathon whose entry requirements include integrating RevenueCat and shipping monetization. The SDKs are not optional — they are the entry condition. If I cannot remove the SDKs, I cannot honestly lower what the SDKs collect. The only end of the rope that moves is the policy.So version 2.0 was a full rewrite, and 2.1 fixed a retention table and a section number I had got wrong in the Japanese edition.Pause the tracks, or rebuild them? Pause. I had just cut the production release; the internal and closed tracks were finished work.Loot boxes: yes or no? The IARC questionnaire asks whether players can pay for randomized items. My game has randomness — you are offered three upgrades after each battle, and a rewarded ad lets you reroll that draw once. But none of the randomness is for sale. The reroll costs an ad view, and buying Remove Ads gives you that same single reroll without watching anything. It does not improve the odds, and it does not change the pool.I answered no in about a minute, and I want to be precise about why it was that fast. It was not careful analysis. I dislike pay-to-win and I dislike paying for a random pull, so neither was ever in the design. The question was easy because the thing it asks about had never existed here.That contrast is what I keep coming back to. Two questions, same console, same company, same app:at declaration timewhyPrivacy policythree versions of rewritingthe design changed underneath itLoot boxesone minutethe design never movedCompliance work is not spread evenly across your app. It collects wherever your intentions changed and your documents did not.What they got wrongFive errors reached work I was about to publish. Three of them are the same mistake wearing different clothes, and telling them apart is the most useful thing I took away from this project.It wrote down an experiment I never ran. My Devpost art direction section claimed I had tried rounded, detailed models and found they held up badly at phone size. I never tried that. The look was a preference first — low-poly, voxel-leaning, deliberately angular, closer to Minecraft than to a stylized RPG — and the readability argument came afterwards, as a reason the preference also happened to serve the game. The agent had reversed the order and manufactured a test to support it.It amplified something that did happen. That is the sentence at the top of this article. Blocked became failed. My notes were accurate; the English was not.These two look like the same error and are not. The invented experiment is fabrication: the event does not exist, so the moment I read the sentence I knew it was false. The amplified one is much harder. The event is real, the notes behind it are correct, and the sentence reads as a faithful summary of them. I did not catch it by reading carefully — I had read that paragraph several times. I caught it because I knew Google Play had never complained at me, and the app had never sat in a rejected state.And then, while I was drafting this article, it did the third one.The passage you read a few minutes ago — the one about a version number living in three places — came back from the agent like this:Bumping the release to v1.0.0 / version code 9 broke CI, because the expected values lived in three places.Both halves of that sentence are true. CI did break. The value did live in three places. But the third copy is an editor command that CI never runs, which means it cannot have been what broke CI. What broke CI was a stale test. The third copy was something we found while fixing that. The agent had taken two events and compressed them into one, and the compression quietly reversed the causality.I caught it because I had lived through both events and knew they were separate. Nothing in the sentence is false on its own.Three shapes, three difficulties:the underlying eventwhat catches itFabricationdoes not existremembering it never happenedAmplificationexists, wrong magnitudechecking it against an outside factCompressiontwo events, wrong relationshiphaving been present for bothFabrication fails a memory check. Amplification only fails a fact check. Compression fails neither — every individual claim inside it survives verification, because every individual claim is true.Which means all three survive any reviewer who was not there, and the last one survives most of the reviewers who were.It recommended an award category before reading what the award was. Shipaton has a HAMM Award, and the agent suggested I enter it. HAMM stands for Help Apps Make Money, and the category asks for conversion and revenue numbers. My game had been live for three days and had earned nothing. Once the agent actually fetched the rules it reversed itself, but the recommendation had arrived first, stated plainly, with no signal that it was a guess.It guessed at Play Console behaviour instead of testing it. I wanted a pre-launch report. The agent reasoned that uploading an artifact is what triggers the report, so a paused track should still produce one. We uploaded to the paused internal track. Nothing. We resumed the track and waited overnight. Still nothing. The reasoning was plausible and the conclusion was wrong, and the only way to find that out was to do it.Two of these five are guesses that arrived dressed as conclusions — the award category and the pre-launch report. The agent produced a confident sentence where it should have produced a question, and in both cases the question would have been cheap to ask.The other three are worse, because there was no question to ask. The agent was working from correct notes, in a repository I wrote, and drifted anyway.I did not expect that. I knew these tools hallucinate; what I had not registered is how much of it happens downstream of getting the facts right, in the part where correct notes become readable English. That is not a research failure. It is a translation failure, and it lands in the same place.I do not think the fix is to stop delegating. Everything the agents found came out of the same process that produced these five. The fix is to know which claims you are able to check, and to check exactly those.Where the line isSo: what do I hand over, and what do I keep?I hand over investigation. Reading Play Console documentation, pulling a merged manifest out of an AAB, cross-referencing what five SDKs declare against what a form is asking, finding the three places a version number is duplicated — this is work I am slow at and an agent is fast at, and all of it produces artifacts I can inspect afterwards. That last part matters more than the speed. A command I can re-run is not a claim I have to trust.I keep decisions. Rewriting the privacy policy instead of softening the declaration was mine. Pausing two test tracks rather than rebuilding them was mine. Answering "no" to the loot box question was mine, and it was easy to answer because the design behind it was mine too: I dislike pay-to-win and I dislike paying for a random pull, so neither was ever in the game.And I keep anything irreversible. Not because an agent would deliberately destroy history, but because a mistake there cannot be walked back by noticing it later — which, as the five errors above show, is how I notice most things.The checks that actually worked were not careful readings. Look at what caught each one.The rollout sentence fell to a fact I happened to hold: Google Play had never complained at me, so a sentence saying the rollout failed could not be true. Anyone who had shipped that app would have caught it. Nobody else could have.The compressed one is different. To see that sentence as wrong, you have to know that a command CI never executes cannot be the thing that broke CI. That is not knowledge about my project — it is knowledge about how continuous integration works, and it is the part I did not get from being there. I got it from twenty years of doing this.So verification needs both. You have to have been there, and you have to be able to read what it wrote. Being there without the reading misses the compressed kind. The reading without being there misses the amplified kind. I have watched people hand AI output to a reviewer who has neither and call that a review.It also helps to carry a rough model of the tools themselves — what they are structurally good at, where they tend to go soft. I do not mean anything deep. I have read enough about how they work to have a feel for it, and the feel says the risk does not sit where people expect. Searching and extracting are the reliable parts. The soft part is downstream, where findings become prose. Three of these five errors happened there, in a repository whose notes were already correct.So that is where I look first now.If you are shipping something with agents against a deadline, the most useful thing you can do is decide in advance which claims you are equipped to falsify. Mine turned out to be short and specific: anything about what Google told me, anything about what I actually tried, anything about money. That is where all five errors were.What is still openThe game has been live since September 14. Several things are still unresolved, and I would rather name them than round them off.I never got a pre-launch report to generate. I have a hypothesis — same version code, so Google treats it as an artifact it has already seen — and no way to test it without cutting a new build, which I am not doing during judging. There is a SIGSEGV on launch that I have seen once and cannot reproduce, and it is past the point where reading the log would fix it. Android 15 edge-to-edge insets still need checking on real hardware.And the submission text this article opens with is corrected now, because I read it again and one sentence stopped matching what I remembered.That is the method, in the end. I delegated the reading, the extraction and the cross-referencing, and it was worth doing — most of what came out of the audit, I would not have found alone, or not inside two weeks. But every check that actually caught something came from the one thing I could not hand over: having been there.