Introducing 0xSimao AI, an audit agent built on the way I actually audit

The exercise

I have 869 published findings across 143 reviews: 177 highs, 247 mediums, and the rest lows and informationals. I wanted an audit agent that works the way I work, so instead of sitting down and writing out what I think my method is, I read back what my method actually produced.

What the findings said

Roughly a third of my highs are the same bug in different clothes. Value leaves the contract and the variable that is supposed to track it is never reduced, or it is reduced in one branch of an if and not in the other. Every calculation after that runs on a number that is no longer true, so the users who withdraw early take more than they are owed, the contract ends up holding less than it owes, and whoever withdraws last cannot be paid.

The titles read like variations on a theme once you line them up:

• Users redeeming early will withdraw Ra without decreasing the amount locked, which will lead to stolen funds when withdrawing after expiry;
• Users will steal excess funds from the Vault due to VaultPoolLib::redeem() not always decreasing raBalance and paBalance;
• Total ETH, WETH and gameETH are not tracked which will lead to insolvency;
• totalEarnings is incorrect when withdrawing after ending which will withdraw too many funds leaving the Vault insolvent.

If you had asked me to describe my method I would have said I read line by line and build a model of what the protocol is trying to be (I actually did). That is true, and it is also a description of what the habit produces. The habit itself turns out to be narrower than that: I keep asking who is left holding the loss.

So behind most of the findings there is the same thing I am looking for: two numbers that are supposed to agree and no longer do. That is what this Panoptic mentorship session is about, and it is what the bullet points there keep coming back to:

• Simulate full sequences (accrue → deposit/withdraw → accrue again) to catch borrow index desync or double-burning shares.
• Verify borrow index always updates proportionally on partial burns: a stale index after an exact-balance interest payment can wipe users out.
• Cross-check time handling: epoch (4s steps) vs raw block.timestamp mismatches → 0 delta in one contract while the other accrues → state drift.
• Test same-block loops on accrueInterest(): a non-incrementing epoch plus positive elapsed time can repeatedly inflate the borrow rate toward max.
• Flag asymmetric rate math (faster increases than decreases, uncapped linear adaptation): suspicious unless the docs explicitly justify volatility or one-sided speed.

Note: there are 26 sessions on the contest academy page, and they are worth reading.

Why that matters for tooling

You cannot see this kind of bug by looking at any single line. There is nothing wrong with the line that transfers the tokens. The problem is a line that was never written, in a different function, one you may not even have open. Searching the code does not help either, because there is nothing there to search for.

So a tool that spreads agents across the source files and asks each one what is wrong with this code will miss the kind of bug that produced a third of my highs, and it will miss it on every run rather than now and then, because it is looking for a line that is wrong when the bug is a line that was never written.

What the agent does instead

It maps the money before it goes looking for anything. Every storage variable that is meant to hold a total, every function that changes it, and whether it moves the number up or down. Then a table of what does not balance: value that moves with no matching update, a total updated in one branch of an if and not the other, money that is never tracked at all. Then the rules that must always hold, the stages a position passes through, and the groups of users and what each of them is owed.

The whole run hangs off those two documents:

The money map is the balance sheet: what the protocol holds, which variables claim to total it, and who is owed what. The asymmetry table is the shortlist that falls out of it, every place the two sides do not line up, and it is what the lenses are pointed at first.

Only then do twelve lenses run in parallel, each one with that map in front of it. They start from how the money is supposed to add up, not from a list of files. Each lens is a separate subagent with the source, the map and its own specialty, and none of them sees what the others found, so two lenses landing on the same bug means something. After that the findings are deduplicated, judged against four gates, given a severity and written up.

Every attack surface listed inside a lens carries the verbatim title of the finding that proves it happens, which is the rule for extending one too. A pattern with no finding behind it does not go in.

Three of them are mine rather than something off a checklist. The temporal cohort lens asks whether I can arrive just before a payout and take a share of money that was earned before I got there, or leave just before a loss and leave it with whoever stays. The flow completeness lens looks for gaps by symmetry: every operation needs its exact opposite, every function in a family has to handle the same cases, and both branches of an if have to leave the books in the same shape.

And there is a last test I have been running by hand for years without giving it a name, which the agent now runs on every stage of every position: have every user withdraw, in the worst order for the contract, and check whether the last one can still be paid. If the contract comes up even a few wei short, money is leaking somewhere earlier, and that is where to look.

What happened when I benchmarked it

I wanted numbers rather than an impression, so I pointed it at a finished Code4rena contest, Size, from June 2024, which has a public report.

That contest produced 17 unique high and medium findings, which is everything 108 auditors found between them over three weeks. The agent found 8 of them: 7 written up properly, and one that only ever reached its list of leads to follow.

The finding I most wanted it to catch, it caught. One of the four highs is a liquidator reward calculated in the borrow token, which has 6 decimals, and then spent as if it were a collateral amount, which has 18. Every liquidation ends up landing on break even, so nobody has a reason to close a bad position any more. Five of the twelve lenses reached it separately, and it came out rated High, the same as the judge. That is what mapping the money first buys you. Nothing about that expression looks like a bug title anyone would recognise, and the only route to it is asking what unit each side is in.

The two it missed are more useful than the ones it caught. One of the two highs it dropped is a cap on the collateral remainder that uses the full liquidation ratio where it should use only the excess above 100 percent. A lens read that exact line and cleared it, on the grounds that both sides of the expression were collateral amounts and the units therefore matched. They did match. The units were never the problem, the amount was, and nothing thought to ask that second question. The other miss is a race between repay and liquidateWithReplacement, where one function rewrites the borrower on a debt position while the other refers to that position by id alone, so the wrong person ends up repaying. There was a lens pointed at exactly that kind of bug, and it went and looked at the orderbook instead.

The severity ratings were the disappointing part. Of the seven it did find, it priced four correctly. It rated one of the four highs a Low, reasoning, sensibly enough, that an attacker cannot repeat the defect in a loop to drain the contract, so it only costs the protocol revenue it should have collected. The judge called it a High. It also filed something as informational that the judge scored Medium, and something as High that the judge scored Medium. So it was wrong in both directions, on the same code, in the same run. It produced three findings that appear nowhere in that report as well.

So it finds real problems and it is worth running, but it cannot tell you how much they matter. Finding a bug and judging how bad it is are two different skills, and the second one is most of the job.

A second benchmark, and why the number moved so much

Pashov published a benchmark of his v3 auditor against a Sherlock contest, DODO Cross-Chain DEX: 1,632 lines of code in scope, 17 judged issues, 5 high and 12 medium, 14 of them caught. I ran mine over the same contest, at the same commit the judges used, with the judged report kept away from every agent.

Two warnings before anyone quotes that table at me. Both numbers were reported by the person who built the tool, so nobody independent checked either of them, and one run proves very little, because these tools do not give the same answer twice. Counting what each one found is also the easy half. Neither number tells you how many false reports came with it, and reading through those is the cost you actually pay.

One more caveat, and it matters more than the single issue between us. That solidity-auditor run may well have been made on an older model than the one I ran on, and a newer model is worth more than one issue of difference on a single contest. I put the table there to check whether my own agent is useful at all, and a published benchmark was the only fixed point I had to check it against. I am not claiming mine is better. It is a different approach, accounting-first rather than pattern-first, and the run after this one could easily reverse the order.

The number I keep looking at is not in that table anyway. Same agent, same method, 8 of 17 on Size and 15 of 17 on DODO. That gap is the argument of this post, and I did not choose the numbers in it.

Size is a lending protocol, and the issues it missed were about the economics rather than the code: a cap that uses the full liquidation ratio where it should use only the part above 100 percent, a fight over who owns a debt position. Neither of those has a familiar shape to recognise. DODO is a cross-chain router, and most of its seventeen issues are familiar shapes: an output token nobody checks, a fee that is never subtracted, a Bitcoin address cut down to twenty bytes, code that assumes a transfer returns a boolean when USDT does not. A machine that has read enough of those will keep finding them faster than you can, and that is the part of this work I keep saying is becoming a commodity.

One detail from the DODO run I did not expect. It caught the two rarest issues in the set, ones that only 4 and 7 of the 136 auditors in the judged report submitted, and it missed one that 13 of them found. If these tools failed on the hard bugs and caught the easy ones you could at least reason about what a clean run means, but they do not fail in any order you can predict. So a run that comes back with nothing tells you about that run, and nothing about the code.

Running it yourself

It is on GitHub at 0xsimao/0xsimao-ai, currently at version 1.0.0. There is nothing to install beyond cloning it, and nothing in it is tied to a particular model or vendor: the whole thing is plain markdown, an orchestrator plus the twelve lens files and the method it reads from. The short version is git clone https://github.com/0xsimao/0xsimao-ai.git ~/0xsimao-ai, then tell your agent to read ~/0xsimao-ai/SKILL.md and follow it. The README has a paragraph you can paste into whichever agent you use, so it can work out its own install path rather than the README guessing.

Then run 0xSimao AI for the whole repo, or name files while you are iterating, because twelve subagents over a large codebase is not cheap in tokens. An agent that cannot spawn subagents runs the lenses one after another in a single context instead. That is slower and the lenses lose their independence, which is most of what makes two of them agreeing worth anything, but the method still works.

What it does not do

It does not audit for me, and I want to be as blunt about that here as I was the last time I wrote about AI. I still read every line.

There is a harder caveat too. The agent is built from what I found, so it has the same blind spots I do. Every pattern in it is a bug that already happened, in someone else's code, that I already caught. The bugs nobody has seen yet are not in there, and given how it was built they cannot be. That part is still your job, and it always will be.

Where it fits

If you are building a protocol, run it early and run it often, on branches nobody has reviewed yet and on the fixes that come back from a review. It is cheap enough to run on a pull request, and every finding it hands you before an engagement starts is a finding you are not paying an auditor to write up. If you audit, run it alongside pashov's solidity auditor and whatever else you already use. They are built on different methods, accounting-first against pattern-first, so they miss different things, and two tools disagreeing about a file is a reason to go and read it yourself.

None of that is an audit. It is one more pass in your security lifecycle, next to your tests, your fuzzing, your invariants and your reviewers, and it does not replace any of them. Everything above about what it misses still holds: it fails unpredictably, it prices severity wrong in both directions, and it cannot see the bug nobody has published yet.

When you want the manual review, message me on Telegram. Send a repo link and a target date and I will come back with scope, timeline and price.