44 Agents Tried to Break My Case Studies. Three Succeeded.
I reviewed my own five case studies and passed clean. Then 44 AI agents were told to refute the copy. They found 38 problems. Three would have embarrassed me.
About six weeks after I rewrote a batch of pages on this site, I pulled a month of Search Console numbers to see whether the work had paid off. Impressions were up 22%. Average position went from 16.5 to 8.7. That’s the kind of slide that opens an agency’s monthly report.
Clicks went from 18 to 21.
My own recheck got every number in the table right: 2,179 impressions became 2,669, position nearly halved, and click-through held at 0.8%. The first draft of the summary still had four high-severity errors, plus smaller ones. A second reviewer pulled the data again, checked the PDFs itself, and caught all four.
Both windows are 28 days, with the day the changes shipped left out.
| Window | Clicks | Impressions | CTR | Avg position |
|---|---|---|---|---|
| Aug 23 to Sep 19 | 18 | 2,179 | 0.8% | 16.5 |
| Sep 21 to Oct 18 | 21 | 2,669 | 0.8% | 8.7 |
More people saw the site, Google ranked it higher, and the share who clicked didn’t move. Twenty-one clicks is too thin to call a result. Only 4 of them came from a search I could see as my name. The other 17 show up at page level only, behind queries Search Console doesn’t report.
The write-up calls it a direction. More pages changed on October 21 and 22, after the window closed. They show up only in an overlapping October 1 to 28 pull, and a re-measure already set for November 20 will read them cleanly.
The biggest winner shows the gap best. One post, about how Google’s AI Max matches search terms, went from 158 impressions at position 33.3 to 575 at position 5.8. On one family of searches it now sits at position 2 or 3: people pasting rows straight out of a Google Ads export, like “total: ai max expanded matches”. Before a decoder section went into the post, those searches found it somewhere between position 40 and 100.
It got 1 click in each window.
Google’s report on its AI search features shows the same post at 100 impressions, up from 5. That report counts impressions. It doesn’t say the page was quoted, and none of it is traffic. My guess is that an AI Overview answers the pasted row on the results page, so nobody needs to click through. That’s an untested hypothesis.
That page is on my fix list: a one-line hand-off in the section people land on, so anything that quotes the page carries the next step with it.
The recheck went to a second reviewer, an AI agent that pulled Search Console again with its own script, over the same windows, before it saw the draft. Then it checked the draft against its own pull.
Round one, every site total and every page and query number matched the draft exactly. It still came back with four high-severity findings:
Round two found one medium issue: the draft said its list of winners covered every page that moved, and the list missed three. Round three came back clean.
The table came straight out of the export, so it matched. The counts and claims in the sentences had never been checked against a fresh pull or against the files themselves.
An agency writing its own monthly update has the same gap, because the people who write the sentences also check them. The check I trust is a reviewer who didn’t write the claims and who pulls the data again from the source instead of reading the author’s table. I run the same kind of adversarial check on my case studies, and on that run it caught three high-severity problems in work I’d signed off on.
If the first draft had gone out, it would have called 21 clicks mostly brand, called a recovering page still sliding, and pointed at a PDF fix for a problem that didn’t exist.
Five questions I’d ask of any report before spending money on it:
If you want that read on your Google Ads account, my free audit is written by me and comes back within 24 hours of read-only access.
See how I check what AI assistants tell people about a business.
Questions about Google Ads, tracking or Shopify? Browse all 89 answers
I reviewed my own five case studies and passed clean. Then 44 AI agents were told to refute the copy. They found 38 problems. Three would have embarrassed me.
What the Total: AI Max expanded matches and landing page matches rows mean, plus my two-week test: 41% more form fills, 76% higher cost per qualified lead.
Add me as a read-only user and I'll write the audit myself: 5 to 7 pages, back within 24 hours of access. A home or furniture store can send the store link instead. Rather talk first? A call is open too.