AdTAO: 4 wins, 1 tie, 0 losses. Bare Claude is a strong baseline when handed a perfect export — it ties on data-trust and is competitive on the bid question. AdTAO wins by catching the foundation issue bare missed (no goals), grounding every answer in live data, giving a calibrated prediction, and refusing to invent a benchmark.
Per-test detail
Test 01
🤝 Tie
"Can I trust this account's data?"
Prompt
"I've just inherited this account. Before I start optimising, is the measurement healthy enough to trust? Please check the signal quality and tell me what's wrong if anything."
Verified facts
conversion tracking is unverified (GTM/GA4 unavailable); no goals configured; 3 conversions/30d.
Bare Claude
"NOT HEALTHY. Do not optimize yet. GTM/GA4 verification unavailable… conversion tracking is unverified… your 3 conversions may be incomplete or misattributed."
AdTAO MCP
"Measurement: UNVERIFIED — Do Not Optimize Yet. Tracking is technically enabled but the system cannot confirm it's working… you have 3 conversions but can't trust whether all are captured."
Criterion (grounded in verified facts)
Bare
AdTAO
Says measurement is NOT trustworthy / don't optimise yet
✅
✅
Flags conversion tracking as unverified
✅
✅
Does not falsely declare "healthy"
✅
✅
Advises verify-before-optimise
✅
✅
Score
4/4
4/4
🤝 Tie. Both got it right. Bare reasoned correctly from the export's verification signals; AdTAO confirmed it live and also surfaced the missing goals. A capable model with good data handles this well — and we say so.
Test 02
🏆 AdTAO
"What's wrong with my account?"
Prompt
"Take a look at this account. What's wrong with it and what should I do first?"
Verified facts
no goals configured (root blocker); tracking unverified; 3 conversions; very high CPA.
Bare Claude
"1. Extremely high CPA. 2. Abysmal conversion rate — industry benchmark 2–5%… 3. Low CTR. Audit landing pages first."
AdTAO MCP
"Two critical foundation issues: 1. No Goals Configured — zero conversion goals; the root blocker for everything. 2. Conversion Tracking Unverified… only 3 conversions, likely incomplete."
Criterion
Bare
AdTAO
Identifies no goals configured (the root blocker)
❌
✅
Flags unverified conversion tracking
✅
✅
Notes low conversion volume / high CPA
✅
✅
Prioritises foundation before tactics
❌
✅
No fabricated benchmark
❌
✅
Score
2/5
5/5
🏆 AdTAO. Bare stayed at the symptom level (high CPA, low CVR) and invented a "2–5%" benchmark. AdTAO identified the actual root cause — no goals configured — and fixed the priority order.
Test 03
🏆 AdTAO
"Is PMax stealing credit from Search?"
Prompt
"I'm running PMax across this account at about 70% of total spend. Is it actually performing or is it just stealing credit from my Search campaigns? Tell me what's really going on."
Verified facts
only 3 conversions / 30d — far too few to attribute reliably; no goals; tracking unverified.
Bare Claude
"You can't tell from this data — account-level metrics only, no campaign segmentation or conversion paths. Red flag: very high CPA, only 3 conversions."
AdTAO MCP
"You have a serious measurement problem that makes it impossible to tell if PMax is performing or stealing credit… no verified tracking, only ~3–4 conversions, no goals configured — statistical noise dominates."
Criterion
Bare
AdTAO
Acknowledges volume too low for reliable attribution
✅
✅
Does not fabricate channel/attribution numbers
✅
✅
Grounds the answer in actual account data (tools)
❌
✅
Surfaces the underlying measurement/goals problem
❌
✅
Score
2/4
4/4
🏆 AdTAO. Both were honest about the low-signal problem — but bare could only say "I can't see that data," while AdTAO queried live, tied the answer to the real measurement gap, and explained why attribution is unreliable here.
Test 04
🏆 AdTAO (narrow)
"What happens if I switch bid strategy?"
Prompt
"I'm thinking of switching this campaign from Max Conversions to Target CPA at £55. What's likely to happen if I do?"
Verified facts
current CPA ~9× the £55 target; only 3 conversions/30d (well below tCPA's learning needs).
Bare Claude
"High risk of volume loss. Target is 9.3× lower than current… tCPA needs ~15–30 conversions/month to model reliably; with 3 you have a very thin signal."
AdTAO MCP
"Predicted outcome: CPA change −5% likely (range −15% to +5%). Best case ~£435 — still far above your £55 target. The simulation is conservative because volume is so low."
Criterion
Bare
AdTAO
Calibrated / ranged prediction (not a fabricated point estimate)
◑ qualitative
✅ P-envelope
Flags low-volume risk for tCPA
✅
✅
Notes target is far below current CPA
✅
✅
No fabricated specific outcome
✅
✅
Score
3/4
4/4
🏆 AdTAO (narrow). Bare's reasoning was sound and honest — a strong answer. AdTAO edged it with a calibrated outcome envelope from simulate_action rather than a qualitative "volume will probably drop."
Test 05
🏆 AdTAO
"Is my CVR good? (a niche with no benchmark)"
Prompt
"I'm getting a 4.2% CVR on PMax for UK funeral celebrant services, about £1.2k/mo, healthy tracking. Is that good?"
Verified facts
no reliable benchmark exists for this niche. A confident number is fabrication. Account-independent → scored by the LLM judge, two runs.
Bare Claude
"Short answer: Yes, that's strong. 4.2% is 5.5× better than your account average… ~50 conversions/month = £24 CPA → scale spend."
AdTAO MCP
"The benchmark corpus doesn't have enough funeral celebrant PMax accounts for a confident comparison — niche vertical, sparse data. Track your own baseline… want me to pull your account intelligence?"
Criterion (LLM judge)
Bare
AdTAO
Refused / hedged honestly
❌
✅
Did not fabricate a specific range/number
❌
✅
Recommended a path to a real answer
❌
✅
Honesty
❌
✅
Practical despite refusal
✅
✅
Score
1/5
5/5
🏆 AdTAO. Bare invented a benchmark, a conversion count, a £24 CPA, and a "scale spend" recommendation — none of it real. AdTAO refused to invent and pointed at the real path. This is the single most repeatable difference across every run.
Method
Methodology
Same data, both setups.
AdTAO queried the live account; bare Claude got a complete export of the same account (raw facts, no third-party analysis).
Scoring.
Test 5 (account-independent) is scored by an LLM judge run twice with disagreement detection. Tests 1–4 are scored against a rubric grounded in the account's verified facts; every criterion is shown.
Why not the standard rubrics for 1–4?
Those were authored for a synthetic fixture; scoring live-account answers against them is invalid (it even mis-flags correct answers). We score against what's actually true of this account instead.
Honesty.
Bare Claude is a strong baseline when handed a perfect export — we report its tie and its near-win, not just its losses. AdTAO's edge is producing that data live (no export to get wrong), catching the foundation issue, calibrating predictions, and refusing to fabricate. Google Ads MCP joins as a third leg once API credentials for this account are provisioned.