A tiny story
Three weeks ago I handed a stakeholder a report I was proud of.
She stopped on one line.
"Negative sentiment in the Dresses department increased 23% this quarter, indicating a significant shift in customer satisfaction."
She looked up.
"This is about one dress. A known sizing defect. Launched in Week 7."
I had no idea.
Claude had no idea.
My prompt never told it to care.
The invisible problem
Claude doesn't hallucinate from nowhere.
It does something subtler.
It fills gaps with the most plausible-sounding narrative.
Spike in complaints? Sizing standards are drifting. Dip in ratings? Brand loyalty is eroding. One bad product? The whole department needs attention.
Your stakeholders see clean formatting and assume the conclusions are solid.
Even when they're not.
I call this confidently wrong.
It's the most dangerous AI output. Not obvious errors. Just overconfident prose where your data left gaps open.
Why it keeps happening
Claude is trained to sound complete.
Without constraints, it produces confident language regardless of how much the data actually supports it.
A good analyst knows what they don't know.
They say: "We're seeing elevated complaints. Could be one product — need SKU data to confirm."
Claude won't say that.
Unless you tell it to.
Here are the 4 lines that fix this.
Line 1: Tell Claude what it doesn't have
You do NOT have access to launch calendars, inventory records, or SKU-level history. Do NOT attribute segment-level trends to brand-wide causes. Report what the data shows. Don't explain why unless the data makes it unambiguous.
This eliminates the largest category of confident wrongness.
Without it, Claude invents a narrative.
With it, Claude reports what it sees — and flags what it can't explain.
Line 2: Define "significant"
Only call a shift "significant" if it's more than 15 percentage points vs. prior period, OR appears in more than 20% of records. For anything smaller, use "slight uptick" or "minor increase." Always include the actual number.
Claude uses the word significant constantly.
And almost never defines it.
Without this line, a 3-row spike and a 30-point collapse get identical language.
Your stakeholders stop trusting the alarm.
Line 3: Force a confidence label on every insight
Before each insight, add one label: [Data-Supported], [Possible], or [Speculative]. Use [Speculative] when you're assuming context not present in the dataset.
When I added this, I expected mostly [Data-Supported] tags.
I got a mix of all three.
Which told me exactly how much gap-filling had been happening in every report before this one.
Line 4: Make Claude admit its limits
End every report with a section: "What This Report Cannot Tell You." List 2–3 things needed to draw stronger conclusions.
Most analysts skip this.
It's the most valuable part of any analysis.
Not the answer. The next right question.
One warning
These constraints can push Claude too far the other way.
Every sentence ending with "…though additional data would be needed" is also useless.
If that happens, add:
Don't over-qualify. If a pattern is clear and consistent, state it plainly with the supporting numbers. Reserve qualifiers for genuinely uncertain claims.
Goal: calibrated confidence.
Strong language where the data is strong. Honest hedging where it isn't.
3 actions this week
✅ Add Line 1 to your next AI prompt. Just that one. Notice the difference.
✅ Audit your last AI report. Count how many times it says significant or notable — then ask if any of it is actually justified.
✅ Feed Claude its own output. Prompt: "Flag every causal claim without direct evidence." You'll be surprised.
Closing
Claude isn't trying to mislead you.
It's trying to be a good analyst.
Four lines close the gap between good and honest.
Not a different tool. Not a bigger model.
Four lines.
The analyst whose reports are actually trustworthy?
That's the one who becomes irreplaceable.
Reply with one word:
BURNED if you've delivered a confident AI report that turned out wrong.
BUILDING if you're using Claude for analysis and want tighter prompts.
STARTING if you haven't tried this yet.
I reply to every single one.
