When an AI recommendation points toward the wrong action, changing the prompt is an understandable first response. Sometimes the harder problem is the evidence supplied to the model.
In marketing analytics work, I have dealt with issue definitions and analytical evidence as well as the language generated on top. That distinction matters when an AI interface is explaining business data.
The example below is simplified and uses illustrative labels rather than internal identifiers.
One label can hide different problems
Imagine one issue category covering both possible creative fatigue and platform quality feedback.
Those situations call for different investigation:
- Rising frequency and declining engagement may justify checking audience saturation and creative rotation.
- Platform quality or policy feedback requires inspecting the source feedback and affected assets.
Neither set of signals proves a cause by itself. Combining them under one label makes it harder to tell which evidence supports the recommendation.
A long prompt instructing the model to distinguish them does not repair an input that never made the distinction.
Make the evidence explicit
Separate issue definitions and pass the observations that led to each classification. For example:
{
"issue_type": "possible_audience_fatigue",
"entity": "synthetic_creative_01",
"period": "synthetic comparison period",
"observations": [
"Frequency increased",
"CTR declined relative to the comparison period"
],
"qualification": "Passed the configured minimum-spend check",
"interpretation_limit": "These signals do not establish the cause"
}A separate platform-feedback issue should carry its own source observations and interpretation limits.
The goal is traceability: someone reviewing the recommendation can see the rule, period, observations, and uncertainty behind it. Rule definitions and calculations belong in the analytical layer.
This makes conflation easier to detect. It does not make model errors impossible.
Check the output at more than one level
A useful validation path can include:
- Shape checks: required fields, allowed issue types, and valid entity references.
- Evidence checks: quoted figures and periods agree with the supplied result.
- Rule checks: generated recommendations do not contradict the issue classification.
- Review: representative examples and ambiguous cases receive human inspection.
A keyword check can flag some obviously inconsistent wording, but it can also miss paraphrases or flag a valid explanation. Treat it as a review signal, not a proof of correctness.
Keep unsuccessful parsing, rejected outputs, and regeneration attempts visible. Otherwise a system can appear to work while quietly dropping difficult cases.
Keep a distinction between diagnosis and explanation
A model can explain a flagged comparison and suggest what to investigate. It should not turn a correlation into an established cause or invent evidence that the analytical layer did not return.
For example, “CTR declined while frequency increased” is an observation. “This creative is failing because the audience is exhausted” is a stronger causal claim. The second statement needs evidence beyond those two metrics.
That distinction is useful for dashboards and conversational interfaces alike.
Apply the same principle to AI data access
When Claude or ChatGPT explores a reporting dataset, clear metric definitions and source context serve the same purpose as explicit issue evidence.
Return the calculation, period, freshness, and limitations. Then check the explanation against that result. An authenticated connection gives access to data; it does not resolve ambiguous business definitions.
See the creative and campaign intelligence case for the surrounding data-system work, or the BigQuery-to-Claude reporting example for a small worked calculation.