Log a Hash of the Messages Array Before Every Call
Your AI agent is not dumb. Your code is editing its memory behind its back. The model answers perfectly in a clean playground, then goes sideways the moment it runs through your app. Before you file a bug against the model, look at the one thing you control: the messages array you send on every call.
Why the model is fine but your agent isn't
Chat APIs are stateless. The provider keeps no memory of your conversation — you own the full history and resend the entire messages list on every request (Anthropic and OpenAI both work this way). That design is clean, but it means every layer between your user and the model gets a chance to touch the array first: a summarizer that trims old turns, a retriever that injects context, a middleware that reorders system messages, a retry that drops the last turn. The model only ever sees the array you hand it. If that array is quietly wrong, the model's answer is quietly wrong — and it looks like the model hallucinated.
The one-line audit: hash the history
You can't eyeball a 40-message array on every call. So don't. Compute a stable hash of the messages right before you send them, and log it:
import hashlib, json
def messages_fingerprint(messages):
blob = json.dumps(messages, sort_keys=True, ensure_ascii=False)
return hashlib.sha256(blob.encode("utf-8")).hexdigest()[:12]
fp = messages_fingerprint(messages)
logger.info("llm.call turns=%d fp=%s", len(messages), fp)
response = client.chat.completions.create(model=MODEL, messages=messages)Now the rule is trivial to check: the hash should only change when a new turn is appended. If the fingerprint changes between two calls but no user or assistant turn was added, some layer rewrote history. sort_keys=True makes the serialization order-independent, so key ordering never triggers a false alarm — a changed hash means the content or shape of the array actually changed.
Reading the signal
| What you see | What it means |
|---|---|
Hash changes, turns count went up by 1 | Normal — a new turn was appended. |
Hash changes, turns count unchanged | A layer edited an existing turn in place. |
Hash changes, turns count went down | Something dropped or truncated history. |
| System message hash differs across calls | Prompt caching just broke — every call is a cache miss. |
| Same fingerprint, wrong answer | Now you can trust the array — investigate the model or params. |
That last row is the real payoff: the fingerprint lets you rule the plumbing out. Once the hash is stable and correct, you've earned the right to blame the model.
Power tricks
- Hash the system prompt separately. Log a second fingerprint for just
messages[0] (or the system field). A byte-identical system prefix is what prompt caching keys on; if that hash drifts — an interpolated timestamp, a request ID, a username — you're paying full input price on every call and probably don't know it.
- Diff, don't just detect. When two adjacent calls have the same turn count but
different hashes, log a compact diff of the array (or the per-message hashes) so you see which turn was mutated, not just that something was.
- Assert in tests. Turn the invariant into a test: run your wrapper, capture the array
at the boundary, and assert the fingerprint only advances when you append. This catches a context-mangling regression in CI instead of in production.
- Log at the boundary, not the top. Put the fingerprint call as close to the network
call as you can — ideally in the client wrapper itself. Anything logged upstream of a middleware can't see what that middleware did.
Resources
- Using the Messages API — the API is stateless; you own the conversation history
- OpenAI Chat Completions reference — the
messagesarray and roles - Python
hashlib— SHA-256 for a stable fingerprint - Python
json.dumps—sort_keysfor order-independent serialization - Reel: Log a Hash of the Messages Array Before Every Call (Yeda AI Tips #154)
Yeda AI Tips — one AI tip a day keeps the busywork away.