Hardening AI evidence: input completeness, sequence integrity, and the AI event ledger

The previous post showed a working Python agent that timestamps every prompt and every model response. It's a clean pattern, ~150 lines, and for many use cases it's enough. For some use cases — high-stakes decisioning, regulated environments, anything where the records might end up in front of an adversarial reader — it isn't. This post is about why, and what to add to make the records hold up.

The short version: stamping each turn proves that turn happened. It doesn't prove the turn is complete, doesn't prove the sequence wasn't edited, and doesn't address the data-protection implications of keeping prompts in cleartext alongside their stamps. Turning the simple agent into something an auditor can't take apart needs three additions, none of them difficult once you've decided to make them.

This is the stricter post in a series. If the first one was "here's the pattern," this one is "here's how the pattern survives scrutiny."

What we're actually building

Once you tighten everything in this post, what you have stops looking like a log file and starts looking like an AI event ledger — an append-only, externally anchored record of every interaction the model had, structured so that you can prove not just individual turns but the integrity of the whole sequence.

The shift in framing matters more than it sounds. Logs are operational artefacts; people accept that they get rotated, truncated, sampled, and occasionally lost. Ledgers are evidence; people expect them to be complete, ordered, and durable. The differences below are what gets you from one to the other.

Problem 1: input completeness

The simple agent stamps user_input and reply. For a chat with no tools and no retrieval, that's all the model saw and the evidence is complete. For anything more sophisticated — a RAG system, a tool-using agent, a retrieval-augmented copilot — it is not.

The model's actual input on a given turn looks more like:

system_prompt
+ tool_definitions
+ conversation_history (previous turns)
+ retrieved_chunks (from RAG)
+ tool_results (from prior tool calls this turn)
+ user_message
+ model_parameters (temperature, top_p, model version, etc.)

If you only stamp user_message, you've committed to one component of the input. A counterparty arguing the model gave bad advice can ask: what was the system prompt? What documents did you retrieve? Which version of the model? Which temperature? Without those, you've proven the user typed a question; you have not proven the question the model answered.

The fix is to stamp the full input as the model received it. Most LLM APIs let you serialise the exact request body — messages array, system prompt, tools list, parameters. Hash that:

import hashlib, json

def hash_request(request_body: dict) -> str:
    """Stable hash of an Anthropic-shaped request — including system prompt,
    messages, tool definitions, and parameters. JSON keys sorted so that
    re-serialisation produces the same hash for the same input."""
    canonical = json.dumps(request_body, sort_keys=True, separators=(',', ':'))
    return hashlib.sha256(canonical.encode('utf-8')).hexdigest()

# Build the request the SDK will send
request_body = {
    "model":      "claude-opus-4-7",
    "max_tokens": 1024,
    "system":     SYSTEM_PROMPT,
    "tools":      TOOLS,
    "messages":   history + [{"role": "user", "content": user_input}],
    "temperature": 0.0,
}

# Stamp this — not just the user_input
ts_input = stamp(json.dumps(request_body, sort_keys=True), f"input-{turn_id}")
response  = client.messages.create(**request_body)

Two notes on this. First, this includes the conversation history on every turn, which means each turn's input stamp implicitly commits to all prior turns — useful but expensive (you're hashing more text). Second, retrieved chunks need to be part of the request body, which is the case for any system that hands the LLM the retrieved text directly in messages or system. If your retrieval happens inside a tool call, the chunks land in the next turn's tool_results, which is also part of the request body. Either way, hash what the model actually sees.

For tool-using agents, this means stamping at every model invocation — including intermediate planning and tool-call rounds, not just the user-facing exchange. A single user question may produce five model calls under the hood. All five are part of the evidence.

What you give up by not doing this: the ability to defend the model's output. If you can only show what the user typed, the most a reviewer can say is "the user asked a reasonable question." Whether the answer was reasonable depends entirely on what the model was working with.

Problem 2: sequence integrity

JSONL is convenient. It is also trivially modifiable: a malicious or panicked operator can delete a line, reorder lines, edit a line, or splice in a fake line, and the file still parses as valid JSONL. The individual timestamps stamped by Sigill don't catch any of this — each timestamp is bound to its own content, not to its position in the file.

For low-stakes cases, this is fine; the operator who would tamper with the log is also the operator who controls everything else, so they're not your threat model. For higher-stakes cases — where the log might be subpoenaed, or where the operator and the auditor are explicitly different parties — sequence integrity becomes load-bearing.

There are two cheap improvements and one strong one.

Cheap: chain each entry to the previous one's hash

Each log line includes the SHA-256 of the previous line. Truncating, reordering, or modifying any line breaks the chain — the next line's prev_hash no longer matches.

prev_hash = "0" * 64  # genesis

def append(record: dict):
    global prev_hash
    record["prev_hash"] = prev_hash
    serialised = json.dumps(record, sort_keys=True, separators=(',', ':'))
    record_hash = hashlib.sha256(serialised.encode('utf-8')).hexdigest()
    with LOG_FILE.open("a") as f:
        f.write(serialised + "\n")
    prev_hash = record_hash

This catches insertion, deletion, reordering, and modification of any line, provided the verifier can recompute the chain forward from the start and compare the resulting head hash to a periodically timestamped anchor. Stamp the current head hash through Sigill on a regular schedule — hourly or daily — and any tampering before the most recent anchor is detectable. The cost is one stamp per period, regardless of log volume; the security boundary is the anchor interval.

Batch: build a Merkle tree of the day's records

If the log volume is high and per-turn stamping is expensive, batch the day's records into a Merkle tree and stamp the root once. Anyone wanting to prove a specific record need only walk a logarithmic-sized inclusion proof — they don't need the whole log.

import hashlib

def merkle_root(leaves: list[bytes]) -> bytes:
    if not leaves: return b'\x00' * 32
    while len(leaves) > 1:
        if len(leaves) % 2:
            leaves.append(leaves[-1])  # duplicate odd tail
        leaves = [
            hashlib.sha256(leaves[i] + leaves[i+1]).digest()
            for i in range(0, len(leaves), 2)
        ]
    return leaves[0]

One Merkle root per day, one Sigill stamp per day, billions of records committed. This is how transparency logs (Certificate Transparency, Sigstore Rekor) work, and it's the right answer for any high-volume AI workload.

The trade-off: Merkle is cheaper in stamp volume but more expensive in verifier complexity. Inclusion proofs require more verifier code than a chained log, and proof generation needs the full leaf set or an indexed copy. Pick chained-log for hourly cron-stamped low-volume agents; pick Merkle for high-volume systems where the per-turn stamp cost dominates.

Strong: do both

Chain the entries (so any single edit between stamps is caught) and stamp the chain head periodically (so any tampering is bounded by the stamp interval). This is what an "AI event ledger" looks like in practice. Sigstore's Rekor is a much larger version of this same shape.

Problem 3: data handling

The hash-only-leaves-the-runner story is true for what reaches the TSA, but it's incomplete in the inverse: the JSONL log on disk contains the prompts and responses in cleartext. For systems where prompts can contain personal data, regulated information, or trade secrets, the log is now a data store with the same classification as the underlying conversations.

Three things to acknowledge and address:

1. The log inherits the data classification of the highest-classified message it contains. A medical-domain agent's log is medical data. A financial-domain agent's log is financial data. Storage, encryption, access controls, and retention need to match — locally encrypted at rest, access-controlled, retained according to the same rules as the conversations themselves.

2. GDPR right-to-erasure conflicts with the immutability story. A user asking "delete everything you have on me" creates tension with an append-only ledger that proves history. The standard pattern is cryptographic erasure: the ledger keeps the hashes of records, but the cleartext records themselves are deleted. The chain is preserved, the proof structure is preserved, but the original content is gone. You can prove a record existed at time T without being able to reproduce its contents.

Whether the remaining hash is itself personal data is a legal judgement, not a technical one. Hashes can substantially reduce privacy risk, but they may still be treated as personal data in some contexts — particularly if the hash space is small enough to brute-force, if linkage to other records is possible, or if the underlying data is structured (e.g. an email address) such that a determined attacker could enumerate candidates. Talk to your DPO about whether cryptographic erasure satisfies your specific erasure obligations; don't assume it does because the cleartext is gone.

3. Stamping the request body means the system prompt is in your evidence. If your system prompt contains anything you don't want to commit to durable evidence — proprietary instructions, prompt-injection defences you don't want to telegraph — be aware that stamping the request body commits to it cryptographically. If that's a problem, stamp a structured manifest (model, parameters, hash-of-system-prompt, hash-of-tools, full user message, full retrieval chunks) rather than the full request body. You preserve the evidence of what was used while keeping the contents of the system prompt private.

None of these are blockers. All three are foreseeable and tractable. They just need to be designed into the ledger from day one rather than discovered three months in.

Putting it together

The minimal hardened agent has the following shape, with each change marked:

def main():
    history = []
    prev_hash = "0" * 64                                      # NEW

    while True:
        user_input = input("you > ").strip()
        turn_id    = int(time.time() * 1000)

        # Build the full request the model will see
        request_body = {                                      # NEW: full input
            "model":       MODEL,
            "max_tokens":  1024,
            "system":      SYSTEM_PROMPT,
            "tools":       TOOLS,
            "messages":    history + [{"role": "user", "content": user_input}],
            "temperature": 0.0,
        }

        # Stamp the full input
        canonical_input = json.dumps(request_body, sort_keys=True, separators=(',', ':'))
        ts_input = stamp(canonical_input, f"input-{turn_id}")

        # Call the model
        response = client.messages.create(**request_body)
        reply    = response.content[0].text

        # Stamp the output (also include model version + finish_reason as evidence)
        canonical_output = json.dumps({                       # NEW: structured output
            "reply":          reply,
            "model":          response.model,
            "finish_reason":  response.stop_reason,
            "usage":          response.usage.model_dump(),
        }, sort_keys=True, separators=(',', ':'))
        ts_output = stamp(canonical_output, f"output-{turn_id}")

        # Append a chained log entry
        record = {                                            # NEW: chained record
            "turn_id":   turn_id,
            "at":        datetime.utcnow().isoformat(),
            "input":     user_input,
            "output":    reply,
            "ts_input":  ts_input,
            "ts_output": ts_output,
            "prev_hash": prev_hash,
        }
        serialised  = json.dumps(record, sort_keys=True, separators=(',', ':'))
        record_hash = hashlib.sha256(serialised.encode('utf-8')).hexdigest()
        with LOG_FILE.open("a") as f:
            f.write(serialised + "\n")
        prev_hash = record_hash

        history.append({"role": "user",      "content": user_input})
        history.append({"role": "assistant", "content": reply})

A separate hourly cron stamps prev_hash to anchor the chain head. That's the whole hardening.

What you now have, properly named: a tamper-evident, externally-anchored, sequenced ledger of model interactions, where each entry commits to the model's full input and full output, the chain commits to the order, and periodic Sigill stamps anchor the chain to durable wall-clock time. An adversarial reader can attack any of these — they can claim the system prompt is wrong, the model version isn't what you say it is, the chain's been replayed — but each attack runs into a cryptographic check they have to defeat to land. That's the difference between a record that's suggestive and a record that's evidence.

What this still doesn't solve

Cryptographic hardening of the log doesn't fix the fundamentally hard problems of AI evidence:

• It doesn't validate the model's reasoning. The ledger can prove the model said X; it can't prove X was correct.
• It doesn't catch a compromised agent that lies about what it sent to the model. The ledger is only as honest as the runtime that writes it.
• It doesn't address model versioning ambiguity beyond what the API tells you. "claude-opus-4-7" is what the API reports; if the provider silently routes to a different snapshot, the ledger faithfully records the lie.
• It doesn't help with the model's training data, alignment, or safety properties — those are upstream concerns the ledger pre-supposes.

The ledger is the foundation other AI-governance controls build on. Necessary, not sufficient.

When this level of rigour is worth it

This is heavier than the simple pattern, and not every AI workload needs it. Honest segmentation:

The simple pattern is the right answer when the audience is you, the operator. The hardened ledger is the right answer when the audience is someone other than you, and especially when you might be the party under question.

If you're building anything in the second category, build it as a ledger from day one. Retrofitting integrity is much harder than designing for it.

The full hardened agent source — including chained log entries, Merkle tree builder, and a verifier — is on GitHub. For questions, contact us.