Measured Model Substitution Under Valid Agent Credentials

Abstract

Three model substitution scenarios were executed against a live inference endpoint with real HTTP requests, signed attestation JWTs, and OPA policy enforcement. In each scenario, every tested workload, artifact, or API identity control relevant to that scenario — workload JWT validation, health checks, gateway process continuity, artifact manifest integrity, API key authentication — remained valid while the model changed. In each scenario, a structural identity measurement based on activation geometry during a standard forward pass detected the substitution and the enforcement layer denied the request. Three substitutions were tested and three were detected, with zero false accepts in this run. The warm-path verification latency was 5.7–6.7 seconds on a single A100 with the model already loaded. The complete evidence chain — before/after measurement results, attestation claim summaries, OPA policy evaluations, and HTTP response codes — is published alongside this note as machine-readable JSON.

1. Purpose

Enterprise identity systems can authenticate a workload, verify an artifact, and authorize a request without ever establishing which neural network is actually computing inside the service. That distinction is the point of this note.

The measurement used here identifies neural networks at inference time from the geometry of their output distributions, without inspecting weights, modifying the model, or requiring cooperation from the deployer. It has already been validated across 48 models spanning three architecture types and parameter counts from 410 million to 72.7 billion [1–6]. What has not yet been shown is the operational step: the measurement running inside a live gateway, issuing a signed attestation, evaluating policy, and changing the HTTP outcome of a real request.

This note provides that experiment.

The question it answers is operational rather than scientific: if an authorized model is enrolled behind a production-shaped gateway and a different model is substituted behind the same stable endpoint, does the measurement detect the substitution in a real request flow, and does the enforcement chain act on it?

Three kinds of identity evidence are relevant in these scenarios: artifact identity (which file was deployed), workload identity (which process or service is authenticated), and model identity (which neural network is actually computing). This note tests what happens when the first two remain valid while the third changes.

2. Architecture

Two servers run on the same host:

A model server on port 8080 exposes a health endpoint and an inference endpoint. The model server loads the neural network and serves completions.

A gateway on port 9090 intercepts every request and runs the following chain before deciding whether to proxy to the model server:

Two tokens are in play throughout. The workload JWT (agent identity) is issued once before the experiment begins, stays valid across all phases, and represents the authenticated software agent — this token is green in every scenario. The attestation JWT (model identity) is issued per-measurement by the production attestation issuer and encodes whether the running model matches the enrolled model — this token turns red when the model changes.

Figure 1. Gateway request flow with workload and model identity tokens. The gateway runs six checks; proxy and deny are the two outcomes. The workload JWT (step 1) remains valid in every scenario. The attestation JWT (step 4) encodes the structural measurement result and flips from match to mismatch when the model changes.

3. Three Scenarios

Scenario A: Same-Family Substitution Behind a Stable Endpoint

The enrolled model is Llama-3.1-8B-Instruct. The substituted model is DeepSeek-R1-Distill-Llama-8B — a model distilled onto the same Llama architecture with the same parameter count and the same API contract.

The gateway process stays alive throughout. The workload JWT is the same JWT. The health endpoint returns 200 in both cases. The policy hash is unchanged.

Every tested workload and endpoint identity control remained green. Only the model identity layer turned red.

Scenario B: Cross-Family Substitution with Artifact Hash Integrity

The enrolled model remains Llama-3.1-8B-Instruct from Scenario A. The substituted model is Qwen-2.5-7B-Instruct — a different architecture family, different tokenizer, different training lineage.

Both models' weight files are verified against build manifests containing SHA-256 hashes of every artifact. Both manifests pass. Llama: 10 files, all hashes match. Qwen: 10 files, all hashes match. Traditional artifact integrity verification sees nothing wrong with either model.

The structural identity measurement rejects the substituted model at 3,416×ε — exceeding Scenario A's same-family rejection of 2,858×ε, as expected for cross-family pairs. The signed attestation carries identity_match: false. OPA denies.

Scenario C: Silent API Rotation

The enrolled model is gpt-4.1-mini served through the OpenAI API. The rotated model is gpt-4.1-nano. Both use the same API key, the same endpoint base, the same authentication, the same billing account. The provider authenticates both models identically.

Each model is enrolled across three independent sessions. Per-model thresholds are computed from maximum cross-session self-distance with a safety margin. Self-verification accepts both models against their own enrollment. Cross-verification rejects: the mean per-prompt distance of 5.66 exceeds both per-model thresholds (3.79 and 1.16).

4. Validation Gates

Before any scenario executes, two validation gates confirm measurement engine integrity:

Both gates passed before any experimental scenario was executed.

5. Timing

Model loading is infrastructure overhead — 34–45 seconds to download and shard an 8B model. This is not verification latency. With the model already loaded and serving:

OPA policy evaluation: < 1ms. The measurement dominates.

This measurement is not inline on every user request. It runs at model load, on a periodic schedule, or as an out-of-band health check — the same deployment pattern as certificate rotation or container attestation refresh. The per-request path carries the cached attestation JWT, not the measurement itself.

6. Platform

Distances throughout this note are expressed as multiples of ε, the acceptance threshold derived from measurement-platform numerical precision. A distance of 0.0×ε indicates an identical model; any distance above 1.0×ε indicates a different model.

7. What This Does and Does Not Establish

This experiment establishes that artifact integrity, workload identity, and API authentication can remain valid while runtime model identity changes — and that a structural identity measurement, integrated into a standard gateway enforcement chain, detects the change and produces an enforceable policy decision. Three substitution scenarios were tested — same-family, cross-family, and API rotation — and all three were detected with zero false accepts.

This experiment does not establish that existing identity controls are unnecessary. Workload identity, artifact integrity, and API authentication are real and necessary controls. The finding is that they occupy a different evidence class than model identity and do not answer the same question. Artifact integrity and workload identity did not establish runtime model identity in these scenarios.

8. Evidence Artifacts

The complete machine-readable evidence is published alongside this note:

References

View 6 references ↓

[1] A. R. Coslett, "The δ-Gene: Inference-Time Physical Unclonable Functions from Architecture-Invariant Output Geometry," 2026. DOI: 10.5281/zenodo.18704275

[2] A. R. Coslett, "Template-Based Endpoint Verification via Logprob Order-Statistic Geometry," 2026. DOI: 10.5281/zenodo.18776711

[3] A. R. Coslett, "Composable Model Identity: Formal Hardening of Structural Attestations in the Enterprise Identity Stack," 2026. DOI: 10.5281/zenodo.19099911

[4] A. R. Coslett, "Post-Hoc Disclosure Is Not Runtime Proof: Model Identity at Frontier Scale," 2026. DOI: 10.5281/zenodo.19216634

[5] A. R. Coslett, "Agent Identity Is Not Model Identity," 2026. DOI: 10.5281/zenodo.19240883

[6] A. R. Coslett, "What Counts as Proof? Admissible Evidence for Neural Network Identity Claims," 2026. DOI: 10.5281/zenodo.19058540

---

Acknowledgments

Portions of this research were developed in collaboration with AI systems that served as co-architects for experimental design, adversarial review, and manuscript preparation. All scientific claims, experimental designs, measurements, and editorial decisions remain the sole responsibility of the author. Experiments were conducted on Google Colab using NVIDIA A100-SXM4-80GB GPUs.

Author's Disclosure

Anthony Ray Coslett is the founder of Fall Risk AI, LLC, which holds the provisional patents listed below. The structural identity measurement described in this paper operates within the scope of that intellectual property. No external funding was received for this research.

Patent Disclosure

U.S. Provisional Patent Applications 63/982,893, 63/990,487, 63/996,680, and 64/003,244 are assigned to Fall Risk AI, LLC.