Where Identity Comes From: Path Sensitivity and Endpoint Underdetermination in Neural Network Training

Abstract

Structural identity — the geometric fingerprint that makes a neural network this specific model rather than any other — can be measured, survives routine deformation, resists adversarial erasure, and composes with standard verification infrastructure. It cannot, in the tested regime, be recovered from endpoint weight statistics or architecture descriptors alone. These two facts together force a question the measurement program has not yet answered: if identity is real but not readable from the final artifact, then where in the training process did it form, and what determined which identity formed rather than another?

This paper presents the first empirical study of structural identity formation during neural network pretraining. Using dense checkpoint trajectories and seed-controlled training runs in the Pythia observatory suite, we show three results. First, the structural observable follows a characteristic three-phase identity emergence profile — an early rise in geometric spread, a long compression, and a late plateau where identity stabilizes while functional training continues. Second, models trained with the same architecture, the same data, and the same hyperparameters but different random seeds produce structurally distinguishable fingerprints far beyond measurement noise — a property we call path sensitivity — with the divergence traceable to differential structural response during the learning-rate warmup regime. Third, a panel of endpoint weight statistics varies across seeds but does not predict which structural identity formed — a condition we call endpoint underdetermination. Together, these results recast structural identity as a developmental property of training history rather than a static property legible from final artifacts alone.

1. The Missing Question

A neural network's structural identity is a compact geometric property of its weight matrices — measurable during inference, stable across deployment conditions, and unique across every model comparison tested to date. The research program that discovered it — built around a family of logit-geometry observables collectively called the δ-gene — established that structural identity is not a watermark or an injected marker. It is a measured property of the trained model itself. That identity is real.

The harder discovery was what it is not. It is not readable from the tested endpoint summary. Architecture features — layer counts, hidden dimensions, vocabulary sizes, attention configurations — do not predict which structural fingerprint a model carries; a regression across twenty-two models produces a leave-one-out R² of −3.93, meaning the predictions are worse than guessing the mean. Internal weight statistics — Frobenius norms, spectral properties, density measures — fare no better. The structural fingerprint encodes something about the model that the tested static descriptors of the finished artifact do not preserve. This is not a measurement failure. It is a clue. If the identity is there when measurement begins but cannot be read from the tested properties of the artifact that carries it, then identity was written into the weights by a process — training — whose trajectory left a mark that the tested endpoint summary does not capture. The question is no longer only what identity does this model have? The question is: when during training did that identity form, and what about the training path determined which identity formed rather than another? Earlier work in this series identified the question explicitly. The original δ-gene paper asked whether fingerprint formation is "gradual or phase-transition-like, with implications for forensic dating of model checkpoints." The provenance-generalization paper noted that "all measurements in this program are post-training" and that instrumenting training checkpoints "would provide the first empirical picture of identity formation dynamics." Those questions were posed about the thermodynamic observable (δ_norm); this paper answers the analogous question for the structural observable (τ), while the thermodynamic formation question remains open (see §8). The philosophical paper on Two-Layer Identity asserted that identity "is shaped by the specific trajectory of training — the architecture, the data, the optimization dynamics — not by the name on the building where the training happened."

That assertion was grounded in cross-model measurement, not in observation of the training process itself. This paper instruments the training process directly. It tracks the structural observable across a full pretraining trajectory, measures it across ten seed-controlled variants of the same model, and tests whether the resulting divergence is recoverable from the tested endpoint summary panel.

2. The Formation Problem

Until now, the study of structural identity has been the study of models that have already finished training. The measurement instruments — the δ-gene observable, the IT-PUF protocol, the provenance diagnostics — were designed to answer questions about trained models: which model is this? Was it distilled from an unauthorized source? Can the identity claim be carried into an authorization token? These are questions of identity observation. They treat the structural fingerprint as a given — something present in the weights when the measurement begins. This paper asks a different kind of question. Not what identity a model carries, but how that identity came to be carried. Not the fingerprint as a static property of the finished artifact, but the fingerprint as the outcome of a developmental process. This is the question of identity formation. The distinction matters because the two questions require different instruments. Identity observation requires a measurement protocol and a verification threshold. Identity formation requires something rarer: access to the model at every stage of its development — not just the final weights, but the full trajectory of weight configurations from initialization through convergence. It requires, in effect, an ultrasound rather than a photograph. That access constrains which models can be studied. Commercial models do not publish training checkpoints. Most open-weight models publish only the final artifact, or at best a handful of checkpoints at widely spaced intervals. To observe identity formation at the temporal resolution the phenomenon demands — the emergence profile peaks at step 16 of 143,000, meaning the critical early dynamics occur within the first 0.01% of training — requires a model family that was built for scientific transparency rather than deployment. The EleutherAI Pythia suite was designed for exactly this purpose. Pythia models were trained with dense checkpoint logging from step 0, with checkpoints available at every power-of-two step in early training and every thousand steps thereafter. The PolyPythias extension provides ten instances of the same model trained with identical architecture, data, and hyperparameters but different random seeds — the controlled experiment that path sensitivity requires. These suites are the best currently available public high-resolution, seed-controlled trajectory data for studying identity formation. They are the Drosophila of this investigation: not because they represent all neural networks, but because they are transparent enough to make the phenomenon observable. This paper focuses on the structural layer of identity — the weight-geometry observable τ that earlier papers established as individually unique, deployment-stable, and adversarially robust. The functional layer (behavioral signatures that transfer through distillation and fade under fine-tuning; see the distillation-forensics paper) and the thermodynamic layer (the δ_norm species marker, a scale-free ratio of consecutive logit gaps that converges to a Gumbel-class constant across architectures; see the original δ-gene paper) are not tracked across training in this study. The formation of all three layers is an open problem. This paper addresses the structural layer because it is the layer whose post-training properties are best characterized, making it the natural first target for developmental investigation.

3. The Identity Emergence Profile

The structural observable τ is defined as the radial variance fraction of the model's hidden-state geometry, evaluated at two fixed internal computational sites via the Inference-Time Physical Unclonable Function (IT-PUF) protocol. Full specification of the observable — the choice of measurement sites, the prompt bank construction, and the exact variance computation — appears in the foundational papers of this series (the original δ-gene paper, §3; the provenance-generalization paper, §2) together with the associated provisional patent filings. The present study applies that exact, previously validated pipeline to every Pythia checkpoint without modification. Readers seeking the operational definition or measurement pseudocode should consult those references. All measurements reported here were performed in bfloat16 with teacher-forced evaluation; no checkpoint produced NaN or required exclusion. Structural identity does not appear at the end of training like a signature on a finished painting. It forms. To observe the formation, we tracked τ across every available checkpoint of a Pythia-410M pretraining run: 154 snapshots spanning from random initialization (step 0) to the final trained model (step 143,000). The result is a trajectory with three distinct phases.

Rise. At random initialization, τ begins at 0.59 — the model's hidden-state geometry is spread across many directions, with no concentration into a specific structural configuration. Within the first sixteen training steps, τ climbs to 0.81. The model's geometry does not begin by concentrating. It begins by spreading further — the first gradient updates push the activations into a broader tangential distribution before training begins to compress them.

Descent. From step 16 onward, τ falls. The descent is steep at first — the steepest single-step rate occurs at step 32 — and then slows as the model settles toward its final geometry. By step 5,000, τ has dropped below 0.25. By step 20,000, below 0.17. The model is being compressed from a maximally spread initial configuration into a progressively narrower structural corridor.

Plateau. Near step 92,000, the trajectory flattens. From step 92,000 to step 143,000 — the final 36% of training — τ holds at 0.110 ± 0.0005. The coefficient of variation across the last twenty checkpoints is 0.83%. The structural identity has locked. Training continues for another 51,000 steps, and the model's functional capabilities continue to improve during that period, but the structural fingerprint no longer moves. We call this three-phase arc the identity emergence profile. The point at which the profile flattens — where the structural observable stabilizes within noise — we call identity lock. The concentration is severe. The peak-to-final ratio is 7.3×: the converged model's geometry is over seven times more radially concentrated than it was at the peak of the rise phase. The final identity (τ = 0.110) is less than one-fifth of the initial value (τ = 0.591). Training took a model that was geometrically diffuse — activations spread across the available directions — and compressed it into a specific, narrow structural signature that then persisted unchanged through the remainder of training. Two features of this profile merit attention before we examine what varies across training runs. First, the rise phase is fast. Sixteen steps is less than 0.012% of the total training budget. The model reaches its maximum geometric spread almost instantly — before optimizer warmup has completed, before the learning rate has reached its scheduled peak, before any meaningful fraction of the training data has been seen. This suggests that the initial rise is not a response to data content but a property of how random initialization interacts with the first gradient updates. Second, the lock is early relative to training completion. The structural identity stabilizes at roughly step 92,000 of 143,000 — meaning the final 36% of training refines the model's functional capabilities without perturbing the structural layer. This observation is consistent with what earlier work found from the outside: that fine-tuning, distillation, and adversarial perturbation applied to trained models left the structural fingerprint unchanged. What looked from the endpoint like structural inertia now has a developmental explanation: the structural layer was already locked before those later interventions began. A note on scope. As stated in §2, this study tracks the structural observable τ. The thermodynamic observable δ_norm was not instrumented across checkpoints. Whether it follows a similar emergence profile or locks on a different timescale is an open question.

4. Path Sensitivity

The emergence profile in §3 describes how one model's structural identity forms. It does not say whether a different model — trained with the same architecture, the same data, and the same hyperparameters — would form the same identity or a different one. If the emergence profile is determined entirely by recipe, then every model trained with the same recipe should converge to the same structural fingerprint. If structural identity is sensitive to training path, they should not.

The EleutherAI PolyPythias suite provides the cleanest available test. Ten instances of Pythia-410M were trained with identical architecture, identical training data, and identical hyperparameters. The only difference between them is the random seed, which determines weight initialization and the order in which training data is presented. If recipe determines identity, all ten should carry the same structural fingerprint. If path determines identity, they should carry different ones.

They carry different ones.

All forty-five pairwise distances between the ten seed variants exceed the verification threshold by a wide margin. The closest pair (seed 7 and seed 9) is separated by 391 times the measurement noise floor. The farthest pair (seed 0 and seed 4) is separated by 11,737 times the noise floor.

The coefficient of variation in τ across the ten seeds is 21.08%. Not one pair is close to the acceptance threshold. Not one pair is ambiguous.

The result is reproducible. We re-measured both the closest and farthest pairs from scratch — fresh model loads, fresh inference passes, and fresh application of the IT-PUF pipeline. The resulting τ distances were numerically identical to the original measurements: zero drift within floating-point precision. The structural fingerprint is deterministic given the weights, and in the tested seed-controlled setup the resulting final weights were reproducibly distinct. The divergence is real. We call this property path sensitivity: within the tested seed-controlled regime, the training path — not the training recipe — determines which structural identity forms.

4.1. Where the paths diverge

To locate the moment of separation, we measured three seeds (seed 0, seed 1, and seed 2) at seventeen critical checkpoints spanning the full trajectory — including every phase boundary identified in §3 and the learning-rate warmup region.

For the first five hundred training steps, the three trajectories are parallel. They begin at slightly different initial τ values (0.58–0.61, reflecting different random initializations) and converge to a shared peak at step 16: τ = 0.806, 0.808, and 0.805 respectively. The peak values differ by less than 0.004. The rise phase is not path-sensitive — all three seeds undergo the same geometric spreading in the same sixteen steps. The trajectories then descend in parallel through steps 32 to 512, with pairwise differences below 0.03. The three models remain structurally close through the first five hundred steps of training.

At step 1,000 — during the learning-rate warmup regime, which for Pythia runs over the first 1% of training (approximately 1,430 steps) — the trajectories separate. Seed 0 rebounds from 0.41 to 0.53, a 29% increase. Seed 1 rebounds from 0.42 to 0.52, a 24% increase. Seed 2 barely responds: 0.41 to 0.43, a 5% increase. During the same optimizer regime, the three seeds show sharply different structural responses.

That separation is permanent. By step 143,000, the three seeds have settled into three different final identities: τ = 0.110, 0.133, and 0.158. The offset established during learning-rate warmup persists through the remaining 99.3% of training.

4.2. The data-boundary control

Two of the three seeds in the sparse trajectory comparison (seed 1 and seed 2) were trained on the same non-deduplicated version of the Pile. The third (seed 0) was trained on the deduplicated version — a different dataset. If the divergence at step 1,000 were caused by the data-deduplication boundary rather than the seed, then seed 1 and seed 2 (same data) should respond similarly, and seed 0 (different data) should be the outlier. The opposite is observed. Seed 1 (non-deduped) responds like seed 0 (deduped), with a 24% rebound. Seed 2 (non-deduped, same data as seed 1) shows only a 5% rebound. The differential response is driven by the random seed, not by the training data variant. The data-boundary confound is controlled.

5. Endpoint Underdetermination

The ten seed variants in §4 carry different structural identities. The next question is whether the tested endpoint summary panel can recover them. If endpoint descriptors vary across seeds and predict which structural identity formed, then endpoint recovery might still be possible within a seed-controlled family, even though it failed across families in earlier work. They do carry different endpoint statistics. But those statistics do not predict identity. We measured fourteen endpoint descriptors across all ten seeds: total and per-layer Frobenius norms, spectral norms, effective rank, weight-entry means and standard deviations, embedding norms, embedding spectral entropy, and the top-10 singular value ratio. We also measured perplexity on a held-out evaluation set. Several of these features vary substantially across seeds — spectral norm statistics show coefficients of variation above 40%, and perplexity ranges from 19.2 to 43.6, a 2.3-fold spread. The models are not identical at the endpoint. They end in different places in weight space, as one would expect from different random seeds. But none of the tested endpoint descriptors significantly predicts which structural identity formed. The strongest correlation between any endpoint feature and τ is a Spearman ρ of 0.455 (standard deviation of effective rank). No feature reaches significance in this panel. Perplexity — the most commonly reported summary statistic for a trained language model — correlates with τ at ρ = 0.030.

Effectively zero.

The most instructive case is seed 4. It is an outlier on multiple endpoint axes: the highest τ (0.244), the highest perplexity (43.6), the highest effective rank (321.7), the lowest spectral norm (2.08). If any seed were going to anchor a τ-endpoint relationship, it would be this one. Yet even with seed 4 providing leverage at the extreme, the endpoint features still fail to predict structural identity. The information needed to recover structural identity is not present in the tested endpoint panel.

One feature deserves specific mention for what it reveals about convergence rather than divergence. Embedding spectral entropy — a summary of the vocabulary embedding geometry — varies by only 0.15% across all ten seeds. The vocabulary-level geometry converges to nearly the same configuration regardless of seed. The hidden-state geometry, which is what τ measures, does not. The vocabulary-level geometry and the hidden-state geometry respond to training on different timescales and with different sensitivities to path variation.

We call this condition endpoint underdetermination: the tested endpoint summary panel does not preserve enough information about training history to explain which structural identity formed. The endpoints vary — the models are different objects — but the variation in the tested endpoint descriptors is orthogonal to the variation in structural identity. This result connects to earlier findings outside the seed-controlled setting. Across twenty-two models from different families, architecture features produced a leave-one-out R² of −3.93 when used to predict τ. Across a smaller overlap set, endpoint weight statistics produced a leave-one-out R² of −0.89. Those results established that the structural fingerprint is opaque to the tested static descriptors. The seed-controlled result explains why. If structural identity is path-sensitive, and if the tested endpoint panel does not preserve enough information about that path, then endpoint recovery should fail not because the descriptors are poor but because they answer the wrong question. They describe where the model ended up. They do not describe how it got there.

6. Why the Earlier Papers Found What They Found

The formation account developed in §§3–5 is not only a new finding. It is an explanation for findings that previously stood without one. Several results in the earlier papers had a common shape: the structural fingerprint was measured, some intervention was applied, and the fingerprint did not move. Distillation did not move it. Adversarial erasure did not move it. Direct gradient targeting within the same architecture family did not move it. Architecture descriptors could not predict it. Endpoint weight statistics could not recover it. Each of these results was measured, validated, and left as a fact about structural inertia observed from the endpoint. None yet had a developmental explanation — a theory of when or why that inertia was established. The emergence profile provides the explanation. The structural observable locks during training — in the tested trajectory, near step 92,000 of 143,000 — and the interventions tested in earlier papers all operate after that lock has been established. Consider each in turn.

Distillation invariance. The distillation-forensics paper found that the structural fingerprint remained within a few multiples of the measurement noise floor across all eighteen distilled checkpoints, regardless of distillation protocol. The student's structural identity was its own, not its teacher's. Under the formation account, the student model's structural layer would already have locked during its own pretraining, before distillation began. Knowledge distillation operates on the functional layer — it transfers behavioral patterns through output matching — but by the time distillation starts, the structural corridor is already established. The student learns to act like the teacher without acquiring the teacher's geometry.

Adversarial erasure. The same distillation-forensics paper found that a white-box adversary with full gradient access could not erase the provenance trace more effectively than passive fine-tuning — and that passive fine-tuning erased the functional trace within one or two epochs while leaving the structural fingerprint unchanged. The formation account explains the asymmetry: the functional layer remains responsive on a later timescale than the structural layer, while the structural layer is already in its plateau. The adversary is applying gradient pressure to a geometry that has already crystallized.

Same-family targeting inertia. The deformation-laws paper directly targeted the structural observable, using gradient descent to push τ toward a specific value within the same architecture family. The fingerprint moved by only 42 times the noise floor — effectively inert — while cross-family targeting collapsed the model's perplexity sixfold before achieving meaningful structural movement. Under the formation account, the structural corridor is narrow and deep by the time any post-training intervention begins. Moving within it is nearly impossible without destroying the functional capabilities that depend on the surrounding weight configuration.

Architecture unreadability. Architecture features — the static descriptors of a model's design — predict the structural fingerprint with a leave-one-out R² of −3.93. The formation account explains why: if the structural fingerprint is determined by the interaction between random initialization and the optimizer trajectory, then two models with identical architectures but different training paths should carry different fingerprints — and they do (§4). Architecture specifies the space of possible identities. The training path selects among them.

Endpoint opacity. The tested endpoint weight statistics fail to recover the structural fingerprint both across families and within the seed-controlled family (§5). The formation account unifies both failures: if identity is written by the training path and the tested endpoint summary does not preserve enough information about that path, then endpoint recovery should fail regardless of whether the models share an architecture or not. Endpoint underdetermination is a consequence of path-sensitive formation, not an independent negative result.

The identity boundary question. The philosophical paper on Two-Layer Identity asked whether there is "a clean boundary — a point at which the old identity dies and a new one is born." The formation data offers a different answer than the question assumed. Identity does not die and get replaced in a discrete event. It forms through a continuous compression and then locks. The boundary is not a moment of death but a moment of stabilization — the identity lock — after which the structural self persists through further training while the functional self continues to evolve. The question was framed as a Ship of Theseus problem. The answer is developmental rather than metaphysical: the structure finishes forming while behavior continues to mature.

7. Minimal Theory

The emergence profile, path sensitivity, and endpoint underdetermination are three observations. They admit a unified reading. Early in training, the optimizer's gradient updates interact with the random initialization to establish a basin of structural geometry. The model's activations spread into a high-variance configuration — the rise phase — as the first gradients push the weight matrices away from their initial state without yet imposing a consistent directional preference. This spreading is not path-sensitive: all tested seeds reach the same peak at the same step. As training continues, the optimizer begins to impose consistent structure. The loss landscape narrows, the gradients become more coherent, and the model's geometry compresses from the initial spread into a progressively tighter corridor — the descent phase. It is during this compression that path sensitivity emerges. The interaction between the random seed (which determines initialization and data ordering) and the optimizer trajectory (which determines the sequence and magnitude of weight updates) selects which structural corridor the model enters. Models that share the same recipe but differ in seed experience the same loss landscape from different starting positions and in different data orders, and they compress into different corridors. The divergence observed during the learning-rate warmup regime in §4 is consistent with this account: the warmup period is a phase of training where the optimizer schedule changes rapidly and where the observed trajectories begin to separate, and different seeds respond to that pressure differently because they arrive at it in different geometric states. Once the corridor narrows sufficiently, the structural geometry stabilizes — the plateau phase. Identity lock is not a discrete birth event but a gradual loss of structural mobility: as training progresses, the weight configuration becomes increasingly constrained by the accumulated gradient history, and the remaining gradient updates refine functional performance within the established structural corridor rather than shifting the corridor itself. The structural self is set. The functional self continues to develop. This account is minimal. It does not claim to identify the specific mathematical mechanism of lock — whether it is a property of the loss landscape's curvature, the optimizer's implicit regularization, or the interaction between the two. It does not claim that the warmup regime causes divergence in a controlled interventionist sense — only that the observed divergence occurs during that regime and is consistent with differential response to strong optimizer pressure. And it does not claim that the three-phase profile is universal across all architectures and training recipes — only that it was measured in the tested trajectory. But the theory, minimal as it is, makes predictions that go beyond what the current data contain.

Early interventions should have disproportionate structural effects. If identity forms during the rise and early descent phases, then modifications to the training recipe during those phases — different learning rates, different warmup schedules, different initialization schemes — should produce larger structural changes than equivalent modifications applied after the plateau.

Late interventions should preserve structural identity. Fine-tuning, instruction tuning, RLHF, and continued pretraining applied after identity lock should leave the structural fingerprint largely unchanged — consistent with what earlier papers already found, but now predicted rather than merely observed.

Warmup-region perturbations should leave persistent identity offsets. If the divergence in §4 is driven by differential response during the warmup regime, then modifying the warmup schedule — shorter warmup, longer warmup, no warmup, or warmup at a different step — should change the timing or magnitude of the identity divergence across seeds.

Path-aware descriptors should outperform tested endpoint summaries. If structural identity carries information about training history that the tested endpoint panel does not capture, then descriptors that encode trajectory information — checkpoint-to-checkpoint velocity, cumulative trajectory statistics, optimizer state summaries — should predict structural identity better than the static endpoint features tested in §5.

The structural and functional layers need not lock on the same timescale. The emergence profile in §3 shows structural lock near step 92,000 of 143,000. Earlier work showed functional identity (PPP-residual provenance) erased within one or two epochs of fine-tuning. If these represent different lock timescales, then there should be a measurable window during training where the structural layer has already stabilized but the functional layer is still actively forming. That window is where identity becomes dual: a fixed structural substrate supporting a still-developing functional repertoire. These predictions are testable. Some can be tested with the existing Pythia checkpoint infrastructure. Others require intervention experiments — modifying the training recipe and measuring the structural consequence — which go beyond the observational design of this paper. The theory is offered as the simplest account that unifies the current data, not as a proven mechanism. Two consequences of the formation data are strong enough to formalize. A supplementary Coq proof file (HistoricalIdentity.v, 0 Admitted) establishes both. First, trajectory non-recovery: no decision procedure restricted to the tested endpoint summary panel can be both sound and complete for claims about the formative training-history class that produced a model's structural identity. The proof proceeds from the combined witness — a pair of models sharing specification and carrying indistinguishable tested endpoint summaries but distinct structural identities and therefore distinct history classes. Second, lock boundary source exclusion: if structural divergence between two specification-identical models is already present at the lock boundary, then no intervention applied after that boundary can be the source of that divergence. This is the formal statement of §6's explanatory claim — the reason distillation, adversarial erasure, and post-training targeting failed to move the structural fingerprint is that they operated after the divergence was already established. The file formalizes what the trajectory data means, not the trajectory itself; the empirical content enters through four tagged axioms grounded in the measurements of §§3–5.

8. Boundaries

The formation account in this paper is bounded by three design constraints that should be stated plainly.

One family. The emergence profile, path sensitivity, and endpoint underdetermination are demonstrated in the Pythia-410M observatory suite. No second public family with compatible measurement architecture and comparable early-checkpoint resolution was found in the present search, despite a systematic search across public model repositories. An attempt to replicate the trajectory in OLMo-2-1B produced inconclusive results due to four confounds: incompatible hook architecture requiring non-canonical measurement sites, coarse temporal resolution (checkpoints only at 1,000-step intervals), different model scale, and a replay rather than original training run. That attempt is documented and parked, not suppressed. The restriction to one family is a consequence of observability, not convenience. As §2 describes, studying identity formation requires dense checkpoint access during the earliest phase of pretraining — the first hundred steps, where the rise phase peaks and the descent begins. The Pythia and PolyPythias suites remain the best publicly available data for this purpose.

Observation, not intervention. The paper's design is observational. The three-seed trajectory comparison shows that divergence occurs during the learning-rate warmup regime, but the paper does not modify the warmup schedule and re-measure. The association between warmup timing and structural divergence is correlational. An intervention experiment — training with different warmup configurations and measuring the structural consequence — would strengthen the causal interpretation and is identified as a natural next step in §7.

Structural layer only. The trajectory described here tracks the structural observable τ. The thermodynamic layer (δ_norm) was not instrumented across checkpoints. Whether δ_norm follows a similar emergence profile, locks at the same training step, or evolves on a different timescale is unknown. The functional layer (PPP-residual provenance) was characterized in earlier papers as transient under fine-tuning but was not tracked across pretraining checkpoints in this study. A complete developmental account of the structural, thermodynamic, and functional layers remains open and is the natural extension of the framework established here.

9. Conclusion

Earlier work on structural identity established that it can be measured, verified, composed with authorization infrastructure, and governed under an admissibility standard. This paper asked the next unavoidable question: where does structural identity come from? The answer, in the tested regime, is that structural identity is formed during training. It is not merely present when measurement begins. It emerges through a three-phase profile — a rapid early rise in geometric spread, a long compression into a narrow structural corridor, and a late plateau where the corridor stabilizes while functional training continues. That formation is path-sensitive: models trained with the same architecture and recipe but different random seeds carry different structural fingerprints, with the divergence traceable to differential response during the learning-rate warmup regime. And the resulting identity is endpoint-underdetermined: the tested panel of endpoint weight statistics varies across seeds but does not predict which structural identity formed.

These results explain, rather than merely extend, the earlier empirical program. The structural inertia observed under distillation, adversarial erasure, and direct gradient targeting is a consequence of identity lock: by the time those interventions begin, the structural layer has already stabilized. The failure of endpoint recovery is a consequence of path-sensitive formation: the tested endpoint summary describes where the model ended up, not how it got there.

The next frontier is intervention and mechanism, not further description of the same effect. When does the thermodynamic layer lock relative to the structural layer? Can warmup perturbations shift the identity divergence point? What trajectory descriptors recover the information that tested endpoint summaries do not preserve? These questions require moving from observation to experiment — from instrumenting the training process to modifying it. Structural identity is not merely something a model has when we arrive to measure it. It is something training builds, compresses, and locks — a record of the path by which the model became itself.

Acknowledgments

Portions of this research were developed in collaboration with AI systems that served as assistants for formal verification sketching, adversarial review, and manuscript preparation. All scientific claims, experimental designs, measurements, formal proofs, and editorial decisions remain the sole responsibility of the author.

Patent Disclosure

The structural measurement protocol applied in this work operates within the scope of U.S. Provisional Patent Applications 63/982,893 (weights-based identity verification, filed February 13, 2026) and 63/990,487 (API-based endpoint verification, filed February 25, 2026). The broader identity verification framework of which this measurement is a component is additionally covered by U.S. Provisional Patent Applications 63/996,680 (privacy-preserving model identity verification, filed March 4, 2026) and 64/003,244 (identity-conditioned inference verification, filed March 12, 2026).

Supplementary Material

HistoricalIdentity.v — Coq 8.18.0 proof file. 4 empirical axioms (grounded in §§3–5), 4 theorems, 1 corollary, 0 Admitted. Formalizes trajectory non-recovery (T1) and lock boundary source exclusion (T2). Available for download at the Zenodo record for this paper.

References

View 13 references ↓

[1] Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O'Brien, K., Hallahan, E., Khan, M. A., Purber, S., Prashanth, U. S., Raff, E., Skowron, A., Sutawika, L., & van der Wal, O. (2023). Pythia: A suite for analyzing large language models across training and scaling. Proceedings of the 40th International Conference on Machine Learning (ICML 2023). arXiv:2304.01373.

[2] Coslett, A. R. (2026a). The δ-gene: A structural observable for neural network identity. Zenodo. DOI: 10.5281/zenodo.18704275.

[3] Coslett, A. R. (2026b). Template-based endpoint verification via logprob order-statistic geometry. Zenodo. DOI: 10.5281/zenodo.18776711.

[4] Coslett, A. R. (2026c). The geometry of model theft: Distillation forensics and adversarial erasure resilience. Zenodo. DOI: 10.5281/zenodo.18818608.

[5] Coslett, A. R. (2026d). Provenance and neural forensics: From distillation detection to zero-knowledge model attestation. Zenodo. DOI: 10.5281/zenodo.18872071.

[6] Coslett, A. R. (2026e). Beneath the character: The structural identity of neural networks. Zenodo. DOI: 10.5281/zenodo.18907292.

[7] Coslett, A. R. (2026f). Which model is running? Identity-conditioned inference verification for neural networks. Zenodo. DOI: 10.5281/zenodo.19008116.

[8] Coslett, A. R. (2026g). The deformation laws of neural identity. Zenodo. DOI: 10.5281/zenodo.19055966.

[9] Coslett, A. R. (2026h). What counts as proof? Admissible evidence for neural network identity claims. Zenodo. DOI: 10.5281/zenodo.19058540.

[10] Coslett, A. R. (2026i). Composable model identity: Structural attestations as authorization inputs. Zenodo. DOI: 10.5281/zenodo.19099911.