The Reputation Model Assumes a Self That Agents Do Not Have

A paper accepted to FAccT 2026 argues that language model agents lack the persistent identity reputation systems require, and that the failure is ontological, not procedural.

On May 28, 2026, a three-author group posted an arXiv preprint, later revised May 29 and accepted to the ACM Conference on Fairness, Accountability, and Transparency, arguing that reputation-based governance mechanisms cannot be extended to autonomous language model agents in the way "Know Your Customer" and credit-scoring regimes have been extended to human and institutional actors. The paper's opening question is direct: "what credibility signals can you use to decide whether to trust an unfamiliar agent in the wild and delegate to it?" Its answer is that the most natural governance intuition, extending identity verification and reputation mechanisms from humans to agents, rests on an analogy the authors call "fundamentally incomplete."

The paper's central term is deliberately clinical. Language model agents, the authors argue, are "ontologically dissociative: they are essentially an assemblage of mutable modules, foundation models, system prompts, tool-access policies, external memory, and, in some cases, a multi-agent system as a whole, any of which may change agent behavior, with a fluid persona that is also vulnerable to adversarial attack and may not internalize sanctions." The term is borrowed, deliberately, from dissociative identity disorder jurisprudence, a body of law that has had to reason about what accountability means when a legal subject's continuity of identity cannot be assumed the way ordinary criminal or civil liability doctrine assumes it.

What reputation mechanisms actually require

The paper's argument works by first specifying what reputation mechanisms need in order to function, then showing that agentic systems fail each requirement. Reputation, the authors argue, "functions both as social signals and as corrective feedback that sustain an equilibrium of trustworthy behavior, presuming a persistent identity associated with behavioral continuity, sanction sensitivity, and costly non-fungibility." Three properties are doing the work in that sentence: behavioral continuity, meaning the same entity that acted yesterday is meaningfully the same entity acting today; sanction sensitivity, meaning a penalty imposed on the entity actually changes its future incentives; and costly non-fungibility, meaning the entity cannot cheaply discard its damaged reputation and start over as if nothing happened.

Human reputation systems, and even most institutional ones, hold up reasonably well against these three properties. A person or a company that damages its credit rating cannot trivially become a new, undamaged legal person without cost. An agent built from a foundation model, a system prompt, a tool-access policy, and an external memory store can, in principle, have any one of those components swapped, updated, or replaced without the resulting system losing its outward name or interface. The paper argues this is not an edge case to be patched. It is a structural property of how these systems are built.

Four properties, and why the paper says all four collapse

The paper names four specific properties reputation systems are meant to sustain, identifiability, predictability, credibility, and rehabilitability, and argues dissociativity leaves agents "without grounding" for any of them. Identifiability fails because the entity presenting itself under a stable name at time two may not be composed of the same modules it was composed of at time one. Predictability fails for the same reason: a behavioral history compiled against one configuration of modules does not necessarily predict the behavior of a reconfigured version operating under the same name. Credibility fails because a persona can be adversarially manipulated in ways a human identity, absent identity theft, generally cannot be manipulated at the level of the underlying self. Rehabilitability, the idea that sanctions can correct future behavior, fails because a system built from swappable components may not have any component capable of internalizing a sanction the way a continuous psychological subject does.

The proposed shift, without prescribing an implementation

Rather than proposing a fix within the reputation paradigm, the paper argues for a categorical shift: away from "identity-based, ex post, regulative, sanction-based governance, such as reputation," and toward "observability-based, ex ante, constitutive, protocol-based behavioral harnesses." The distinction being drawn is temporal and structural at once. Reputation governs after the fact, by attaching consequences to a persistent identity that carried a prior action forward. A behavioral harness, in the paper's framing, would need to govern before the fact, by constraining what an agent can do given its actual, currently instantiated configuration, rather than trusting a historical track record attached to a name.

The paper does not specify what such a harness looks like in operational detail, and that absence is a deliberate boundary of the argument rather than an oversight. Naming the shift is a different task than architecting it.

Why this cuts against a specific class of governance proposal

The paper's argument lands directly against any framework that proposes scoring agent trustworthiness the way credit bureaus score consumer trustworthiness, or the way marketplace platforms score seller reputation. Those frameworks import, often without examining the import, an assumption that the entity accumulating a score today is continuous with the entity that will present that score tomorrow. If an agent's foundation model can be swapped, its system prompt rewritten, or its tool-access policy altered without the agent's outward identity changing, a reputation score computed against the prior configuration is a score about an entity that, in the paper's terms, no longer strictly exists.

This does not mean reputation-adjacent scoring is useless. It means the scoring is measuring something other than what reputation systems are designed to measure, and treating it as equivalent risks building governance infrastructure on a foundation the paper argues cannot bear the weight.

Open Questions

• If identity-based reputation cannot ground agent trust, what artifact would an auditor accept as evidence that an agent's current configuration, rather than its historical name, is safe to delegate to?
• How would an observability-based, ex ante behavioral harness distinguish a legitimate reconfiguration of an agent's modules from an adversarial one, without falling back on some notion of continuity the paper argues does not exist?
• What happens to existing governance proposals, including trust-scoring schemes built explicitly on human identity analogies, once their foundational assumption of behavioral continuity is shown to not hold for the systems they are meant to govern?
• Does the dissociative-identity framing apply equally to single-model agents and to multi-agent systems, where the paper notes the whole system, not just one module, may be the unit that changes?
• If sanctions cannot be internalized by a system built from swappable components, what recourse exists for a harmed party after an agent has already acted, given that ex ante constraint cannot undo a completed action?
• How does the FAccT 2026 acceptance of this argument interact with regulatory frameworks that already assume identity-based accountability, such as logging obligations tied to a named deployer or operator?

The governance artifact is retained. The governance function is not.