The Audit Premise Does Not Carry Over

On July 30, 2026, the Cybersecurity and Infrastructure Security Agency issued Open Source Software: Security Principles and Practices, a federal-civilian resource that consolidates several years of open-source risk-management work into one document. The press release is direct about its statutory anchor. The guidance implements Executive Order 14144 and Executive Order 14306, the two orders that together define the current federal posture on secure software supply chains. The document is written for agencies that already use open-source software everywhere, and it consolidates a decade of accumulated practice into one place.

There is a paragraph inside that press release that is not consolidation. It is a boundary.

The guidance treats open source artificial intelligence systems as a separate evaluation object from open source software. The primary text reads that "for open source AI systems, the guidance urges agencies to obtain sufficient transparency into all relevant components, including training data, of the AI system before deeming the product as OSS for risk management purposes." The sentence continues: "Only with transparency and access can agencies understand and study the software, analyze it for vulnerabilities, and remediate any found vulnerabilities or risks."

The paragraph does not close a definition. It opens one. It asserts that the OSS label does not automatically carry to systems that share the surface of open source distribution but do not share the surface of open source audit.

What the OSS Audit Premise Actually Is

Federal open-source practice, as it has been assembled across the CISA Open Source Software Security Roadmap since 2023 and the earlier Enduring Security Framework recommended practices for managing OSS and SBOMs, rests on a single evidentiary premise. The premise is that source code visibility permits audit. An agency can look at the code, an agency can review the dependency tree, an agency can compare the compiled artifact to the source, an agency can compile from source in a controlled environment, and an agency can inspect the same artifact any adversary would inspect.

Software Bill of Materials work, licensing review, secure-by-design guidance, and the July 30 principles document all sit on that premise. Vet the package, examine the dependency chain, verify provenance, and the object being evaluated is the object being deployed.

The premise has an operational history. It answered questions like log4shell and xz utils, which the CISA release explicitly names as exemplars of the visibility argument. When the audit premise held, the response cycle worked. Agencies could identify affected components in their own installations, patch, verify, and continue.

Where the Audit Premise Fails

The July 30 language names, without elaboration, the boundary at which the audit premise stops carrying.

An open-weight AI model is distributed under an open license. The weights are downloadable. The inference code is often permissively licensed. An agency can, in the ordinary OSS sense, review and audit those artifacts. The problem is that the artifacts do not describe the system.

Training data is typically not released. Training methodology is typically not fully specified. Alignment parameters, safety training procedures, and reinforcement learning setups are typically documented at a level of abstraction that does not permit reproduction. In one recent illustration, the UK AI Security Institute plus CAISI preliminary assessment of Kimi K3 reported that a leading open-weight model, evaluated on cyber capability benchmarks four days before its planned open-weight release, produced results whose safeguard behavior could not be persistently controlled once the weights were public. The visible artifact, the weights, was audited. The invisible components, the training corpus and the alignment stack, were not available for the same audit.

The CISA guidance frames the boundary in evaluative terms. Secondary reporting on the same document, notably Open Source For You's coverage, attaches the label C4 Framework to the trustworthiness assessment method the guidance describes and states that agencies are instructed to apply stricter, distinct evaluation criteria to open weight AI models because their licenses do not typically expose the training corpus, the architecture at implementation granularity, or the alignment configuration. The CISA release itself does not use the C4 label. It uses functional language. The guidance "includes established principles for patching, a framework to evaluate trustworthiness and risk tolerance, and best practices to engage with OSS securely, responsibly and sustainably." The evaluative frame is present. The label is downstream.

The Instrument and the Function

The federal open-source security instrument now covers open source AI. What it does not cover is the artifact from which an open source AI system's behavior originates.

An agency vetting a Python package examines the code the machine will execute. An agency vetting an open-weight model examines a distribution of numbers whose relationship to the model's behavior is empirical rather than legible. The training corpus that shaped those numbers is not part of the license. The alignment training that shaped the safety behavior is not part of the license. The evaluation pipeline that would allow the agency to reproduce the vendor's stated safety properties is not part of the license.

The instrument treats the two as adjacent. The function they can perform is not adjacent. An audit of source code closes over the artifact. An audit of weights does not.

This is not the first governance instrument in the current window to describe a gap of this shape. The Kaur evidential ceiling paper published a week earlier described an evaluation regime that can only prove what its test procedures can elicit. The HANDBOOK.md benchmark published two days before the CISA guidance described a policy document retained in context that decays as the operative control on agent behavior. Each of these instruments describes a different layer of the same problem. The artifact is retained. The function the artifact was written to perform is bounded by structural features of the object being governed, not by the license under which it is distributed.

What Remains on the Table

• What does an agency do when the license permits redistribution but the training data does not permit reproduction?
• Who holds the specification of open weight when the vendor's audit surface is not the audit surface the federal instrument was built to evaluate?
• If the guidance names transparency into training data as a precondition for the OSS label, what evidentiary standard resolves whether a given release meets it?
• Where does the observation of an open weight AI system reside once the weights are public and the deployment safeguards cannot be applied uniformly across every downstream instance?

The governance artifact is retained. The governance function is not.