build: bind M1 acceptance provenance

This commit is contained in:
Codex 2026-10-21 15:26:33 +02:00
parent fe2625e423
commit 9230ec514b
9 changed files with 252 additions and 22 deletions

View file

@ -18,7 +18,8 @@ The repository now provides:
toolchain; toolchain;
- a non-executing verifier for externally produced Windows artifacts; - a non-executing verifier for externally produced Windows artifacts;
- a checksummed acceptance-kit generator containing only allowlisted public - a checksummed acceptance-kit generator containing only allowlisted public
synthetic inputs; synthetic inputs, with provenance binding to the repository commit and
non-executing artifact-verification report;
- a fail-closed manual desktop-acceptance report and validator; - a fail-closed manual desktop-acceptance report and validator;
- pinned upstream contract/license manifests and a drift checker; - pinned upstream contract/license manifests and a drift checker;
- a fail-closed API-23 adapter isolating Rayforge's internal driver registry; - a fail-closed API-23 adapter isolating Rayforge's internal driver registry;

View file

@ -83,16 +83,21 @@ Use only the isolated no-device profile and an empty work area. Do not connect
or configure a controller. or configure a controller.
Prepare a fresh, checksummed kit containing only the public synthetic inputs Prepare a fresh, checksummed kit containing only the public synthetic inputs
and a new report template: and a new report template. Bind the passing non-executing artifact-verification
report into the kit at creation time:
```powershell ```powershell
python scripts/prepare_m1_acceptance_kit.py python scripts/prepare_m1_acceptance_kit.py `
--artifact-verification .build/artifacts/rayforge-windows-verification.json
``` ```
The default output is `.build/acceptance-kit/`. The generator uses a fixed The default output is `.build/acceptance-kit/`. The generator uses a fixed
public allowlist, records the repository commit and refuses to overwrite a public allowlist, records the repository commit and refuses to overwrite a
non-empty directory. It does not include the application bundle, private non-empty directory. It does not include the application bundle, private
LightBurn evidence or any machine data. LightBurn evidence or any machine data. Acceptance-kit schema 2 binds the
generated manual report to the kit manifest, repository commit and copied
artifact-verification report by SHA-256. Editing, replacing or referring to an
external provenance report therefore fails validation.
1. Pass the non-executing artifact verifier above. 1. Pass the non-executing artifact verifier above.
2. Launch the portable build with a fresh application-data directory. 2. Launch the portable build with a fresh application-data directory.
@ -121,7 +126,8 @@ python scripts/validate_m1_manual_acceptance.py `
--output .build/acceptance-kit/manual-validation.json --output .build/acceptance-kit/manual-validation.json
``` ```
The validator verifies evidence hashes and relative paths. It reports The validator verifies the kit manifest, repository commit, artifact report,
evidence hashes and relative paths. It reports
`incomplete` for any unrun or blocked scenario and `failed` for a failed step, `incomplete` for any unrun or blocked scenario and `failed` for a failed step,
connected hardware, non-isolated app data, changed evidence or any stop connected hardware, non-isolated app data, changed evidence or any stop
condition. Even a complete `passed` report still requires human review and condition. Even a complete `passed` report still requires human review and

View file

@ -330,7 +330,9 @@ validator. Missing steps remain incomplete, stop conditions fail the run, and
hashed screenshots stay outside the public repository pending sanitization. hashed screenshots stay outside the public repository pending sanitization.
A reproducible kit generator copies only allowlisted synthetic inputs into the A reproducible kit generator copies only allowlisted synthetic inputs into the
ignored build area, records their hashes and refuses to overwrite an existing ignored build area, records their hashes and refuses to overwrite an existing
test run. test run. Its schema-2 provenance chain binds the editable manual report to the
exact kit manifest, repository commit and non-executing artifact-verification
report; substituted or changed inputs fail validation.
The pinned upstream contracts and license notices are now machine-verifiable. The pinned upstream contracts and license notices are now machine-verifiable.
Rayforge's documented addon lifecycle is viable, but production driver Rayforge's documented addon lifecycle is viable, but production driver
registration still needs a supported symmetric registry API. The current registration still needs a supported symmetric registry API. The current

View file

@ -83,8 +83,9 @@ evidence for:
- a versioned manual desktop-acceptance template and validator that cannot - a versioned manual desktop-acceptance template and validator that cannot
pass missing steps, connected hardware, changed evidence or stop conditions; pass missing steps, connected hardware, changed evidence or stop conditions;
- a deterministic acceptance-kit generator that copies only four allowlisted - a deterministic acceptance-kit generator that copies only four allowlisted
public synthetic inputs, hashes them and refuses to overwrite an existing public template/input sources, hashes the three immutable inputs, refuses to
run directory; overwrite an existing run directory, and binds the report to the exact kit,
repository commit and artifact-verification report;
- pinned Rayforge and MeerK40t license/source contracts with drift tests, plus - pinned Rayforge and MeerK40t license/source contracts with drift tests, plus
checkpoint schema 2 requiring the Rayforge MIT notice in the portable ZIP checkpoint schema 2 requiring the Rayforge MIT notice in the portable ZIP
and beside the installer manifest; and beside the installer manifest;

View file

@ -11,8 +11,8 @@ from pathlib import Path
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
REPORT_TEMPLATE = "tests/fixtures/m1-manual-acceptance-template.json"
KIT_FILES = { KIT_FILES = {
"manual-report.json": "tests/fixtures/m1-manual-acceptance-template.json",
"inputs/synthetic-mopa-import.lbrn2": "tests/fixtures/synthetic-mopa-import.lbrn2", "inputs/synthetic-mopa-import.lbrn2": "tests/fixtures/synthetic-mopa-import.lbrn2",
"inputs/synthetic-jcz-device.lbdev": "tests/fixtures/synthetic-jcz-device.lbdev", "inputs/synthetic-jcz-device.lbdev": "tests/fixtures/synthetic-jcz-device.lbdev",
"inputs/synthetic-pirateray-recipe-metadata.json": ( "inputs/synthetic-pirateray-recipe-metadata.json": (
@ -36,7 +36,25 @@ def _repository_commit() -> str:
return result.stdout.strip() return result.stdout.strip()
def create_kit(output: Path, repository_commit: str | None = None) -> dict: def create_kit(
output: Path,
repository_commit: str | None = None,
artifact_verification: Path | None = None,
) -> dict:
verification_source = None
if artifact_verification is not None:
verification_source = artifact_verification.resolve()
verification = json.loads(
verification_source.read_text(encoding="utf-8-sig")
)
if (
verification.get("status") != "passed"
or verification.get("hardware_access") is not False
or verification.get("executed_binaries") is not False
or verification.get("manual_interaction_required") is not True
):
raise ValueError("artifact-verification report did not pass the no-device gate")
output = output.resolve() output = output.resolve()
if output.exists() and any(output.iterdir()): if output.exists() and any(output.iterdir()):
raise ValueError(f"acceptance-kit directory is not empty: {output}") raise ValueError(f"acceptance-kit directory is not empty: {output}")
@ -56,12 +74,18 @@ def create_kit(output: Path, repository_commit: str | None = None) -> dict:
"sha256": _sha256(target), "sha256": _sha256(target),
}) })
commit = repository_commit or _repository_commit()
template_path = ROOT / REPORT_TEMPLATE
manifest = { manifest = {
"schema_version": 1, "schema_version": 2,
"kit": "M1 packaged no-device desktop acceptance", "kit": "M1 packaged no-device desktop acceptance",
"repository_commit": repository_commit or _repository_commit(), "repository_commit": commit,
"hardware_access_authorized": False, "hardware_access_authorized": False,
"contains_private_evidence": False, "contains_private_evidence": False,
"report_template": {
"source": REPORT_TEMPLATE,
"sha256": _sha256(template_path),
},
"files": records, "files": records,
} }
manifest_path = output / "kit-manifest.json" manifest_path = output / "kit-manifest.json"
@ -70,6 +94,25 @@ def create_kit(output: Path, repository_commit: str | None = None) -> dict:
encoding="utf-8", encoding="utf-8",
newline="\n", newline="\n",
) )
report = json.loads(template_path.read_text(encoding="utf-8"))
report["acceptance_kit"] = {
"manifest_path": "kit-manifest.json",
"manifest_sha256": _sha256(manifest_path),
"repository_commit": commit,
}
if verification_source is not None:
verification_target = output / "artifact-verification.json"
shutil.copyfile(verification_source, verification_target)
report["artifact_verification"] = {
"status": "passed",
"report_path": verification_target.name,
"report_sha256": _sha256(verification_target),
}
(output / "manual-report.json").write_text(
json.dumps(report, indent=2) + "\n",
encoding="utf-8",
newline="\n",
)
return manifest return manifest
@ -80,8 +123,13 @@ def main() -> int:
type=Path, type=Path,
default=ROOT / ".build" / "acceptance-kit", default=ROOT / ".build" / "acceptance-kit",
) )
parser.add_argument(
"--artifact-verification",
type=Path,
help="passing JSON output from verify_rayforge_windows_checkpoint.py",
)
args = parser.parse_args() args = parser.parse_args()
result = create_kit(args.output) result = create_kit(args.output, artifact_verification=args.artifact_verification)
print(f"kit={args.output.resolve()}") print(f"kit={args.output.resolve()}")
print(f"files={len(result['files'])}") print(f"files={len(result['files'])}")
print(f"repository_commit={result['repository_commit']}") print(f"repository_commit={result['repository_commit']}")

View file

@ -32,6 +32,7 @@ STOP_CONDITIONS = (
) )
STEP_STATUSES = {"passed", "failed", "blocked", "not_run"} STEP_STATUSES = {"passed", "failed", "blocked", "not_run"}
SHA256 = re.compile(r"^[0-9a-f]{64}$") SHA256 = re.compile(r"^[0-9a-f]{64}$")
GIT_COMMIT = re.compile(r"^[0-9a-f]{40}$")
def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]: def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]:
@ -58,11 +59,66 @@ def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]:
return errors return errors
def _bound_json(report_root: Path, binding: dict, label: str) -> tuple[dict | None, list[str]]:
value = binding.get("report_path") or binding.get("manifest_path") or ""
relative = PurePosixPath(value)
if not value or relative.is_absolute() or ".." in relative.parts:
return None, [f"unsafe {label} path: {value!r}"]
path = report_root.joinpath(*relative.parts)
if not path.is_file():
return None, [f"{label} is missing: {value}"]
digest = binding.get("report_sha256") or binding.get("manifest_sha256") or ""
actual = hashlib.sha256(path.read_bytes()).hexdigest()
if not SHA256.fullmatch(digest) or digest != actual:
return None, [f"{label} SHA-256 mismatch: {value}"]
try:
return json.loads(path.read_text(encoding="utf-8-sig")), []
except (UnicodeDecodeError, json.JSONDecodeError):
return None, [f"{label} is not valid JSON: {value}"]
def _validate_kit(report_root: Path, binding: dict) -> list[str]:
manifest, errors = _bound_json(report_root, binding, "acceptance-kit manifest")
if errors:
return errors
if manifest.get("schema_version") != 2:
errors.append("unsupported acceptance-kit schema")
if manifest.get("kit") != "M1 packaged no-device desktop acceptance":
errors.append("unexpected acceptance-kit name")
commit = binding.get("repository_commit", "")
if not GIT_COMMIT.fullmatch(commit) or manifest.get("repository_commit") != commit:
errors.append("acceptance-kit repository commit mismatch")
if manifest.get("hardware_access_authorized") is not False:
errors.append("acceptance kit does not preserve the hardware gate")
if manifest.get("contains_private_evidence") is not False:
errors.append("acceptance kit contains private evidence")
for record in manifest.get("files", []):
errors.extend(_validate_evidence(report_root, [record]))
return errors
def _validate_artifact_report(report_root: Path, binding: dict) -> list[str]:
verification, errors = _bound_json(
report_root, binding, "artifact-verification report"
)
if errors:
return errors
if verification.get("status") != "passed":
errors.append("artifact verification did not pass")
if verification.get("hardware_access") is not False:
errors.append("artifact verification does not preserve the hardware gate")
if verification.get("executed_binaries") is not False:
errors.append("artifact verification executed binaries")
if verification.get("manual_interaction_required") is not True:
errors.append("artifact verification dropped the manual interaction gate")
return errors
def validate(report_path: Path) -> dict: def validate(report_path: Path) -> dict:
report_path = report_path.resolve() report_path = report_path.resolve()
report = json.loads(report_path.read_text(encoding="utf-8-sig")) report = json.loads(report_path.read_text(encoding="utf-8-sig"))
errors = [] errors = []
if report.get("schema_version") != 1: if report.get("schema_version") != 2:
errors.append("unsupported report schema") errors.append("unsupported report schema")
if report.get("checkpoint") != "M1 packaged no-device desktop acceptance": if report.get("checkpoint") != "M1 packaged no-device desktop acceptance":
errors.append("unexpected checkpoint name") errors.append("unexpected checkpoint name")
@ -83,6 +139,10 @@ def validate(report_path: Path) -> dict:
environment = report.get("environment", {}) environment = report.get("environment", {})
artifact = report.get("artifact_verification", {}) artifact = report.get("artifact_verification", {})
kit = report.get("acceptance_kit", {})
errors.extend(_validate_kit(report_path.parent, kit))
artifact_errors = _validate_artifact_report(report_path.parent, artifact)
errors.extend(artifact_errors)
errors.extend(_validate_evidence(report_path.parent, report.get("evidence", []))) errors.extend(_validate_evidence(report_path.parent, report.get("evidence", [])))
explicit_failure = ( explicit_failure = (
@ -101,7 +161,7 @@ def validate(report_path: Path) -> dict:
and environment.get("fresh_app_data") is True and environment.get("fresh_app_data") is True
and bool(environment.get("windows_version")) and bool(environment.get("windows_version"))
and artifact.get("status") == "passed" and artifact.get("status") == "passed"
and SHA256.fullmatch(artifact.get("report_sha256", "")) is not None and not artifact_errors
) )
status = "failed" if explicit_failure or errors else "passed" if complete else "incomplete" status = "failed" if explicit_failure or errors else "passed" if complete else "incomplete"
return { return {

View file

@ -1,8 +1,14 @@
{ {
"schema_version": 1, "schema_version": 2,
"checkpoint": "M1 packaged no-device desktop acceptance", "checkpoint": "M1 packaged no-device desktop acceptance",
"acceptance_kit": {
"manifest_path": null,
"manifest_sha256": null,
"repository_commit": null
},
"artifact_verification": { "artifact_verification": {
"status": "not_run", "status": "not_run",
"report_path": null,
"report_sha256": null "report_sha256": null
}, },
"environment": { "environment": {

View file

@ -4,7 +4,12 @@ import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from scripts.prepare_m1_acceptance_kit import KIT_FILES, ROOT, create_kit from scripts.prepare_m1_acceptance_kit import (
KIT_FILES,
REPORT_TEMPLATE,
ROOT,
create_kit,
)
class M1AcceptanceKitTests(unittest.TestCase): class M1AcceptanceKitTests(unittest.TestCase):
@ -14,6 +19,11 @@ class M1AcceptanceKitTests(unittest.TestCase):
manifest = create_kit(output, repository_commit="a" * 40) manifest = create_kit(output, repository_commit="a" * 40)
self.assertFalse(manifest["hardware_access_authorized"]) self.assertFalse(manifest["hardware_access_authorized"])
self.assertFalse(manifest["contains_private_evidence"]) self.assertFalse(manifest["contains_private_evidence"])
self.assertEqual(2, manifest["schema_version"])
self.assertEqual(
hashlib.sha256((ROOT / REPORT_TEMPLATE).read_bytes()).hexdigest(),
manifest["report_template"]["sha256"],
)
self.assertEqual( self.assertEqual(
{record["path"] for record in manifest["files"]}, {record["path"] for record in manifest["files"]},
set(KIT_FILES), set(KIT_FILES),
@ -23,6 +33,12 @@ class M1AcceptanceKitTests(unittest.TestCase):
self.assertEqual(record["bytes"], path.stat().st_size) self.assertEqual(record["bytes"], path.stat().st_size)
self.assertEqual(record["sha256"], hashlib.sha256(path.read_bytes()).hexdigest()) self.assertEqual(record["sha256"], hashlib.sha256(path.read_bytes()).hexdigest())
self.assertEqual(record["classification"], "synthetic-public") self.assertEqual(record["classification"], "synthetic-public")
report = json.loads((output / "manual-report.json").read_text(encoding="utf-8"))
self.assertEqual("a" * 40, report["acceptance_kit"]["repository_commit"])
self.assertEqual(
hashlib.sha256((output / "kit-manifest.json").read_bytes()).hexdigest(),
report["acceptance_kit"]["manifest_sha256"],
)
def test_manifest_is_reproducible_for_same_commit(self): def test_manifest_is_reproducible_for_same_commit(self):
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory: with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
@ -50,6 +66,48 @@ class M1AcceptanceKitTests(unittest.TestCase):
actual = json.loads((output / "kit-manifest.json").read_text(encoding="utf-8")) actual = json.loads((output / "kit-manifest.json").read_text(encoding="utf-8"))
self.assertEqual(actual, expected) self.assertEqual(actual, expected)
def test_passing_artifact_verification_is_bound_into_report(self):
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
root = Path(directory)
verification = root / "verification.json"
verification.write_text(json.dumps({
"status": "passed",
"hardware_access": False,
"executed_binaries": False,
"manual_interaction_required": True,
}), encoding="utf-8")
output = root / "kit"
create_kit(
output,
repository_commit="e" * 40,
artifact_verification=verification,
)
copied = output / "artifact-verification.json"
report = json.loads((output / "manual-report.json").read_text(encoding="utf-8"))
copied_digest = hashlib.sha256(copied.read_bytes()).hexdigest()
self.assertEqual("passed", report["artifact_verification"]["status"])
self.assertEqual(
copied_digest,
report["artifact_verification"]["report_sha256"],
)
def test_failed_artifact_verification_is_rejected(self):
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
root = Path(directory)
verification = root / "verification.json"
verification.write_text(json.dumps({
"status": "failed",
"hardware_access": False,
"executed_binaries": False,
"manual_interaction_required": True,
}), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "did not pass"):
create_kit(
root / "kit",
repository_commit="f" * 40,
artifact_verification=verification,
)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

View file

@ -20,12 +20,41 @@ class M1ManualAcceptanceTests(unittest.TestCase):
path.write_text(json.dumps(report), encoding="utf-8") path.write_text(json.dumps(report), encoding="utf-8")
return path return path
def _passing(self, root: Path) -> dict: def _bind_reports(self, root: Path, report: dict) -> None:
report = copy.deepcopy(self.template) manifest = {
"schema_version": 2,
"kit": "M1 packaged no-device desktop acceptance",
"repository_commit": "b" * 40,
"hardware_access_authorized": False,
"contains_private_evidence": False,
"files": [],
}
manifest_path = root / "kit-manifest.json"
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
report["acceptance_kit"] = {
"manifest_path": manifest_path.name,
"manifest_sha256": hashlib.sha256(manifest_path.read_bytes()).hexdigest(),
"repository_commit": "b" * 40,
}
verification = {
"status": "passed",
"hardware_access": False,
"executed_binaries": False,
"manual_interaction_required": True,
}
verification_path = root / "artifact-verification.json"
verification_path.write_text(json.dumps(verification), encoding="utf-8")
report["artifact_verification"] = { report["artifact_verification"] = {
"status": "passed", "status": "passed",
"report_sha256": "a" * 64, "report_path": verification_path.name,
"report_sha256": hashlib.sha256(
verification_path.read_bytes()
).hexdigest(),
} }
def _passing(self, root: Path) -> dict:
report = copy.deepcopy(self.template)
self._bind_reports(root, report)
report["environment"] = { report["environment"] = {
"windows_version": "synthetic-test-environment", "windows_version": "synthetic-test-environment",
"hardware_disconnected": True, "hardware_disconnected": True,
@ -37,7 +66,7 @@ class M1ManualAcceptanceTests(unittest.TestCase):
key: False for key in report["stop_conditions"] key: False for key in report["stop_conditions"]
} }
screenshot = root / "evidence" / "offline-state.png" screenshot = root / "evidence" / "offline-state.png"
screenshot.parent.mkdir() screenshot.parent.mkdir(exist_ok=True)
screenshot.write_bytes(b"synthetic screenshot placeholder") screenshot.write_bytes(b"synthetic screenshot placeholder")
report["evidence"] = [{ report["evidence"] = [{
"path": "evidence/offline-state.png", "path": "evidence/offline-state.png",
@ -47,7 +76,11 @@ class M1ManualAcceptanceTests(unittest.TestCase):
def test_template_is_valid_but_incomplete(self): def test_template_is_valid_but_incomplete(self):
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory: with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
result = validate(self._write(Path(directory), self.template)) root = Path(directory)
report = copy.deepcopy(self.template)
self._bind_reports(root, report)
report["artifact_verification"]["status"] = "not_run"
result = validate(self._write(root, report))
self.assertEqual(result["status"], "incomplete") self.assertEqual(result["status"], "incomplete")
self.assertTrue(result["schema_valid"]) self.assertTrue(result["schema_valid"])
self.assertEqual(result["required_steps"], len(STEP_IDS)) self.assertEqual(result["required_steps"], len(STEP_IDS))
@ -88,6 +121,21 @@ class M1ManualAcceptanceTests(unittest.TestCase):
result = validate(self._write(root, report)) result = validate(self._write(root, report))
self.assertEqual(result["status"], "failed") self.assertEqual(result["status"], "failed")
def test_unbound_or_changed_provenance_reports_are_rejected(self):
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
root = Path(directory)
report = self._passing(root)
report["artifact_verification"]["report_sha256"] = "0" * 64
result = validate(self._write(root, report))
self.assertEqual(result["status"], "failed")
self.assertIn("artifact-verification report SHA-256 mismatch", result["errors"][0])
report = self._passing(root)
report["acceptance_kit"]["repository_commit"] = "c" * 40
result = validate(self._write(root, report))
self.assertEqual(result["status"], "failed")
self.assertIn("acceptance-kit repository commit mismatch", result["errors"])
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()