build: bind M1 acceptance provenance
This commit is contained in:
parent
fe2625e423
commit
9230ec514b
9 changed files with 252 additions and 22 deletions
|
|
@ -18,7 +18,8 @@ The repository now provides:
|
||||||
toolchain;
|
toolchain;
|
||||||
- a non-executing verifier for externally produced Windows artifacts;
|
- a non-executing verifier for externally produced Windows artifacts;
|
||||||
- a checksummed acceptance-kit generator containing only allowlisted public
|
- a checksummed acceptance-kit generator containing only allowlisted public
|
||||||
synthetic inputs;
|
synthetic inputs, with provenance binding to the repository commit and
|
||||||
|
non-executing artifact-verification report;
|
||||||
- a fail-closed manual desktop-acceptance report and validator;
|
- a fail-closed manual desktop-acceptance report and validator;
|
||||||
- pinned upstream contract/license manifests and a drift checker;
|
- pinned upstream contract/license manifests and a drift checker;
|
||||||
- a fail-closed API-23 adapter isolating Rayforge's internal driver registry;
|
- a fail-closed API-23 adapter isolating Rayforge's internal driver registry;
|
||||||
|
|
|
||||||
|
|
@ -83,16 +83,21 @@ Use only the isolated no-device profile and an empty work area. Do not connect
|
||||||
or configure a controller.
|
or configure a controller.
|
||||||
|
|
||||||
Prepare a fresh, checksummed kit containing only the public synthetic inputs
|
Prepare a fresh, checksummed kit containing only the public synthetic inputs
|
||||||
and a new report template:
|
and a new report template. Bind the passing non-executing artifact-verification
|
||||||
|
report into the kit at creation time:
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
python scripts/prepare_m1_acceptance_kit.py
|
python scripts/prepare_m1_acceptance_kit.py `
|
||||||
|
--artifact-verification .build/artifacts/rayforge-windows-verification.json
|
||||||
```
|
```
|
||||||
|
|
||||||
The default output is `.build/acceptance-kit/`. The generator uses a fixed
|
The default output is `.build/acceptance-kit/`. The generator uses a fixed
|
||||||
public allowlist, records the repository commit and refuses to overwrite a
|
public allowlist, records the repository commit and refuses to overwrite a
|
||||||
non-empty directory. It does not include the application bundle, private
|
non-empty directory. It does not include the application bundle, private
|
||||||
LightBurn evidence or any machine data.
|
LightBurn evidence or any machine data. Acceptance-kit schema 2 binds the
|
||||||
|
generated manual report to the kit manifest, repository commit and copied
|
||||||
|
artifact-verification report by SHA-256. Editing, replacing or referring to an
|
||||||
|
external provenance report therefore fails validation.
|
||||||
|
|
||||||
1. Pass the non-executing artifact verifier above.
|
1. Pass the non-executing artifact verifier above.
|
||||||
2. Launch the portable build with a fresh application-data directory.
|
2. Launch the portable build with a fresh application-data directory.
|
||||||
|
|
@ -121,7 +126,8 @@ python scripts/validate_m1_manual_acceptance.py `
|
||||||
--output .build/acceptance-kit/manual-validation.json
|
--output .build/acceptance-kit/manual-validation.json
|
||||||
```
|
```
|
||||||
|
|
||||||
The validator verifies evidence hashes and relative paths. It reports
|
The validator verifies the kit manifest, repository commit, artifact report,
|
||||||
|
evidence hashes and relative paths. It reports
|
||||||
`incomplete` for any unrun or blocked scenario and `failed` for a failed step,
|
`incomplete` for any unrun or blocked scenario and `failed` for a failed step,
|
||||||
connected hardware, non-isolated app data, changed evidence or any stop
|
connected hardware, non-isolated app data, changed evidence or any stop
|
||||||
condition. Even a complete `passed` report still requires human review and
|
condition. Even a complete `passed` report still requires human review and
|
||||||
|
|
|
||||||
|
|
@ -330,7 +330,9 @@ validator. Missing steps remain incomplete, stop conditions fail the run, and
|
||||||
hashed screenshots stay outside the public repository pending sanitization.
|
hashed screenshots stay outside the public repository pending sanitization.
|
||||||
A reproducible kit generator copies only allowlisted synthetic inputs into the
|
A reproducible kit generator copies only allowlisted synthetic inputs into the
|
||||||
ignored build area, records their hashes and refuses to overwrite an existing
|
ignored build area, records their hashes and refuses to overwrite an existing
|
||||||
test run.
|
test run. Its schema-2 provenance chain binds the editable manual report to the
|
||||||
|
exact kit manifest, repository commit and non-executing artifact-verification
|
||||||
|
report; substituted or changed inputs fail validation.
|
||||||
The pinned upstream contracts and license notices are now machine-verifiable.
|
The pinned upstream contracts and license notices are now machine-verifiable.
|
||||||
Rayforge's documented addon lifecycle is viable, but production driver
|
Rayforge's documented addon lifecycle is viable, but production driver
|
||||||
registration still needs a supported symmetric registry API. The current
|
registration still needs a supported symmetric registry API. The current
|
||||||
|
|
|
||||||
|
|
@ -83,8 +83,9 @@ evidence for:
|
||||||
- a versioned manual desktop-acceptance template and validator that cannot
|
- a versioned manual desktop-acceptance template and validator that cannot
|
||||||
pass missing steps, connected hardware, changed evidence or stop conditions;
|
pass missing steps, connected hardware, changed evidence or stop conditions;
|
||||||
- a deterministic acceptance-kit generator that copies only four allowlisted
|
- a deterministic acceptance-kit generator that copies only four allowlisted
|
||||||
public synthetic inputs, hashes them and refuses to overwrite an existing
|
public template/input sources, hashes the three immutable inputs, refuses to
|
||||||
run directory;
|
overwrite an existing run directory, and binds the report to the exact kit,
|
||||||
|
repository commit and artifact-verification report;
|
||||||
- pinned Rayforge and MeerK40t license/source contracts with drift tests, plus
|
- pinned Rayforge and MeerK40t license/source contracts with drift tests, plus
|
||||||
checkpoint schema 2 requiring the Rayforge MIT notice in the portable ZIP
|
checkpoint schema 2 requiring the Rayforge MIT notice in the portable ZIP
|
||||||
and beside the installer manifest;
|
and beside the installer manifest;
|
||||||
|
|
|
||||||
|
|
@ -11,8 +11,8 @@ from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[1]
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
REPORT_TEMPLATE = "tests/fixtures/m1-manual-acceptance-template.json"
|
||||||
KIT_FILES = {
|
KIT_FILES = {
|
||||||
"manual-report.json": "tests/fixtures/m1-manual-acceptance-template.json",
|
|
||||||
"inputs/synthetic-mopa-import.lbrn2": "tests/fixtures/synthetic-mopa-import.lbrn2",
|
"inputs/synthetic-mopa-import.lbrn2": "tests/fixtures/synthetic-mopa-import.lbrn2",
|
||||||
"inputs/synthetic-jcz-device.lbdev": "tests/fixtures/synthetic-jcz-device.lbdev",
|
"inputs/synthetic-jcz-device.lbdev": "tests/fixtures/synthetic-jcz-device.lbdev",
|
||||||
"inputs/synthetic-pirateray-recipe-metadata.json": (
|
"inputs/synthetic-pirateray-recipe-metadata.json": (
|
||||||
|
|
@ -36,7 +36,25 @@ def _repository_commit() -> str:
|
||||||
return result.stdout.strip()
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
def create_kit(output: Path, repository_commit: str | None = None) -> dict:
|
def create_kit(
|
||||||
|
output: Path,
|
||||||
|
repository_commit: str | None = None,
|
||||||
|
artifact_verification: Path | None = None,
|
||||||
|
) -> dict:
|
||||||
|
verification_source = None
|
||||||
|
if artifact_verification is not None:
|
||||||
|
verification_source = artifact_verification.resolve()
|
||||||
|
verification = json.loads(
|
||||||
|
verification_source.read_text(encoding="utf-8-sig")
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
verification.get("status") != "passed"
|
||||||
|
or verification.get("hardware_access") is not False
|
||||||
|
or verification.get("executed_binaries") is not False
|
||||||
|
or verification.get("manual_interaction_required") is not True
|
||||||
|
):
|
||||||
|
raise ValueError("artifact-verification report did not pass the no-device gate")
|
||||||
|
|
||||||
output = output.resolve()
|
output = output.resolve()
|
||||||
if output.exists() and any(output.iterdir()):
|
if output.exists() and any(output.iterdir()):
|
||||||
raise ValueError(f"acceptance-kit directory is not empty: {output}")
|
raise ValueError(f"acceptance-kit directory is not empty: {output}")
|
||||||
|
|
@ -56,12 +74,18 @@ def create_kit(output: Path, repository_commit: str | None = None) -> dict:
|
||||||
"sha256": _sha256(target),
|
"sha256": _sha256(target),
|
||||||
})
|
})
|
||||||
|
|
||||||
|
commit = repository_commit or _repository_commit()
|
||||||
|
template_path = ROOT / REPORT_TEMPLATE
|
||||||
manifest = {
|
manifest = {
|
||||||
"schema_version": 1,
|
"schema_version": 2,
|
||||||
"kit": "M1 packaged no-device desktop acceptance",
|
"kit": "M1 packaged no-device desktop acceptance",
|
||||||
"repository_commit": repository_commit or _repository_commit(),
|
"repository_commit": commit,
|
||||||
"hardware_access_authorized": False,
|
"hardware_access_authorized": False,
|
||||||
"contains_private_evidence": False,
|
"contains_private_evidence": False,
|
||||||
|
"report_template": {
|
||||||
|
"source": REPORT_TEMPLATE,
|
||||||
|
"sha256": _sha256(template_path),
|
||||||
|
},
|
||||||
"files": records,
|
"files": records,
|
||||||
}
|
}
|
||||||
manifest_path = output / "kit-manifest.json"
|
manifest_path = output / "kit-manifest.json"
|
||||||
|
|
@ -70,6 +94,25 @@ def create_kit(output: Path, repository_commit: str | None = None) -> dict:
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
newline="\n",
|
newline="\n",
|
||||||
)
|
)
|
||||||
|
report = json.loads(template_path.read_text(encoding="utf-8"))
|
||||||
|
report["acceptance_kit"] = {
|
||||||
|
"manifest_path": "kit-manifest.json",
|
||||||
|
"manifest_sha256": _sha256(manifest_path),
|
||||||
|
"repository_commit": commit,
|
||||||
|
}
|
||||||
|
if verification_source is not None:
|
||||||
|
verification_target = output / "artifact-verification.json"
|
||||||
|
shutil.copyfile(verification_source, verification_target)
|
||||||
|
report["artifact_verification"] = {
|
||||||
|
"status": "passed",
|
||||||
|
"report_path": verification_target.name,
|
||||||
|
"report_sha256": _sha256(verification_target),
|
||||||
|
}
|
||||||
|
(output / "manual-report.json").write_text(
|
||||||
|
json.dumps(report, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
newline="\n",
|
||||||
|
)
|
||||||
return manifest
|
return manifest
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -80,8 +123,13 @@ def main() -> int:
|
||||||
type=Path,
|
type=Path,
|
||||||
default=ROOT / ".build" / "acceptance-kit",
|
default=ROOT / ".build" / "acceptance-kit",
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--artifact-verification",
|
||||||
|
type=Path,
|
||||||
|
help="passing JSON output from verify_rayforge_windows_checkpoint.py",
|
||||||
|
)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
result = create_kit(args.output)
|
result = create_kit(args.output, artifact_verification=args.artifact_verification)
|
||||||
print(f"kit={args.output.resolve()}")
|
print(f"kit={args.output.resolve()}")
|
||||||
print(f"files={len(result['files'])}")
|
print(f"files={len(result['files'])}")
|
||||||
print(f"repository_commit={result['repository_commit']}")
|
print(f"repository_commit={result['repository_commit']}")
|
||||||
|
|
|
||||||
|
|
@ -32,6 +32,7 @@ STOP_CONDITIONS = (
|
||||||
)
|
)
|
||||||
STEP_STATUSES = {"passed", "failed", "blocked", "not_run"}
|
STEP_STATUSES = {"passed", "failed", "blocked", "not_run"}
|
||||||
SHA256 = re.compile(r"^[0-9a-f]{64}$")
|
SHA256 = re.compile(r"^[0-9a-f]{64}$")
|
||||||
|
GIT_COMMIT = re.compile(r"^[0-9a-f]{40}$")
|
||||||
|
|
||||||
|
|
||||||
def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]:
|
def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]:
|
||||||
|
|
@ -58,11 +59,66 @@ def _validate_evidence(report_root: Path, records: list[dict]) -> list[str]:
|
||||||
return errors
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _bound_json(report_root: Path, binding: dict, label: str) -> tuple[dict | None, list[str]]:
|
||||||
|
value = binding.get("report_path") or binding.get("manifest_path") or ""
|
||||||
|
relative = PurePosixPath(value)
|
||||||
|
if not value or relative.is_absolute() or ".." in relative.parts:
|
||||||
|
return None, [f"unsafe {label} path: {value!r}"]
|
||||||
|
path = report_root.joinpath(*relative.parts)
|
||||||
|
if not path.is_file():
|
||||||
|
return None, [f"{label} is missing: {value}"]
|
||||||
|
digest = binding.get("report_sha256") or binding.get("manifest_sha256") or ""
|
||||||
|
actual = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||||
|
if not SHA256.fullmatch(digest) or digest != actual:
|
||||||
|
return None, [f"{label} SHA-256 mismatch: {value}"]
|
||||||
|
try:
|
||||||
|
return json.loads(path.read_text(encoding="utf-8-sig")), []
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
return None, [f"{label} is not valid JSON: {value}"]
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_kit(report_root: Path, binding: dict) -> list[str]:
|
||||||
|
manifest, errors = _bound_json(report_root, binding, "acceptance-kit manifest")
|
||||||
|
if errors:
|
||||||
|
return errors
|
||||||
|
if manifest.get("schema_version") != 2:
|
||||||
|
errors.append("unsupported acceptance-kit schema")
|
||||||
|
if manifest.get("kit") != "M1 packaged no-device desktop acceptance":
|
||||||
|
errors.append("unexpected acceptance-kit name")
|
||||||
|
commit = binding.get("repository_commit", "")
|
||||||
|
if not GIT_COMMIT.fullmatch(commit) or manifest.get("repository_commit") != commit:
|
||||||
|
errors.append("acceptance-kit repository commit mismatch")
|
||||||
|
if manifest.get("hardware_access_authorized") is not False:
|
||||||
|
errors.append("acceptance kit does not preserve the hardware gate")
|
||||||
|
if manifest.get("contains_private_evidence") is not False:
|
||||||
|
errors.append("acceptance kit contains private evidence")
|
||||||
|
for record in manifest.get("files", []):
|
||||||
|
errors.extend(_validate_evidence(report_root, [record]))
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_artifact_report(report_root: Path, binding: dict) -> list[str]:
|
||||||
|
verification, errors = _bound_json(
|
||||||
|
report_root, binding, "artifact-verification report"
|
||||||
|
)
|
||||||
|
if errors:
|
||||||
|
return errors
|
||||||
|
if verification.get("status") != "passed":
|
||||||
|
errors.append("artifact verification did not pass")
|
||||||
|
if verification.get("hardware_access") is not False:
|
||||||
|
errors.append("artifact verification does not preserve the hardware gate")
|
||||||
|
if verification.get("executed_binaries") is not False:
|
||||||
|
errors.append("artifact verification executed binaries")
|
||||||
|
if verification.get("manual_interaction_required") is not True:
|
||||||
|
errors.append("artifact verification dropped the manual interaction gate")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
def validate(report_path: Path) -> dict:
|
def validate(report_path: Path) -> dict:
|
||||||
report_path = report_path.resolve()
|
report_path = report_path.resolve()
|
||||||
report = json.loads(report_path.read_text(encoding="utf-8-sig"))
|
report = json.loads(report_path.read_text(encoding="utf-8-sig"))
|
||||||
errors = []
|
errors = []
|
||||||
if report.get("schema_version") != 1:
|
if report.get("schema_version") != 2:
|
||||||
errors.append("unsupported report schema")
|
errors.append("unsupported report schema")
|
||||||
if report.get("checkpoint") != "M1 packaged no-device desktop acceptance":
|
if report.get("checkpoint") != "M1 packaged no-device desktop acceptance":
|
||||||
errors.append("unexpected checkpoint name")
|
errors.append("unexpected checkpoint name")
|
||||||
|
|
@ -83,6 +139,10 @@ def validate(report_path: Path) -> dict:
|
||||||
|
|
||||||
environment = report.get("environment", {})
|
environment = report.get("environment", {})
|
||||||
artifact = report.get("artifact_verification", {})
|
artifact = report.get("artifact_verification", {})
|
||||||
|
kit = report.get("acceptance_kit", {})
|
||||||
|
errors.extend(_validate_kit(report_path.parent, kit))
|
||||||
|
artifact_errors = _validate_artifact_report(report_path.parent, artifact)
|
||||||
|
errors.extend(artifact_errors)
|
||||||
errors.extend(_validate_evidence(report_path.parent, report.get("evidence", [])))
|
errors.extend(_validate_evidence(report_path.parent, report.get("evidence", [])))
|
||||||
|
|
||||||
explicit_failure = (
|
explicit_failure = (
|
||||||
|
|
@ -101,7 +161,7 @@ def validate(report_path: Path) -> dict:
|
||||||
and environment.get("fresh_app_data") is True
|
and environment.get("fresh_app_data") is True
|
||||||
and bool(environment.get("windows_version"))
|
and bool(environment.get("windows_version"))
|
||||||
and artifact.get("status") == "passed"
|
and artifact.get("status") == "passed"
|
||||||
and SHA256.fullmatch(artifact.get("report_sha256", "")) is not None
|
and not artifact_errors
|
||||||
)
|
)
|
||||||
status = "failed" if explicit_failure or errors else "passed" if complete else "incomplete"
|
status = "failed" if explicit_failure or errors else "passed" if complete else "incomplete"
|
||||||
return {
|
return {
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,14 @@
|
||||||
{
|
{
|
||||||
"schema_version": 1,
|
"schema_version": 2,
|
||||||
"checkpoint": "M1 packaged no-device desktop acceptance",
|
"checkpoint": "M1 packaged no-device desktop acceptance",
|
||||||
|
"acceptance_kit": {
|
||||||
|
"manifest_path": null,
|
||||||
|
"manifest_sha256": null,
|
||||||
|
"repository_commit": null
|
||||||
|
},
|
||||||
"artifact_verification": {
|
"artifact_verification": {
|
||||||
"status": "not_run",
|
"status": "not_run",
|
||||||
|
"report_path": null,
|
||||||
"report_sha256": null
|
"report_sha256": null
|
||||||
},
|
},
|
||||||
"environment": {
|
"environment": {
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,12 @@ import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from scripts.prepare_m1_acceptance_kit import KIT_FILES, ROOT, create_kit
|
from scripts.prepare_m1_acceptance_kit import (
|
||||||
|
KIT_FILES,
|
||||||
|
REPORT_TEMPLATE,
|
||||||
|
ROOT,
|
||||||
|
create_kit,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class M1AcceptanceKitTests(unittest.TestCase):
|
class M1AcceptanceKitTests(unittest.TestCase):
|
||||||
|
|
@ -14,6 +19,11 @@ class M1AcceptanceKitTests(unittest.TestCase):
|
||||||
manifest = create_kit(output, repository_commit="a" * 40)
|
manifest = create_kit(output, repository_commit="a" * 40)
|
||||||
self.assertFalse(manifest["hardware_access_authorized"])
|
self.assertFalse(manifest["hardware_access_authorized"])
|
||||||
self.assertFalse(manifest["contains_private_evidence"])
|
self.assertFalse(manifest["contains_private_evidence"])
|
||||||
|
self.assertEqual(2, manifest["schema_version"])
|
||||||
|
self.assertEqual(
|
||||||
|
hashlib.sha256((ROOT / REPORT_TEMPLATE).read_bytes()).hexdigest(),
|
||||||
|
manifest["report_template"]["sha256"],
|
||||||
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
{record["path"] for record in manifest["files"]},
|
{record["path"] for record in manifest["files"]},
|
||||||
set(KIT_FILES),
|
set(KIT_FILES),
|
||||||
|
|
@ -23,6 +33,12 @@ class M1AcceptanceKitTests(unittest.TestCase):
|
||||||
self.assertEqual(record["bytes"], path.stat().st_size)
|
self.assertEqual(record["bytes"], path.stat().st_size)
|
||||||
self.assertEqual(record["sha256"], hashlib.sha256(path.read_bytes()).hexdigest())
|
self.assertEqual(record["sha256"], hashlib.sha256(path.read_bytes()).hexdigest())
|
||||||
self.assertEqual(record["classification"], "synthetic-public")
|
self.assertEqual(record["classification"], "synthetic-public")
|
||||||
|
report = json.loads((output / "manual-report.json").read_text(encoding="utf-8"))
|
||||||
|
self.assertEqual("a" * 40, report["acceptance_kit"]["repository_commit"])
|
||||||
|
self.assertEqual(
|
||||||
|
hashlib.sha256((output / "kit-manifest.json").read_bytes()).hexdigest(),
|
||||||
|
report["acceptance_kit"]["manifest_sha256"],
|
||||||
|
)
|
||||||
|
|
||||||
def test_manifest_is_reproducible_for_same_commit(self):
|
def test_manifest_is_reproducible_for_same_commit(self):
|
||||||
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
||||||
|
|
@ -50,6 +66,48 @@ class M1AcceptanceKitTests(unittest.TestCase):
|
||||||
actual = json.loads((output / "kit-manifest.json").read_text(encoding="utf-8"))
|
actual = json.loads((output / "kit-manifest.json").read_text(encoding="utf-8"))
|
||||||
self.assertEqual(actual, expected)
|
self.assertEqual(actual, expected)
|
||||||
|
|
||||||
|
def test_passing_artifact_verification_is_bound_into_report(self):
|
||||||
|
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
verification = root / "verification.json"
|
||||||
|
verification.write_text(json.dumps({
|
||||||
|
"status": "passed",
|
||||||
|
"hardware_access": False,
|
||||||
|
"executed_binaries": False,
|
||||||
|
"manual_interaction_required": True,
|
||||||
|
}), encoding="utf-8")
|
||||||
|
output = root / "kit"
|
||||||
|
create_kit(
|
||||||
|
output,
|
||||||
|
repository_commit="e" * 40,
|
||||||
|
artifact_verification=verification,
|
||||||
|
)
|
||||||
|
copied = output / "artifact-verification.json"
|
||||||
|
report = json.loads((output / "manual-report.json").read_text(encoding="utf-8"))
|
||||||
|
copied_digest = hashlib.sha256(copied.read_bytes()).hexdigest()
|
||||||
|
self.assertEqual("passed", report["artifact_verification"]["status"])
|
||||||
|
self.assertEqual(
|
||||||
|
copied_digest,
|
||||||
|
report["artifact_verification"]["report_sha256"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_failed_artifact_verification_is_rejected(self):
|
||||||
|
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
verification = root / "verification.json"
|
||||||
|
verification.write_text(json.dumps({
|
||||||
|
"status": "failed",
|
||||||
|
"hardware_access": False,
|
||||||
|
"executed_binaries": False,
|
||||||
|
"manual_interaction_required": True,
|
||||||
|
}), encoding="utf-8")
|
||||||
|
with self.assertRaisesRegex(ValueError, "did not pass"):
|
||||||
|
create_kit(
|
||||||
|
root / "kit",
|
||||||
|
repository_commit="f" * 40,
|
||||||
|
artifact_verification=verification,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
|
|
@ -20,12 +20,41 @@ class M1ManualAcceptanceTests(unittest.TestCase):
|
||||||
path.write_text(json.dumps(report), encoding="utf-8")
|
path.write_text(json.dumps(report), encoding="utf-8")
|
||||||
return path
|
return path
|
||||||
|
|
||||||
def _passing(self, root: Path) -> dict:
|
def _bind_reports(self, root: Path, report: dict) -> None:
|
||||||
report = copy.deepcopy(self.template)
|
manifest = {
|
||||||
|
"schema_version": 2,
|
||||||
|
"kit": "M1 packaged no-device desktop acceptance",
|
||||||
|
"repository_commit": "b" * 40,
|
||||||
|
"hardware_access_authorized": False,
|
||||||
|
"contains_private_evidence": False,
|
||||||
|
"files": [],
|
||||||
|
}
|
||||||
|
manifest_path = root / "kit-manifest.json"
|
||||||
|
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||||
|
report["acceptance_kit"] = {
|
||||||
|
"manifest_path": manifest_path.name,
|
||||||
|
"manifest_sha256": hashlib.sha256(manifest_path.read_bytes()).hexdigest(),
|
||||||
|
"repository_commit": "b" * 40,
|
||||||
|
}
|
||||||
|
verification = {
|
||||||
|
"status": "passed",
|
||||||
|
"hardware_access": False,
|
||||||
|
"executed_binaries": False,
|
||||||
|
"manual_interaction_required": True,
|
||||||
|
}
|
||||||
|
verification_path = root / "artifact-verification.json"
|
||||||
|
verification_path.write_text(json.dumps(verification), encoding="utf-8")
|
||||||
report["artifact_verification"] = {
|
report["artifact_verification"] = {
|
||||||
"status": "passed",
|
"status": "passed",
|
||||||
"report_sha256": "a" * 64,
|
"report_path": verification_path.name,
|
||||||
|
"report_sha256": hashlib.sha256(
|
||||||
|
verification_path.read_bytes()
|
||||||
|
).hexdigest(),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def _passing(self, root: Path) -> dict:
|
||||||
|
report = copy.deepcopy(self.template)
|
||||||
|
self._bind_reports(root, report)
|
||||||
report["environment"] = {
|
report["environment"] = {
|
||||||
"windows_version": "synthetic-test-environment",
|
"windows_version": "synthetic-test-environment",
|
||||||
"hardware_disconnected": True,
|
"hardware_disconnected": True,
|
||||||
|
|
@ -37,7 +66,7 @@ class M1ManualAcceptanceTests(unittest.TestCase):
|
||||||
key: False for key in report["stop_conditions"]
|
key: False for key in report["stop_conditions"]
|
||||||
}
|
}
|
||||||
screenshot = root / "evidence" / "offline-state.png"
|
screenshot = root / "evidence" / "offline-state.png"
|
||||||
screenshot.parent.mkdir()
|
screenshot.parent.mkdir(exist_ok=True)
|
||||||
screenshot.write_bytes(b"synthetic screenshot placeholder")
|
screenshot.write_bytes(b"synthetic screenshot placeholder")
|
||||||
report["evidence"] = [{
|
report["evidence"] = [{
|
||||||
"path": "evidence/offline-state.png",
|
"path": "evidence/offline-state.png",
|
||||||
|
|
@ -47,7 +76,11 @@ class M1ManualAcceptanceTests(unittest.TestCase):
|
||||||
|
|
||||||
def test_template_is_valid_but_incomplete(self):
|
def test_template_is_valid_but_incomplete(self):
|
||||||
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
||||||
result = validate(self._write(Path(directory), self.template))
|
root = Path(directory)
|
||||||
|
report = copy.deepcopy(self.template)
|
||||||
|
self._bind_reports(root, report)
|
||||||
|
report["artifact_verification"]["status"] = "not_run"
|
||||||
|
result = validate(self._write(root, report))
|
||||||
self.assertEqual(result["status"], "incomplete")
|
self.assertEqual(result["status"], "incomplete")
|
||||||
self.assertTrue(result["schema_valid"])
|
self.assertTrue(result["schema_valid"])
|
||||||
self.assertEqual(result["required_steps"], len(STEP_IDS))
|
self.assertEqual(result["required_steps"], len(STEP_IDS))
|
||||||
|
|
@ -88,6 +121,21 @@ class M1ManualAcceptanceTests(unittest.TestCase):
|
||||||
result = validate(self._write(root, report))
|
result = validate(self._write(root, report))
|
||||||
self.assertEqual(result["status"], "failed")
|
self.assertEqual(result["status"], "failed")
|
||||||
|
|
||||||
|
def test_unbound_or_changed_provenance_reports_are_rejected(self):
|
||||||
|
with tempfile.TemporaryDirectory(dir=ROOT / ".build") as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
report = self._passing(root)
|
||||||
|
report["artifact_verification"]["report_sha256"] = "0" * 64
|
||||||
|
result = validate(self._write(root, report))
|
||||||
|
self.assertEqual(result["status"], "failed")
|
||||||
|
self.assertIn("artifact-verification report SHA-256 mismatch", result["errors"][0])
|
||||||
|
|
||||||
|
report = self._passing(root)
|
||||||
|
report["acceptance_kit"]["repository_commit"] = "c" * 40
|
||||||
|
result = validate(self._write(root, report))
|
||||||
|
self.assertEqual(result["status"], "failed")
|
||||||
|
self.assertIn("acceptance-kit repository commit mismatch", result["errors"])
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue