feat: validate recipe compatibility metadata

This commit is contained in:
Codex 2026-10-20 20:46:16 +02:00
parent 232f1119d9
commit 7dc1abc8a7
10 changed files with 620 additions and 1 deletions

View file

@ -102,6 +102,7 @@ project workspace or development destination.
- [Open-source reuse landscape](docs/research/open-source-landscape.md)
- [LightBurn-to-Rayforge UX capability map](docs/research/lightburn-rayforge-ux-capability-map.md)
- [Rayforge layer visibility and output spike](docs/research/rayforge-layer-model-spike.md)
- [Rayforge material-recipe model spike](docs/research/rayforge-recipe-model-spike.md)
- [Proposed Rayforge/BalorMK reuse decision](docs/adr/0002-reuse-rayforge-and-balormk.md)
- [Live USB enumeration](docs/research/live-usb-enumeration-2026-10-17.md)
- [LightBurn device-export analysis](docs/research/lightburn-device-exports-2026-10-19.md)

View file

@ -68,7 +68,7 @@ required by real workshop jobs and supported by evidence.
| Edit common settings quickly | The layer list exposes mode, speed, power, passes and applicable galvo fields | Rayforge's generated laser-step settings expose power, speed, frequency and pulse width; serialization is proven offline | Proven offline at the model/UI-contract level | Confirm editing and save/reload in the packaged no-device GUI |
| Reveal complexity progressively | LightBurn Beginner Mode hides some advanced settings; PirateRay calls for Beginner, Advanced and Expert levels without moving the canvas | Rayforge capability sections can hide machine-dependent fields, but three user-facing disclosure levels are not proven | Product requirement | Define the smallest beginner field set from real job needs before adding controls |
| Preserve MOPA units and values | Galvo frequency is presented in kHz and Q-pulse width in ns | Rayforge's workflow uses Hz and ns and preserves both; LightBurn project import does not yet map frequency or pulse width into workflow settings | Partly proven; import gated | Keep explicit conversion at the import boundary and wait for a known-working `.lbrn2` before mapping `QPulseWidth` |
| Apply a reusable material recipe | LightBurn libraries organize presets by material, thickness and description and apply/link them to layers | Rayforge has material recipes, but workshop provenance, approval, machine/lens compatibility and portable personal collections are not proven | Source-inspected plus product requirement | Map the existing recipe schema before designing a new library UI; specify provenance and compatibility gaps |
| Apply a reusable material recipe | LightBurn libraries organize presets by material, thickness and description and apply/link them to layers | Rayforge recipes already match machine, head, material, thickness and step type and preserve Hz/ns settings; the external addon now validates versioned provenance plus exact source/lens/correction compatibility while withholding hardware authorization | Proven offline at model/source level; approval policy and portable collections remain | Integrate the validated metadata into recipe selection before changing the existing recipe UI |
| Inspect planned behavior before any run | LightBurn Preview reflects enabled layers and planned order | Rayforge produced a populated `JobArtifact` with Ops, path length and no machine code | Proven offline; human interaction untested | Verify layer filtering, order, hatch and warnings in the packaged no-device GUI |
| Reject a visibly invalid job | Preview/preflight should reveal unintended output; the target product must reject profile mismatch and out-of-field geometry | Existing JCZ negative tests reject profile mismatch, out-of-field geometry, invalid values and unsupported Ops | Proven offline at encoder boundary | Surface these failures as understandable preflight messages without weakening the fail-closed gate |
| Keep machine and lens selection out of the normal start flow | LightBurn device settings are separate from ordinary layer work | Rayforge has machine profiles and capability-gated settings; the desired administrator-oriented placement is not interaction-tested | Product requirement | Verify default-profile startup and move profile/lens changes behind an advanced path if upstream behavior requires it |
@ -170,6 +170,7 @@ Project evidence:
- [Open-source landscape](open-source-landscape.md)
- [Graphical workflow model spike](rayforge-graphical-workflow-spike.md)
- [Layer visibility and output spike](rayforge-layer-model-spike.md)
- [Material-recipe model spike](rayforge-recipe-model-spike.md)
- [Windows GUI preflight](rayforge-windows-gui-preflight.md)
- [Synthetic LightBurn MOPA import spike](rayforge-lightburn-mopa-import-spike.md)
- [Synthetic LightBurn JCZ device-import spike](rayforge-lightburn-device-import-spike.md)

View file

@ -0,0 +1,112 @@
# Rayforge material-recipe model spike
- Date: 2026-10-20
- Rayforge commit: `c28918c87f8273dd1d244f5a057addc192807fa5`
- Scope: offline recipe schema, matching, persistence and extension seams
- Transport: none
## Result
Rayforge already provides the main recipe mechanism needed by the MOPA
workbench. A recipe is a YAML-backed preset that can target a machine, laser
head, material, thickness range and operation type. Applied settings are
filtered through the selected step type's allowlist, and individual values can
be marked “leave unchanged”. This is a substantially better reuse point than a
second PirateRay recipe engine.
The pinned laser-step contract includes power, speed, frequency in Hz and pulse
width in ns. A synthetic recipe carrying those fields matched only when its
machine, material, thickness and step context were complete. It rejected a
wrong material and missing stock context. An unknown setting survived recipe
storage but was excluded when applying values to the synthetic step.
## Existing fit
The current model covers:
- stable recipe identity, name and description;
- machine and laser-head compatibility;
- material and thickness compatibility;
- operation/step-type compatibility;
- setting-level apply/leave-unchanged behavior;
- post-processing settings;
- specificity-based recipe selection;
- one user-editable YAML file per recipe;
- forward-compatible unknown metadata through `Recipe.extra`.
Rayforge materials and recipes are separate concepts. Materials describe stock
identity and appearance; recipes carry process settings and can refer to a
material UID. That separation fits the PirateRay requirement that recipes are
optional accelerators rather than mandatory material wizards.
## PirateRay gaps
The source does not expose native, validated fields for:
- evidence provenance;
- approval or validation status;
- source/profile revision;
- lens compatibility;
- correction-profile compatibility;
- named personal, makerspace-shared or project collections.
Machine and head matching use identifiers, but those identifiers alone do not
prove that a recipe is safe for a JPT source, mounted lens or correction file.
The existing `extra` round-trip can preserve namespaced PirateRay metadata,
but preservation is not validation and the stock Rayforge UI does not explain
or enforce those fields.
## Narrow implementation direction
Reuse `Recipe`, `RecipeManager`, the existing selector/editor and step
allowlists. The smallest extension should define a versioned, namespaced
PirateRay metadata block with explicit provenance and compatibility fields,
then validate it before a recipe can receive a stronger label than
“unverified”. Keep unknown or incomplete compatibility fail-closed for
encoding and physical execution.
Do not fork the recipe schema merely to add collections. First test whether
separate manager directories or a small library wrapper can provide personal
and makerspace-shared views while retaining Rayforge's file-per-recipe format.
Project-local portability remains a separate, unproven requirement.
The exact evidence required for “makerspace-approved” remains a product and
safety decision for Kaz. This spike deliberately does not invent that policy.
## Implemented offline metadata gate
The external no-device addon now contains a version-1 `pirateray` metadata
validator. It requires non-empty provenance and exact matches for machine
profile, source type, lens profile and correction-profile hash. Missing or
mismatched values are rejected independently.
The only currently accepted status is `unverified`. The tempting
`makerspace-approved` label is rejected until its evidence and review policy
has been decided. Even successfully validated metadata returns
`encoding_allowed: false` and `hardware_use_allowed: false`; this contract
organizes evidence but grants no controller capability.
The synthetic public fixture and regression tests exercise the contract
without using physical machine identifiers or settings.
## Reproduce
Run against the pinned checkout with the isolated Rayforge Python environment:
```powershell
python scripts/spike_rayforge_recipe_model.py <rayforge-checkout> `
--output tests/fixtures/rayforge-recipe-model.json
```
## Source anchors
- `rayforge/core/recipe.py`: schema, matching, allowlisting and `extra`
- `rayforge/core/recipe_manager.py`: YAML persistence and selection
- `rayforge/core/material.py`: material identity and appearance
- `rayforge/core/material_library.py`: material-library storage
- `rayforge/builtin_addons/rayforge-addon-laser/laser_essentials/steps/laser_step.py`:
laser recipe fields and Hz/ns contracts
- `rayforge/ui_gtk/doceditor/recipes/`: existing editor and selector
No USB enumeration, controller command, motion, physical framing, arming or
laser output occurred.

View file

@ -0,0 +1,186 @@
"""Characterize Rayforge recipe matching and metadata extension seams."""
from __future__ import annotations
import argparse
import hashlib
import json
import subprocess
import sys
from pathlib import Path
from types import SimpleNamespace
RAYFORGE_COMMIT = "c28918c87f8273dd1d244f5a057addc192807fa5"
def checkout_commit(root: Path) -> str:
result = subprocess.run(
["git", "rev-parse", "HEAD"],
cwd=root,
check=True,
capture_output=True,
text=True,
)
return result.stdout.strip()
class SyntheticLaserStep:
power = 0.5
frequency = 30000
pulse_width = 200
@classmethod
def recipe_keys(cls) -> tuple[str, ...]:
return ("power", "frequency", "pulse_width")
def characterize(root: Path) -> dict:
commit = checkout_commit(root)
if commit != RAYFORGE_COMMIT:
raise RuntimeError(
f"Expected Rayforge {RAYFORGE_COMMIT}, found {commit}"
)
sys.path.insert(0, str(root))
from rayforge.core.recipe import Recipe
recipe = Recipe.from_dict(
{
"uid": "synthetic-jpt-marking",
"name": "Synthetic JPT marking",
"description": "Public model fixture; not machine settings",
"target_step_types": ["SyntheticLaserStep"],
"target_machine_id": "synthetic-jpt-profile",
"material_uid": "synthetic-stainless-steel",
"min_thickness_mm": 0.5,
"max_thickness_mm": 2.0,
"setting_dicts": [
{"name": "power", "value": 0.5, "recipe_apply": True},
{
"name": "frequency",
"value": 30000,
"recipe_apply": True,
},
{
"name": "pulse_width",
"value": 200,
"recipe_apply": True,
},
{
"name": "unsupported_private_field",
"value": "ignored",
"recipe_apply": True,
},
{
"name": "air_assist",
"value": True,
"recipe_apply": False,
},
],
"pirateray_provenance": {
"status": "synthetic-unverified",
"source": "public-model-fixture",
},
}
)
matching_stock = SimpleNamespace(
material_uid="synthetic-stainless-steel", thickness=1.0
)
wrong_stock = SimpleNamespace(
material_uid="synthetic-aluminum", thickness=1.0
)
machine = SimpleNamespace(id="synthetic-jpt-profile", heads=[])
roundtrip = Recipe.from_dict(recipe.to_dict())
allowed_settings = recipe.get_settings_for_step(SyntheticLaserStep())
recipe_source = (root / "rayforge" / "core" / "recipe.py").read_text(
encoding="utf-8"
)
manager_source = (
root / "rayforge" / "core" / "recipe_manager.py"
).read_text(encoding="utf-8")
laser_source = (
root
/ "rayforge"
/ "builtin_addons"
/ "rayforge-addon-laser"
/ "laser_essentials"
/ "steps"
/ "laser_step.py"
).read_text(encoding="utf-8")
return {
"rayforge_commit": commit,
"hardware_access": False,
"model": {
"applied_settings": recipe.get_applied_settings(),
"step_allowlisted_settings": allowed_settings,
"unknown_setting_rejected_by_step": (
"unsupported_private_field" not in allowed_settings
),
"disabled_setting_left_unchanged": (
"air_assist" not in recipe.get_applied_settings()
),
"matches_complete_context": recipe.matches(
[matching_stock], machine, "SyntheticLaserStep"
),
"rejects_wrong_material": not recipe.matches(
[wrong_stock], machine, "SyntheticLaserStep"
),
"rejects_missing_stock_context": not recipe.matches(
[], machine, "SyntheticLaserStep"
),
"extra_metadata_roundtrip": roundtrip.extra,
},
"source_contract": {
"machine_match": "target_machine_id" in recipe_source,
"head_match": "selected_head_uid" in recipe_source,
"material_match": "material_uid" in recipe_source,
"thickness_match": all(
key in recipe_source
for key in ("min_thickness_mm", "max_thickness_mm")
),
"step_type_match": "target_step_types" in recipe_source,
"file_per_recipe_yaml": (
'f"{recipe_id}.yaml"' in manager_source
),
"forward_compatible_extra_metadata": (
"result.update(self.extra)" in recipe_source
),
"frequency_hz": "Laser pulse frequency in Hz" in laser_source,
"pulse_width_ns": "Laser pulse width in ns" in laser_source,
"native_approval_status": "approval_status" in recipe_source,
"native_provenance": "provenance" in recipe_source,
"native_lens_match": "lens" in recipe_source,
"native_correction_match": "correction" in recipe_source,
"native_named_collections": "collection" in manager_source,
},
"conclusion": (
"Reuse Rayforge recipes and their matching/allowlist behavior. "
"Add PirateRay provenance, approval, lens and correction metadata "
"through a narrow validated extension before adding new UI."
),
}
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("rayforge_checkout", type=Path)
parser.add_argument("--output", type=Path)
args = parser.parse_args()
result = characterize(args.rayforge_checkout.resolve())
payload = json.dumps(result, indent=2, sort_keys=True) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(payload, encoding="utf-8")
else:
print(payload, end="")
print(f"sha256={hashlib.sha256(payload.encode()).hexdigest()}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,97 @@
"""Fail-closed PirateRay metadata for Rayforge recipes."""
from __future__ import annotations
from typing import Any, Mapping
METADATA_KEY = "pirateray"
SCHEMA_VERSION = 1
SUPPORTED_STATUS = "unverified"
COMPATIBILITY_KEYS = (
"machine_profile_id",
"source_type",
"lens_profile_id",
"correction_profile_sha256",
)
class RecipeMetadataError(ValueError):
"""Raised when recipe evidence or compatibility is incomplete."""
def _require_mapping(value: Any, label: str) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
raise RecipeMetadataError(f"{label} must be an object")
return value
def _require_text(value: Any, label: str) -> str:
if not isinstance(value, str) or not value.strip():
raise RecipeMetadataError(f"{label} must be non-empty text")
return value
def validate_recipe_metadata(
metadata: Mapping[str, Any],
active_context: Mapping[str, Any],
) -> dict[str, Any]:
"""Validate evidence and exact compatibility without enabling hardware.
Only ``unverified`` recipes are representable until an approval policy is
reviewed. Compatibility identifiers are opaque evidence-backed values;
this function does not infer or normalize machine configuration.
"""
metadata = _require_mapping(metadata, METADATA_KEY)
if metadata.get("schema_version") != SCHEMA_VERSION:
raise RecipeMetadataError("unsupported PirateRay recipe schema")
if metadata.get("status") != SUPPORTED_STATUS:
raise RecipeMetadataError(
"recipe status is unsupported until approval policy is reviewed"
)
provenance = _require_mapping(
metadata.get("provenance"), "provenance"
)
provenance_kind = _require_text(
provenance.get("kind"), "provenance.kind"
)
provenance_reference = _require_text(
provenance.get("reference"), "provenance.reference"
)
compatibility = _require_mapping(
metadata.get("compatibility"), "compatibility"
)
active_context = _require_mapping(active_context, "active_context")
normalized_compatibility = {}
for key in COMPATIBILITY_KEYS:
expected = _require_text(
compatibility.get(key), f"compatibility.{key}"
)
actual = _require_text(active_context.get(key), f"active_context.{key}")
if actual != expected:
raise RecipeMetadataError(f"recipe {key} does not match")
normalized_compatibility[key] = expected
return {
"schema_version": SCHEMA_VERSION,
"status": SUPPORTED_STATUS,
"provenance": {
"kind": provenance_kind,
"reference": provenance_reference,
},
"compatibility": normalized_compatibility,
"encoding_allowed": False,
"hardware_use_allowed": False,
}
def validate_rayforge_recipe(recipe: Any, active_context: Mapping[str, Any]):
"""Read the namespaced block from ``Recipe.extra`` and validate it."""
extra = getattr(recipe, "extra", None)
if not isinstance(extra, Mapping) or METADATA_KEY not in extra:
raise RecipeMetadataError("PirateRay recipe metadata is missing")
return validate_recipe_metadata(extra[METADATA_KEY], active_context)

View file

@ -19,6 +19,7 @@
"pirate_ray_jcz/__init__.py",
"pirate_ray_jcz/offline_session.py",
"pirate_ray_jcz/profile_import.py",
"pirate_ray_jcz/recipe_metadata.py",
"pirate_ray_jcz/worker.py",
"rayforge-addon.yaml"
],

View file

@ -0,0 +1,45 @@
{
"conclusion": "Reuse Rayforge recipes and their matching/allowlist behavior. Add PirateRay provenance, approval, lens and correction metadata through a narrow validated extension before adding new UI.",
"hardware_access": false,
"model": {
"applied_settings": {
"frequency": 30000,
"power": 0.5,
"pulse_width": 200,
"unsupported_private_field": "ignored"
},
"disabled_setting_left_unchanged": true,
"extra_metadata_roundtrip": {
"pirateray_provenance": {
"source": "public-model-fixture",
"status": "synthetic-unverified"
}
},
"matches_complete_context": true,
"rejects_missing_stock_context": true,
"rejects_wrong_material": true,
"step_allowlisted_settings": {
"frequency": 30000,
"power": 0.5,
"pulse_width": 200
},
"unknown_setting_rejected_by_step": true
},
"rayforge_commit": "c28918c87f8273dd1d244f5a057addc192807fa5",
"source_contract": {
"file_per_recipe_yaml": true,
"forward_compatible_extra_metadata": true,
"frequency_hz": true,
"head_match": true,
"machine_match": true,
"material_match": true,
"native_approval_status": false,
"native_correction_match": false,
"native_lens_match": false,
"native_named_collections": false,
"native_provenance": false,
"pulse_width_ns": true,
"step_type_match": true,
"thickness_match": true
}
}

View file

@ -0,0 +1,14 @@
{
"schema_version": 1,
"status": "unverified",
"provenance": {
"kind": "synthetic-public",
"reference": "tests/fixtures/synthetic-pirateray-recipe-metadata.json"
},
"compatibility": {
"machine_profile_id": "synthetic-jpt-profile",
"source_type": "synthetic-jpt-mopa",
"lens_profile_id": "synthetic-110mm-lens",
"correction_profile_sha256": "sha256-synthetic-not-a-machine-hash"
}
}

View file

@ -0,0 +1,92 @@
import json
import unittest
from copy import deepcopy
from pathlib import Path
import sys
from types import SimpleNamespace
ROOT = Path(__file__).parents[1]
ADDON = ROOT / "spikes" / "rayforge-jcz-no-device-addon"
FIXTURE = (
ROOT / "tests" / "fixtures" / "synthetic-pirateray-recipe-metadata.json"
)
sys.path.insert(0, str(ADDON))
from pirate_ray_jcz.recipe_metadata import ( # noqa: E402
COMPATIBILITY_KEYS,
RecipeMetadataError,
validate_rayforge_recipe,
validate_recipe_metadata,
)
class RayforgeRecipeMetadataTests(unittest.TestCase):
def setUp(self):
self.metadata = json.loads(FIXTURE.read_text(encoding="utf-8"))
self.context = dict(self.metadata["compatibility"])
def test_complete_synthetic_metadata_is_preserved_but_not_authorized(self):
result = validate_recipe_metadata(self.metadata, self.context)
self.assertEqual(1, result["schema_version"])
self.assertEqual("unverified", result["status"])
self.assertFalse(result["encoding_allowed"])
self.assertFalse(result["hardware_use_allowed"])
self.assertEqual(self.context, result["compatibility"])
def test_every_compatibility_axis_fails_closed_on_mismatch(self):
for key in COMPATIBILITY_KEYS:
with self.subTest(key=key):
context = dict(self.context)
context[key] = f"wrong-{key}"
with self.assertRaisesRegex(
RecipeMetadataError, f"recipe {key} does not match"
):
validate_recipe_metadata(self.metadata, context)
def test_missing_evidence_and_context_fail_closed(self):
cases = []
missing_provenance = deepcopy(self.metadata)
del missing_provenance["provenance"]
cases.append((missing_provenance, self.context))
missing_compatibility = deepcopy(self.metadata)
del missing_compatibility["compatibility"]["lens_profile_id"]
cases.append((missing_compatibility, self.context))
missing_context = dict(self.context)
del missing_context["correction_profile_sha256"]
cases.append((self.metadata, missing_context))
for metadata, context in cases:
with self.subTest(metadata=metadata, context=context):
with self.assertRaises(RecipeMetadataError):
validate_recipe_metadata(metadata, context)
def test_unreviewed_approval_label_is_rejected(self):
metadata = deepcopy(self.metadata)
metadata["status"] = "makerspace-approved"
with self.assertRaisesRegex(RecipeMetadataError, "approval policy"):
validate_recipe_metadata(metadata, self.context)
def test_rayforge_extra_is_the_only_accepted_seam(self):
recipe = SimpleNamespace(extra={"pirateray": self.metadata})
result = validate_rayforge_recipe(recipe, self.context)
self.assertEqual("synthetic-public", result["provenance"]["kind"])
with self.assertRaises(RecipeMetadataError):
validate_rayforge_recipe(SimpleNamespace(extra={}), self.context)
def test_module_has_no_hardware_transport(self):
source = (
ADDON / "pirate_ray_jcz" / "recipe_metadata.py"
).read_text(encoding="utf-8").lower()
for forbidden in (
"import usb",
"import serial",
"import socket",
"pyusb",
):
self.assertNotIn(forbidden, source)
if __name__ == "__main__":
unittest.main()

View file

@ -0,0 +1,70 @@
import json
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
FIXTURE = ROOT / "tests" / "fixtures" / "rayforge-recipe-model.json"
SCRIPT = ROOT / "scripts" / "spike_rayforge_recipe_model.py"
class RayforgeRecipeModelSpikeTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.result = json.loads(FIXTURE.read_text(encoding="utf-8"))
def test_recipe_matching_fails_closed_without_required_context(self):
model = self.result["model"]
self.assertTrue(model["matches_complete_context"])
self.assertTrue(model["rejects_wrong_material"])
self.assertTrue(model["rejects_missing_stock_context"])
def test_recipe_payload_is_allowlisted_per_step(self):
model = self.result["model"]
self.assertTrue(model["unknown_setting_rejected_by_step"])
self.assertTrue(model["disabled_setting_left_unchanged"])
self.assertEqual(
sorted(model["step_allowlisted_settings"]),
["frequency", "power", "pulse_width"],
)
def test_units_and_compatibility_axes_are_present(self):
contract = self.result["source_contract"]
for key in (
"machine_match",
"head_match",
"material_match",
"thickness_match",
"step_type_match",
"frequency_hz",
"pulse_width_ns",
"file_per_recipe_yaml",
"forward_compatible_extra_metadata",
):
self.assertTrue(contract[key], key)
def test_pirateray_metadata_gaps_are_not_claimed_as_native(self):
contract = self.result["source_contract"]
for key in (
"native_approval_status",
"native_provenance",
"native_lens_match",
"native_correction_match",
"native_named_collections",
):
self.assertFalse(contract[key], key)
self.assertEqual(
self.result["model"]["extra_metadata_roundtrip"]
["pirateray_provenance"]["status"],
"synthetic-unverified",
)
def test_spike_is_hardware_free(self):
self.assertFalse(self.result["hardware_access"])
source = SCRIPT.read_text(encoding="utf-8").lower()
for forbidden in ("pyusb", "usb.core", "9588", "9899"):
self.assertNotIn(forbidden, source)
if __name__ == "__main__":
unittest.main()