diff --git a/AGENTS.md b/AGENTS.md index 9d7d7f6..bf9aa38 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -219,6 +219,10 @@ Treat this defensively. The observed enrichment representation can be a space-se If absent or malformed, record a quality flag. Never pretend equal/fallback weights are original SMK weights. +For bulk-only feature schema `artart-visual-features` v1, use canonical `colors`, deduplicate exact +repeated hex values for analysis, and label `1/n` weights explicitly as `uniform`. Preserve source +palette/order and never claim source dominance from these weights. + ## Colour space Use: @@ -240,15 +244,12 @@ Hue is circular. Do not average hue degrees arithmetically. Aggregate colour cen - `centroid_b` - `centroid_chroma` - `centroid_hue_deg` -- `centroid_hue_x = cos(h)` -- `centroid_hue_y = sin(h)` ### Palette distribution -- `dominant_color_weight` - `palette_entropy_01` - `effective_palette_size` -- `palette_spread` +- `palette_spread_oklab` - `lightness_stddev` - `chroma_stddev` - `hue_concentration_01` diff --git a/README.md b/README.md index 4092b66..a9c0c5e 100644 --- a/README.md +++ b/README.md @@ -121,9 +121,10 @@ See: ## Current status -Milestones 0–2 complete: source structure and audits are manifested, the language snapshots have +Milestones 0–3 complete: source structure and audits are manifested, the language snapshots have bounded-memory source Parquets, DA/EN rows are paired into stable SMK entities, and the exact -`smk-public-visual-v1` membership rule is frozen. Visual feature extraction has not started. +`smk-public-visual-v1` membership rule is frozen. Deterministic raw visual features and analytical +palettes now exist; corpus-wide feature normalization and similarity have not started. Audit results are documented in: @@ -134,10 +135,13 @@ Audit results are documented in: - [`docs/RIGHTS_AUDIT.md`](docs/RIGHTS_AUDIT.md) - [`docs/CANONICALIZATION.md`](docs/CANONICALIZATION.md) - [`docs/CORPUS_V1.md`](docs/CORPUS_V1.md) +- [`docs/FEATURE_EXTRACTION_V1.md`](docs/FEATURE_EXTRACTION_V1.md) +- [`docs/FEATURE_EXAMPLES.md`](docs/FEATURE_EXAMPLES.md) Run the audited ingest with: ```bash smk milestone1 smk milestone2 +smk milestone3 ``` diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index dd73c25..96a5dfd 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -54,6 +54,10 @@ Suggested fields: ### `features.parquet` One row per artwork with Feature Specification v1 values and quality flags. +Milestone 3 materializes this as `artwork-visual-features-v1.parquet`, keyed by `smk_entity_id`, +and writes `artwork-palettes-v1.parquet` as one typed row per analytical palette colour. Both are +bound to exact canonical/corpus SHA-256 values and extractor/schema versions. + ### `normalized-features.parquet` One row per artwork per normalization profile/version, or a wide equivalent if simpler to query. diff --git a/docs/FEATURE_EXAMPLES.md b/docs/FEATURE_EXAMPLES.md new file mode 100644 index 0000000..23db1a8 --- /dev/null +++ b/docs/FEATURE_EXAMPLES.md @@ -0,0 +1,58 @@ +# Feature examples + +Values below are deterministic outputs from extractor v1. OKLCH tuples are `(L, C, hue°)`; `null` +hue means chroma below `1e-7`. + +## KMS8892 — required corpus example with physical dimensions + +- Source/analytical palette: 7 / 7 colours, uniform weight 1/7. +- OKLCH: `#908d41` (0.6297, 0.0986, 106.85), `#fff1dc` (0.9637, 0.0315, 78.17), + `#aed7ea` (0.8561, 0.0505, 227.77), `#ceeae5` (0.9152, 0.0302, 183.74), + `#b7ac3e` (0.7331, 0.1295, 103.59), `#888888` (0.6268, ~0, null), and + `#795025` (0.4679, 0.0801, 64.80). +- Centroid OKLab: (0.7418, -0.0118, 0.0406); OKLCH: (0.7418, 0.0423, 106.18°). +- Spread 0.1654; hue concentration 0.7040. +- SMK visual: brightness 7.1344, contrast 2.9756, saturation 2.0800, entropy 9.6865, + colortemp 3.3317. +- Netto 78 × 51.5 cm; area 4,017 cm²; physical/image aspect mismatch 0.000048. +- Flags: `IMAGE_BUNDLE_OK`. + +## KKS8001 — required duplicate-colour QA example, outside corpus + +- Source palette: `#F4C6A3`, `#F4C6A3`, `#9B9B9B`, `#9B9B9B`, `#9B9B9B`. +- Analytical palette: `#f4c6a3` (0.8583, 0.0702, 58.91°) and `#9b9b9b` + (0.6895, ~0, null), each with uniform analytical weight 0.5. +- Centroid OKLab: (0.7739, 0.0181, 0.0301); OKLCH: (0.7739, 0.0351, 58.91°). +- Spread 0.0914; hue concentration 1.0. +- Source visual and geometry are unavailable. +- Flags include `PALETTE_DUPLICATE_SOURCE_COLOR`, `IMAGE_BUNDLE_INCOMPLETE`, + `VISUAL_FIELDS_PARTIAL`, and `PHYSICAL_GEOMETRY_MISSING`. +- It is not added to either corpus feature Parquet. + +## KMS1571 — near-achromatic + +- Palette: `#111111`, `#222222`; both have null hue. +- Centroid OKLCH: (0.2148, ~0, null). +- Spread 0.0372; hue concentration null. +- Netto 66 × 40.5 cm. +- Flags: `PALETTE_ACHROMATIC`, `CENTROID_ACHROMATIC`, + `HUE_CONCENTRATION_UNAVAILABLE`. + +## KAS782 — colourful/high-spread deterministic sample + +- Palette: `#b97438`, `#b7ac3e`, `#000000`, `#33140c`, `#fcaa6b`, uniform weight 0.2. +- Chromatic hues cluster around orange/yellow (35.54°–103.59°); black has null hue. +- Centroid OKLCH: (0.4793, 0.0772, 68.93°). +- Spread 0.2953; hue concentration 0.9149. +- SMK saturation 4.7511 and brightness 3.7184. +- Physical geometry is unavailable; the image is 1080 × 1080 px. + +## KKS5261 — corpus example without Netto geometry + +- Palette: `#b7ac3e`, `#956830`, `#77733d`, `#ffffff`. +- Centroid OKLCH: (0.7081, 0.0714, 93.52°). +- Spread 0.1721; hue concentration 0.9618. +- Image 4,992 × 6,287 px; physical Netto geometry remains null rather than substituting sheet + measurements or pixels. +- Flags include `PHYSICAL_GEOMETRY_MISSING`. + diff --git a/docs/FEATURE_EXTRACTION_V1.md b/docs/FEATURE_EXTRACTION_V1.md new file mode 100644 index 0000000..1fdc486 --- /dev/null +++ b/docs/FEATURE_EXTRACTION_V1.md @@ -0,0 +1,104 @@ +# ArtArt visual features v1 + +Milestone 3 produces deterministic, corpus-independent raw features for every member of the frozen +`smk-public-visual-v1` corpus. + +## Version and provenance + +- feature schema: `artart-visual-features` +- schema version: `1.0.0` +- extractor version: `1.0.0` +- colour implementation: `internal-oklab-published-matrices-v1` +- source corpus: `smk-public-visual-v1` + +Both Parquet outputs embed the canonical/corpus SHA-256 values and a reproducible generation +timestamp anchored to the source manifest. Calculations and storage use float64. + +## Palette policy + +The palette source is canonical SMK bulk `colors`. No documented weights exist, so every analytical +colour has uniform weight `1/n`. Source ordering is preserved but is not interpreted as dominance. +Exact repeated hex values are deduplicated case-insensitively for analysis while source palette, +source rank and source size remain available. Repetition is never treated as weight. + +`KKS8001` demonstrates the rule: five source entries become two analytical colours and receive +`PALETTE_DUPLICATE_SOURCE_COLOR`. It is outside the corpus because its image bundle is incomplete, +so the flag appears in the QA example rather than the corpus feature table. + +There is deliberately no `dominant_color_weight` in v1. Normalized palette entropy is 0 for a +one-colour palette and 1 for every multi-colour palette under uniform weights. Effective palette +size consequently equals analytical palette size. Both definitions remain generic so documented +weights can replace uniform weights later, but neither should be overinterpreted in v1. + +## Colour conversion and features + +Each `#RRGGBB` value follows: + +```text +sRGB -> linear RGB -> OKLab -> OKLCH +``` + +The implementation uses the published OKLab matrices. Hue is null when OKLCH chroma is below +`1e-7`; neutral colours never receive an invented hue of zero. + +Centroids are weighted means in OKLab and only then converted to OKLCH. Additional compact features +are normalized entropy, effective palette size, weighted OKLab mean distance from centroid, +weighted lightness/chroma standard deviation and chroma-weighted circular hue concentration. + +## Geometry + +Net width/height, area, aspect ratio and natural logs exist only for finite positive, unambiguous +canonical Netto dimensions. Gross, sheet, plate, mount and pixels never substitute for missing +physical geometry. Pixel width/height and image aspect are separate digital fields. + +Aspect mismatch is: + +```text +abs(log(net_aspect_ratio / image_aspect_ratio)) +``` + +Its observed median is 0.0126, P95 0.261 and P99 0.654. The conservative diagnostic threshold is +`ln(2) = 0.693147...`, meaning more than a factor-two aspect discrepancy; 19 artworks are flagged. +The flag does not change corpus membership. + +## Corpus diagnostics + +| Measure | Count | +|---|---:| +| Feature rows | 27,254 | +| Analytical palette rows | 113,332 | +| Achromatic palette-colour rows | 63,206 | +| Centroid hue null | 5,766 | +| Hue concentration null | 5,766 | +| Valid Netto width × height | 2,189 | +| Physical geometry missing | 25,065 | +| Aspect mismatch flagged | 19 | + +Selected distributions: + +| Feature | Min | Median | P95 | P99 | Max | +|---|---:|---:|---:|---:|---:| +| centroid L | 0.1771 | 0.7207 | 0.9571 | 0.9892 | 1.0000 | +| centroid chroma | ~0 | 0.0181 | 0.0746 | 0.0976 | 0.1468 | +| OKLab palette spread | 0 | 0.1685 | 0.2943 | 0.3689 | 0.5000 | +| hue concentration | 0.0279 | 0.9833 | 1.0000 | 1.0000 | 1.0000 | +| net area cm² | 5.46 | 3,897.25 | 32,823 | 70,917 | 626,990 | + +Source visual nulls are brightness 62, contrast 62, saturation 871, entropy 1 and colortemp 1. +Finite colortemp remains unmodified (range -1.2760 to 15.7201 in the corpus); 143 canonical extreme +flags are carried through. Nonfinite source values were already canonical null and none enter this +corpus. + +## Proposed future machine vector + +The unfrozen raw candidate is: + +- colour: centroid L/a/b, OKLab spread, lightness stddev, chroma stddev, hue concentration; +- SMK visual: brightness, contrast, saturation, entropy; +- optional geometry: log net width, log net height, log net area, net aspect ratio. + +Do not include centroid hue alongside a/b, colortemp, raw pixel dimensions or palette entropy as an +important independent v1 signal. A future similarity layer should compare only shared available +components and renormalize configured feature weights; it must not impute stored nulls. No vector +normalization or similarity weighting is implemented here. + diff --git a/docs/FEATURE_SPEC_V1.md b/docs/FEATURE_SPEC_V1.md index 6ab670e..408ec82 100644 --- a/docs/FEATURE_SPEC_V1.md +++ b/docs/FEATURE_SPEC_V1.md @@ -6,7 +6,10 @@ Per-artwork feature extraction is deterministic and corpus-independent. Running ## Palette input -Prefer an explicit SMK weighted palette when valid. Preserve the source representation. +Milestone 3 uses canonical SMK bulk `colors` and preserves source representation and order. The +bulk snapshots have no documented weights, so v1 uses explicitly labelled uniform analytical +weights (`1/n`) after deduplicating exact repeated hex values. Repetition is not interpreted as +weight. For `colors_palette_weighted_s`, the observed format may contain repeated space-separated hex tokens. Parse defensively: @@ -16,7 +19,8 @@ For `colors_palette_weighted_s`, the observed format may contain repeated space- 4. retain an explicit `weight_source`, 5. quality-flag malformed/partial input. -If weights are unavailable, do not label equal/fallback weights as original SMK weights. A fallback can be supported if explicitly identified. +Uniform v1 weights are `palette_weighting = "uniform"`, never original SMK weights. Future +enrichment weights must retain separate provenance. Canonical weights must satisfy approximately: @@ -52,15 +56,13 @@ Store: - `centroid_b` - `centroid_chroma` - `centroid_hue_deg` -- `centroid_hue_x` -- `centroid_hue_y` Convert the centroid to OKLCH for chroma/hue. Hue is undefined for sufficiently achromatic centroids; use null rather than inventing a direction. ## Palette statistics -### `dominant_color_weight` -Largest normalized palette weight. +There is no `dominant_color_weight` in v1 because uniform weights do not measure image dominance. +The first source colour and preserved source order are available without claiming dominance. ### `palette_entropy_01` Normalized Shannon entropy: @@ -80,7 +82,7 @@ exp(H) using unnormalized natural-log entropy. -### `palette_spread` +### `palette_spread_oklab` Weighted mean Euclidean OKLab distance from palette colours to the weighted centroid. ### `lightness_stddev` @@ -114,17 +116,16 @@ If multiple enrichment components/versions provide competing values, retain prov ## Geometry -Canonical geometry/features: +Canonical feature geometry: -- `physical_width_cm` -- `physical_height_cm` -- `physical_depth_cm` -- `physical_area_cm2` -- `physical_aspect_ratio` +- `net_width_cm` +- `net_height_cm` +- `net_area_cm2` +- `net_aspect_ratio` - `image_aspect_ratio` -- `log_width_cm` -- `log_height_cm` -- `log_area_cm2` +- `log_net_width_cm` +- `log_net_height_cm` +- `log_net_area_cm2` Rules: diff --git a/docs/IMPLEMENTATION_PLAN.md b/docs/IMPLEMENTATION_PLAN.md index 124806f..b1b380c 100644 --- a/docs/IMPLEMENTATION_PLAN.md +++ b/docs/IMPLEMENTATION_PLAN.md @@ -43,32 +43,24 @@ without collapsing duplicate `object_number` groups. Status: complete. See `CANONICALIZATION.md` and `CORPUS_V1.md`. Visual/colour feature extraction has not started. -## Milestone 3 — Weighted palette + colour conversion +## Milestone 3 — Deterministic per-artwork visual features -- locate ordinary/weighted palette fields in the actual snapshot/enrichment data -- parse `colors_palette_weighted_s` when present -- normalize explicit weights -- implement/verify sRGB -> OKLab -> OKLCH -- preserve malformed input as quality warnings rather than crashing the run - -Done when known palette fixtures produce stable colour rows and valid weights. - -## Milestone 4 — Feature extraction - -Implement: +Implemented for frozen `smk-public-visual-v1`: - perceptual centroid -- palette entropy -- effective palette size +- explicitly uniform bulk-palette weights and duplicate-colour handling +- sRGB -> OKLab -> OKLCH with achromatic hue semantics +- palette entropy/effective size with documented uniform-weight limitation - palette spread - weighted lightness/chroma variation - hue concentration - SMK visual fields + provenance - physical/image geometry + logs -Done when one artwork can produce a complete feature record independently of the corpus. +Status: complete. Raw features remain corpus-independent; no corpus normalization or similarity is +implemented. See `FEATURE_EXTRACTION_V1.md` and `FEATURE_EXAMPLES.md`. -## Milestone 5 — Quality layer +## Milestone 4 — Quality layer Implement documented flags and source/language/aspect checks. @@ -85,13 +77,13 @@ Produce `quality-report.json` with: - palette-size distribution - obvious extreme values -## Milestone 6 — Pilot build +## Milestone 5 — Pilot build Run 100–500 representative artworks. Review distributions and manually inspect selected records. Fix parser/source/math errors only; do not artistically tune features. -## Milestone 7 — Corpus-relative normalization +## Milestone 6 — Corpus-relative normalization Use the frozen `smk-public-visual-v1` entity membership from Milestone 2. Do not redefine corpus membership from an expected count. @@ -104,7 +96,7 @@ membership from an expected count. - implement robust z-score - freeze profile with source-manifest reference -## Milestone 8 — Full dataset build +## Milestone 7 — Full dataset build Run the complete local source snapshots. @@ -112,7 +104,7 @@ Definition of done: Every canonical artwork has valid applicable features or explicit nulls/flags explaining missing/invalid prerequisites. Every normalization result references a frozen profile and exact source manifest. -## Milestone 9 — Similarity +## Milestone 8 — Similarity Implement separate profiles rather than one magic similarity score: @@ -122,7 +114,7 @@ Implement separate profiles rather than one magic similarity score: Use fast feature-vector candidate retrieval followed optionally by weighted-palette optimal-transport reranking. Do not precompute all pairs. -## Milestone 10 — Art Tone v1 +## Milestone 9 — Art Tone v1 Implement versioned, explainable sonification mapping separate from objective features. @@ -138,7 +130,7 @@ Initial mapping: Emit exact inputs alongside every mapped result. -## Milestone 11 — Exploration +## Milestone 10 — Exploration Only after Dataset v1 is trustworthy, start experimental interfaces/notebooks/apps: diff --git a/docs/PIPELINE.md b/docs/PIPELINE.md index 201e70d..a3bb220 100644 --- a/docs/PIPELINE.md +++ b/docs/PIPELINE.md @@ -71,6 +71,13 @@ observations to centimetres and write `artworks.parquet`. Freeze `smk-public-vis explicit rights/image/colors criteria and emit membership reasons. Colour feature extraction is a later stage. +## Stage 3 — deterministic visual features + +For each frozen corpus entity, deduplicate exact source palette colours, assign explicitly uniform +analytical weights, convert sRGB to OKLab/OKLCH, calculate compact palette statistics, carry source +visual values and derive eligible geometry. Emit scalar feature and long-form palette Parquets plus +diagnostics. Do not perform corpus-relative normalization or similarity here. + ## Ingest performance rule The source files are large. Use bounded-memory streaming where practical. A one-time conversion to Parquet/Arrow as an interim representation is encouraged if it simplifies all later stages. diff --git a/src/smk_playground/cli.py b/src/smk_playground/cli.py index 6f894fd..c92a066 100644 --- a/src/smk_playground/cli.py +++ b/src/smk_playground/cli.py @@ -37,5 +37,14 @@ def milestone2(root: Path | None = None) -> None: typer.echo(f"Milestone 2 complete: {summary['canonical_entity_count']} entities") +@app.command("milestone3") +def milestone3(root: Path | None = None) -> None: + """Build deterministic per-artwork visual features for the frozen corpus.""" + from smk_playground.features.milestone3 import run + + summary = run((root or Path.cwd()).resolve()) + typer.echo(f"Milestone 3 complete: {summary['feature_rows']} feature rows") + + if __name__ == "__main__": app() diff --git a/src/smk_playground/features/color.py b/src/smk_playground/features/color.py new file mode 100644 index 0000000..642c1f5 --- /dev/null +++ b/src/smk_playground/features/color.py @@ -0,0 +1,70 @@ +"""Small deterministic sRGB -> OKLab -> OKLCH implementation.""" + +from __future__ import annotations + +import math +import re +from dataclasses import dataclass + +ACHROMATIC_EPSILON = 1e-7 +COLOR_IMPLEMENTATION = "internal-oklab-published-matrices-v1" +HEX_RE = re.compile(r"^#[0-9a-fA-F]{6}$") + + +@dataclass(frozen=True) +class ConvertedColor: + hex: str + srgb_r: float + srgb_g: float + srgb_b: float + oklab_l: float + oklab_a: float + oklab_b: float + oklch_l: float + oklch_c: float + oklch_h_deg: float | None + + +def _linear_channel(channel: float) -> float: + return channel / 12.92 if channel <= 0.04045 else ((channel + 0.055) / 1.055) ** 2.4 + + +def hex_to_oklab(value: str) -> ConvertedColor: + if not isinstance(value, str) or not HEX_RE.fullmatch(value): + raise ValueError(f"Invalid #RRGGBB color: {value!r}") + canonical = value.casefold() + r = int(canonical[1:3], 16) / 255.0 + g = int(canonical[3:5], 16) / 255.0 + b = int(canonical[5:7], 16) / 255.0 + linear_r, linear_g, linear_b = map(_linear_channel, (r, g, b)) + + l = 0.4122214708 * linear_r + 0.5363325363 * linear_g + 0.0514459929 * linear_b + m = 0.2119034982 * linear_r + 0.6806995451 * linear_g + 0.1073969566 * linear_b + s = 0.0883024619 * linear_r + 0.2817188376 * linear_g + 0.6299787005 * linear_b + l_root = math.copysign(abs(l) ** (1 / 3), l) + m_root = math.copysign(abs(m) ** (1 / 3), m) + s_root = math.copysign(abs(s) ** (1 / 3), s) + oklab_l = 0.2104542553 * l_root + 0.7936177850 * m_root - 0.0040720468 * s_root + oklab_a = 1.9779984951 * l_root - 2.4285922050 * m_root + 0.4505937099 * s_root + oklab_b = 0.0259040371 * l_root + 0.7827717662 * m_root - 0.8086757660 * s_root + chroma, hue = oklab_to_oklch(oklab_l, oklab_a, oklab_b)[1:] + return ConvertedColor( + hex=canonical, + srgb_r=r, + srgb_g=g, + srgb_b=b, + oklab_l=oklab_l, + oklab_a=oklab_a, + oklab_b=oklab_b, + oklch_l=oklab_l, + oklch_c=chroma, + oklch_h_deg=hue, + ) + + +def oklab_to_oklch(lightness: float, a: float, b: float) -> tuple[float, float, float | None]: + chroma = math.hypot(a, b) + if chroma < ACHROMATIC_EPSILON: + return lightness, chroma, None + return lightness, chroma, math.degrees(math.atan2(b, a)) % 360.0 + diff --git a/src/smk_playground/features/extract.py b/src/smk_playground/features/extract.py new file mode 100644 index 0000000..3b416ef --- /dev/null +++ b/src/smk_playground/features/extract.py @@ -0,0 +1,214 @@ +"""Pure per-artwork feature extraction for artart-visual-features v1.""" + +from __future__ import annotations + +import math +from typing import Any + +from .color import ACHROMATIC_EPSILON, ConvertedColor, hex_to_oklab, oklab_to_oklch + +PALETTE_SOURCE = "smk_bulk_colors" +PALETTE_WEIGHTING = "uniform" +ASPECT_MISMATCH_THRESHOLD = math.log(2.0) + + +def analytical_palette(source_colors: list[str] | None) -> tuple[list[dict[str, Any]], list[str]]: + flags: list[str] = [] + if not source_colors: + return [], ["PALETTE_EMPTY"] + unique: list[tuple[int, str, str]] = [] + seen: set[str] = set() + for source_rank, source_hex in enumerate(source_colors, start=1): + canonical = source_hex.casefold() + if canonical in seen: + if "PALETTE_DUPLICATE_SOURCE_COLOR" not in flags: + flags.append("PALETTE_DUPLICATE_SOURCE_COLOR") + continue + seen.add(canonical) + unique.append((source_rank, source_hex, canonical)) + if not unique: + return [], sorted({*flags, "PALETTE_EMPTY"}) + weight = 1.0 / len(unique) + rows: list[dict[str, Any]] = [] + for palette_rank, (source_rank, source_hex, canonical) in enumerate(unique, start=1): + converted = hex_to_oklab(canonical) + rows.append(_palette_row(palette_rank, source_rank, source_hex, weight, converted)) + if all(row["oklch_h_deg"] is None for row in rows): + flags.append("PALETTE_ACHROMATIC") + return rows, sorted(flags) + + +def _palette_row( + palette_rank: int, + source_rank: int, + source_hex: str, + weight: float, + color: ConvertedColor, +) -> dict[str, Any]: + return { + "palette_rank": palette_rank, + "source_rank": source_rank, + "source_hex": source_hex, + "hex": color.hex, + "weight": weight, + "srgb_r": color.srgb_r, + "srgb_g": color.srgb_g, + "srgb_b": color.srgb_b, + "oklab_l": color.oklab_l, + "oklab_a": color.oklab_a, + "oklab_b": color.oklab_b, + "oklch_l": color.oklch_l, + "oklch_c": color.oklch_c, + "oklch_h_deg": color.oklch_h_deg, + } + + +def _weighted_mean(values: list[float], weights: list[float]) -> float: + return math.fsum(weight * value for value, weight in zip(values, weights, strict=True)) + + +def _weighted_stddev(values: list[float], weights: list[float], mean: float) -> float: + variance = math.fsum( + weight * (value - mean) ** 2 for value, weight in zip(values, weights, strict=True) + ) + return math.sqrt(max(0.0, variance)) + + +def palette_features(palette: list[dict[str, Any]]) -> tuple[dict[str, float | None], list[str]]: + if not palette: + return { + "centroid_l": None, "centroid_a": None, "centroid_b": None, + "centroid_chroma": None, "centroid_hue_deg": None, + "palette_entropy_01": None, "effective_palette_size": None, + "palette_spread_oklab": None, "lightness_stddev": None, + "chroma_stddev": None, "hue_concentration_01": None, + }, ["PALETTE_EMPTY", "HUE_CONCENTRATION_UNAVAILABLE"] + weights = [row["weight"] for row in palette] + lightness = [row["oklab_l"] for row in palette] + a_values = [row["oklab_a"] for row in palette] + b_values = [row["oklab_b"] for row in palette] + chroma = [row["oklch_c"] for row in palette] + centroid_l = _weighted_mean(lightness, weights) + centroid_a = _weighted_mean(a_values, weights) + centroid_b = _weighted_mean(b_values, weights) + _, centroid_chroma, centroid_hue = oklab_to_oklch(centroid_l, centroid_a, centroid_b) + entropy = -math.fsum(weight * math.log(weight) for weight in weights if weight > 0) + n = len(palette) + entropy_01 = 0.0 if n <= 1 else min(1.0, max(0.0, entropy / math.log(n))) + spread = math.fsum( + row["weight"] * math.sqrt( + (row["oklab_l"] - centroid_l) ** 2 + + (row["oklab_a"] - centroid_a) ** 2 + + (row["oklab_b"] - centroid_b) ** 2 + ) + for row in palette + ) + mean_chroma = _weighted_mean(chroma, weights) + hue_x = 0.0 + hue_y = 0.0 + chromatic_weight = 0.0 + for row in palette: + if row["oklch_h_deg"] is None or row["oklch_c"] < ACHROMATIC_EPSILON: + continue + effective_weight = row["weight"] * row["oklch_c"] + radians = math.radians(row["oklch_h_deg"]) + hue_x += effective_weight * math.cos(radians) + hue_y += effective_weight * math.sin(radians) + chromatic_weight += effective_weight + flags: list[str] = [] + if chromatic_weight < ACHROMATIC_EPSILON: + hue_concentration = None + flags.extend(["PALETTE_ACHROMATIC", "HUE_CONCENTRATION_UNAVAILABLE"]) + else: + hue_concentration = min(1.0, math.hypot(hue_x, hue_y) / chromatic_weight) + if centroid_hue is None: + flags.append("CENTROID_ACHROMATIC") + return { + "centroid_l": centroid_l, + "centroid_a": centroid_a, + "centroid_b": centroid_b, + "centroid_chroma": centroid_chroma, + "centroid_hue_deg": centroid_hue, + "palette_entropy_01": entropy_01, + "effective_palette_size": math.exp(entropy), + "palette_spread_oklab": spread, + "lightness_stddev": _weighted_stddev(lightness, weights, centroid_l), + "chroma_stddev": _weighted_stddev(chroma, weights, mean_chroma), + "hue_concentration_01": hue_concentration, + }, sorted(set(flags)) + + +def geometry_features(canonical: dict[str, Any]) -> tuple[dict[str, float | int | None], list[str]]: + width = canonical.get("net_width_cm") + height = canonical.get("net_height_cm") + image_width = canonical.get("image_width") + image_height = canonical.get("image_height") + valid_physical = all( + isinstance(value, (int, float)) and math.isfinite(value) and value > 0 + for value in (width, height) + ) + valid_image = all( + isinstance(value, int) and not isinstance(value, bool) and value > 0 + for value in (image_width, image_height) + ) + flags: list[str] = [] + if valid_physical: + area = width * height + physical_aspect = width / height + else: + width = height = area = physical_aspect = None + flags.append("PHYSICAL_GEOMETRY_MISSING") + image_aspect = image_width / image_height if valid_image else None + mismatch = ( + abs(math.log(physical_aspect / image_aspect)) + if physical_aspect is not None and image_aspect is not None else None + ) + if ( + ASPECT_MISMATCH_THRESHOLD is not None and mismatch is not None + and mismatch > ASPECT_MISMATCH_THRESHOLD + ): + flags.append("PHYSICAL_IMAGE_ASPECT_MISMATCH") + return { + "net_width_cm": width, + "net_height_cm": height, + "net_area_cm2": area, + "net_aspect_ratio": physical_aspect, + "log_net_width_cm": math.log(width) if width is not None else None, + "log_net_height_cm": math.log(height) if height is not None else None, + "log_net_area_cm2": math.log(area) if area is not None else None, + "image_width_px": image_width if valid_image else None, + "image_height_px": image_height if valid_image else None, + "image_aspect_ratio": image_aspect, + "aspect_mismatch_log": mismatch, + }, flags + + +def extract_artwork(canonical: dict[str, Any]) -> tuple[dict[str, Any], list[dict[str, Any]]]: + source_colors = canonical.get("colors") + palette, palette_flags = analytical_palette(source_colors) + calculated, calculated_flags = palette_features(palette) + geometry, geometry_flags = geometry_features(canonical) + feature_flags = sorted({*palette_flags, *calculated_flags, *geometry_flags}) + canonical_flags = sorted(canonical.get("quality_flags") or []) + feature = { + "smk_entity_id": canonical["smk_entity_id"], + "object_number": canonical.get("object_number"), + "palette_source": PALETTE_SOURCE, + "palette_weighting": PALETTE_WEIGHTING, + "palette_source_size": len(source_colors or []), + "palette_analytic_size": len(palette), + "source_palette": source_colors, + "first_palette_color_hex": source_colors[0] if source_colors else None, + "palette_source_order_preserved": True, + **calculated, + "smk_brightness": canonical.get("brightness"), + "smk_contrast": canonical.get("contrast"), + "smk_saturation": canonical.get("saturation"), + "smk_entropy": canonical.get("entropy"), + "smk_colortemp": canonical.get("colortemp"), + **geometry, + "canonical_quality_flags": canonical_flags, + "feature_quality_flags": feature_flags, + "quality_flags": sorted({*canonical_flags, *feature_flags}), + } + return feature, palette diff --git a/src/smk_playground/features/milestone3.py b/src/smk_playground/features/milestone3.py new file mode 100644 index 0000000..ef412a3 --- /dev/null +++ b/src/smk_playground/features/milestone3.py @@ -0,0 +1,324 @@ +"""Build deterministic per-artwork visual features for smk-public-visual-v1.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import math +import statistics +from collections import Counter +from pathlib import Path +from typing import Any + +import pyarrow as pa +import pyarrow.parquet as pq + +from smk_playground.features.color import ACHROMATIC_EPSILON, COLOR_IMPLEMENTATION +from smk_playground.features.extract import ASPECT_MISMATCH_THRESHOLD, extract_artwork + +FEATURE_SCHEMA_NAME = "artart-visual-features" +FEATURE_SCHEMA_VERSION = "1.0.0" +EXTRACTOR_VERSION = "1.0.0" +CORPUS_VERSION = "smk-public-visual-v1" + +PROVENANCE_FIELDS = [ + ("feature_schema", pa.string()), + ("feature_schema_version", pa.string()), + ("extractor_version", pa.string()), + ("generated_at_utc", pa.string()), + ("canonical_dataset_sha256", pa.string()), + ("corpus_definition", pa.string()), + ("corpus_dataset_sha256", pa.string()), + ("color_implementation", pa.string()), +] + +FEATURE_SCHEMA = pa.schema([ + *PROVENANCE_FIELDS, + ("smk_entity_id", pa.string()), + ("object_number", pa.string()), + ("palette_source", pa.string()), + ("palette_weighting", pa.string()), + ("palette_source_size", pa.int64()), + ("palette_analytic_size", pa.int64()), + ("source_palette", pa.list_(pa.string())), + ("first_palette_color_hex", pa.string()), + ("palette_source_order_preserved", pa.bool_()), + ("centroid_l", pa.float64()), + ("centroid_a", pa.float64()), + ("centroid_b", pa.float64()), + ("centroid_chroma", pa.float64()), + ("centroid_hue_deg", pa.float64()), + ("palette_entropy_01", pa.float64()), + ("effective_palette_size", pa.float64()), + ("palette_spread_oklab", pa.float64()), + ("lightness_stddev", pa.float64()), + ("chroma_stddev", pa.float64()), + ("hue_concentration_01", pa.float64()), + ("smk_brightness", pa.float64()), + ("smk_contrast", pa.float64()), + ("smk_saturation", pa.float64()), + ("smk_entropy", pa.float64()), + ("smk_colortemp", pa.float64()), + ("net_width_cm", pa.float64()), + ("net_height_cm", pa.float64()), + ("net_area_cm2", pa.float64()), + ("net_aspect_ratio", pa.float64()), + ("log_net_width_cm", pa.float64()), + ("log_net_height_cm", pa.float64()), + ("log_net_area_cm2", pa.float64()), + ("image_width_px", pa.int64()), + ("image_height_px", pa.int64()), + ("image_aspect_ratio", pa.float64()), + ("aspect_mismatch_log", pa.float64()), + ("canonical_quality_flags", pa.list_(pa.string())), + ("feature_quality_flags", pa.list_(pa.string())), + ("quality_flags", pa.list_(pa.string())), +]) + +PALETTE_SCHEMA = pa.schema([ + *PROVENANCE_FIELDS, + ("smk_entity_id", pa.string()), + ("object_number", pa.string()), + ("palette_source", pa.string()), + ("palette_weighting", pa.string()), + ("palette_rank", pa.int64()), + ("source_rank", pa.int64()), + ("source_hex", pa.string()), + ("hex", pa.string()), + ("weight", pa.float64()), + ("srgb_r", pa.float64()), + ("srgb_g", pa.float64()), + ("srgb_b", pa.float64()), + ("oklab_l", pa.float64()), + ("oklab_a", pa.float64()), + ("oklab_b", pa.float64()), + ("oklch_l", pa.float64()), + ("oklch_c", pa.float64()), + ("oklch_h_deg", pa.float64()), +]) + +DIAGNOSTIC_FIELDS = ( + "centroid_l", "centroid_chroma", "palette_spread_oklab", "lightness_stddev", + "chroma_stddev", "palette_entropy_01", "effective_palette_size", + "hue_concentration_01", "smk_brightness", "smk_contrast", "smk_saturation", + "smk_entropy", "smk_colortemp", "net_area_cm2", "net_aspect_ratio", + "image_aspect_ratio", "aspect_mismatch_log", +) + + +def sha256_file(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + while chunk := stream.read(1024 * 1024): + digest.update(chunk) + return digest.hexdigest() + + +def percentiles(values: list[float]) -> dict[str, float | int | None]: + finite = sorted(value for value in values if value is not None and math.isfinite(value)) + if not finite: + return {"count": 0, "min": None, "p01": None, "p05": None, "p25": None, + "median": None, "p75": None, "p95": None, "p99": None, "max": None} + + def quantile(probability: float) -> float: + position = (len(finite) - 1) * probability + low = int(position) + high = min(low + 1, len(finite) - 1) + fraction = position - low + return finite[low] * (1 - fraction) + finite[high] * fraction + + return { + "count": len(finite), "min": finite[0], "p01": quantile(0.01), + "p05": quantile(0.05), "p25": quantile(0.25), "median": statistics.median(finite), + "p75": quantile(0.75), "p95": quantile(0.95), "p99": quantile(0.99), + "max": finite[-1], + } + + +def _provenance(root: Path, canonical_path: Path, corpus_path: Path) -> dict[str, str]: + manifest = json.loads( + (root / "data" / "processed" / "source-manifest.json").read_text(encoding="utf-8") + ) + return { + "feature_schema": FEATURE_SCHEMA_NAME, + "feature_schema_version": FEATURE_SCHEMA_VERSION, + "extractor_version": EXTRACTOR_VERSION, + # A fixed manifest timestamp makes identical inputs byte-reproducible across repeated runs. + "generated_at_utc": manifest["generated_at_utc"], + "canonical_dataset_sha256": sha256_file(canonical_path), + "corpus_definition": CORPUS_VERSION, + "corpus_dataset_sha256": sha256_file(corpus_path), + "color_implementation": COLOR_IMPLEMENTATION, + } + + +def _metadata(provenance: dict[str, str]) -> dict[bytes, bytes]: + return {f"smk.{key}".encode(): value.encode() for key, value in provenance.items()} + + +def _example_payload( + feature: dict[str, Any], palette: list[dict[str, Any]], in_corpus: bool, +) -> dict[str, Any]: + return { + "in_corpus": in_corpus, + "smk_entity_id": feature["smk_entity_id"], + "object_number": feature["object_number"], + "source_palette": feature["source_palette"], + "analytical_palette": [ + { + "rank": row["palette_rank"], "source_rank": row["source_rank"], + "hex": row["hex"], "weight": row["weight"], "oklch_l": row["oklch_l"], + "oklch_c": row["oklch_c"], "oklch_h_deg": row["oklch_h_deg"], + } + for row in palette + ], + "centroid": { + key: feature[key] for key in + ("centroid_l", "centroid_a", "centroid_b", "centroid_chroma", "centroid_hue_deg") + }, + "palette_spread_oklab": feature["palette_spread_oklab"], + "hue_concentration_01": feature["hue_concentration_01"], + "source_visual": { + key: feature[key] for key in + ("smk_brightness", "smk_contrast", "smk_saturation", "smk_entropy", "smk_colortemp") + }, + "geometry": { + key: feature[key] for key in + ("net_width_cm", "net_height_cm", "net_area_cm2", "net_aspect_ratio", + "image_width_px", "image_height_px", "image_aspect_ratio", "aspect_mismatch_log") + }, + "flags": feature["quality_flags"], + } + + +def run(root: Path) -> dict[str, Any]: + canonical_path = root / "data" / "processed" / "artworks.parquet" + corpus_path = root / "data" / "processed" / "smk-public-visual-v1.parquet" + features_path = root / "data" / "processed" / "artwork-visual-features-v1.parquet" + palettes_path = root / "data" / "processed" / "artwork-palettes-v1.parquet" + report_path = root / "data" / "reports" / "feature-summary.json" + examples_path = root / "data" / "reports" / "feature-examples.json" + provenance = _provenance(root, canonical_path, corpus_path) + metadata = _metadata(provenance) + feature_writer = pq.ParquetWriter(features_path, FEATURE_SCHEMA.with_metadata(metadata), compression="zstd") + palette_writer = pq.ParquetWriter(palettes_path, PALETTE_SCHEMA.with_metadata(metadata), compression="zstd") + diagnostics: dict[str, list[float]] = {field: [] for field in DIAGNOSTIC_FIELDS} + palette_sizes: Counter[int] = Counter() + flag_counts: Counter[str] = Counter() + feature_flag_counts: Counter[str] = Counter() + visual_nulls: Counter[str] = Counter() + palette_rows_count = 0 + palette_achromatic_colors = 0 + centroid_null_hue_count = 0 + hue_concentration_null_count = 0 + examples: dict[str, dict[str, Any]] = {} + selectors: dict[str, tuple[float, dict[str, Any], list[dict[str, Any]]] | None] = { + "near_achromatic": None, "high_spread": None, + } + physical_example: tuple[dict[str, Any], list[dict[str, Any]]] | None = None + missing_physical_example: tuple[dict[str, Any], list[dict[str, Any]]] | None = None + feature_count = 0 + try: + parquet = pq.ParquetFile(corpus_path) + for batch in parquet.iter_batches(batch_size=1_000): + feature_rows: list[dict[str, Any]] = [] + palette_rows: list[dict[str, Any]] = [] + for canonical in batch.to_pylist(): + feature, palette = extract_artwork(canonical) + feature_count += 1 + palette_sizes[feature["palette_analytic_size"]] += 1 + for flag in feature["quality_flags"]: + flag_counts[flag] += 1 + for flag in feature["feature_quality_flags"]: + feature_flag_counts[flag] += 1 + if feature["centroid_hue_deg"] is None: + centroid_null_hue_count += 1 + if feature["hue_concentration_01"] is None: + hue_concentration_null_count += 1 + for field in DIAGNOSTIC_FIELDS: + value = feature[field] + if value is not None: + diagnostics[field].append(value) + for field in ("smk_brightness", "smk_contrast", "smk_saturation", "smk_entropy", "smk_colortemp"): + if feature[field] is None: + visual_nulls[field] += 1 + if feature["object_number"] == "KMS8892": + examples["KMS8892"] = _example_payload(feature, palette, True) + chroma = feature["centroid_chroma"] + if chroma is not None and ( + selectors["near_achromatic"] is None or chroma < selectors["near_achromatic"][0] + ): + selectors["near_achromatic"] = (chroma, feature, palette) + spread = feature["palette_spread_oklab"] + if spread is not None and chroma is not None and chroma >= 0.07 and ( + selectors["high_spread"] is None or spread > selectors["high_spread"][0] + ): + selectors["high_spread"] = (spread, feature, palette) + if physical_example is None and feature["net_area_cm2"] is not None: + physical_example = (feature, palette) + if missing_physical_example is None and feature["net_area_cm2"] is None: + missing_physical_example = (feature, palette) + feature_rows.append({**provenance, **feature}) + for palette_row in palette: + if palette_row["oklch_h_deg"] is None: + palette_achromatic_colors += 1 + palette_rows.append({ + **provenance, + "smk_entity_id": feature["smk_entity_id"], + "object_number": feature["object_number"], + "palette_source": feature["palette_source"], + "palette_weighting": feature["palette_weighting"], + **palette_row, + }) + if abs(math.fsum(row["weight"] for row in palette) - 1.0) > 1e-12: + raise ValueError(f"Palette weights do not sum to one: {feature['smk_entity_id']}") + feature_writer.write_table(pa.Table.from_pylist(feature_rows, schema=FEATURE_SCHEMA)) + palette_writer.write_table(pa.Table.from_pylist(palette_rows, schema=PALETTE_SCHEMA)) + palette_rows_count += len(palette_rows) + finally: + feature_writer.close() + palette_writer.close() + + for name, selected in selectors.items(): + if selected: + examples[name] = _example_payload(selected[1], selected[2], True) + if physical_example: + examples["with_physical_dimensions"] = _example_payload(*physical_example, True) + if missing_physical_example: + examples["without_physical_dimensions"] = _example_payload(*missing_physical_example, True) + + kks_table = pq.read_table(canonical_path, filters=[("object_number", "=", "KKS8001")]) + if kks_table.num_rows == 1: + kks_feature, kks_palette = extract_artwork(kks_table.to_pylist()[0]) + examples["KKS8001"] = _example_payload(kks_feature, kks_palette, False) + + summary = { + **provenance, + "achromatic_epsilon": ACHROMATIC_EPSILON, + "aspect_mismatch_threshold_log": ASPECT_MISMATCH_THRESHOLD, + "feature_rows": feature_count, + "palette_rows": palette_rows_count, + "palette_size_distribution": dict(sorted(palette_sizes.items())), + "palette_achromatic_color_rows": palette_achromatic_colors, + "centroid_null_hue_count": centroid_null_hue_count, + "hue_concentration_null_count": hue_concentration_null_count, + "feature_quality_flag_counts": dict(sorted(feature_flag_counts.items())), + "all_quality_flag_counts": dict(sorted(flag_counts.items())), + "source_visual_null_counts": dict(visual_nulls), + "distributions": {field: percentiles(values) for field, values in diagnostics.items()}, + } + report_path.write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8") + examples_path.write_text(json.dumps(examples, ensure_ascii=False, indent=2), encoding="utf-8") + return summary + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--root", type=Path, default=Path.cwd()) + args = parser.parse_args() + print(json.dumps(run(args.root.resolve()), indent=2)) + + +if __name__ == "__main__": + main() diff --git a/tests/test_color_features.py b/tests/test_color_features.py new file mode 100644 index 0000000..e0c12b1 --- /dev/null +++ b/tests/test_color_features.py @@ -0,0 +1,86 @@ +import math + +import pytest + +from smk_playground.features.color import hex_to_oklab, oklab_to_oklch +from smk_playground.features.extract import ( + analytical_palette, + extract_artwork, + palette_features, +) + + +@pytest.mark.parametrize( + ("hex_value", "expected"), + [ + ("#ff0000", (0.62795536, 0.22486306, 0.12584630)), + ("#00ff00", (0.86643961, -0.23388757, 0.17949848)), + ("#0000ff", (0.45201372, -0.03245698, -0.31152815)), + ], +) +def test_known_srgb_oklab_reference_colors(hex_value: str, expected: tuple[float, ...]) -> None: + color = hex_to_oklab(hex_value) + assert (color.oklab_l, color.oklab_a, color.oklab_b) == pytest.approx(expected, abs=2e-7) + + +def test_achromatic_oklch_hue_is_null() -> None: + assert oklab_to_oklch(0.5, 0.0, 0.0) == (0.5, 0.0, None) + assert hex_to_oklab("#ffffff").oklch_h_deg is None + + +def test_uniform_weights_and_duplicate_source_colors() -> None: + palette, flags = analytical_palette(["#FF0000", "#ff0000", "#0000ff"]) + assert len(palette) == 2 + assert math.fsum(row["weight"] for row in palette) == pytest.approx(1.0) + assert [row["source_rank"] for row in palette] == [1, 3] + assert flags == ["PALETTE_DUPLICATE_SOURCE_COLOR"] + + +def test_one_color_palette_boundaries() -> None: + palette, _ = analytical_palette(["#ff0000"]) + features, _ = palette_features(palette) + assert features["palette_entropy_01"] == 0.0 + assert features["effective_palette_size"] == pytest.approx(1.0) + assert features["palette_spread_oklab"] == pytest.approx(0.0) + assert features["hue_concentration_01"] == pytest.approx(1.0) + + +def test_two_color_centroid_entropy_spread_and_hue() -> None: + palette, _ = analytical_palette(["#ff0000", "#0000ff"]) + features, _ = palette_features(palette) + red = hex_to_oklab("#ff0000") + blue = hex_to_oklab("#0000ff") + assert features["centroid_l"] == pytest.approx((red.oklab_l + blue.oklab_l) / 2) + assert features["centroid_a"] == pytest.approx((red.oklab_a + blue.oklab_a) / 2) + assert features["palette_entropy_01"] == pytest.approx(1.0) + assert features["effective_palette_size"] == pytest.approx(2.0) + assert 0 < features["palette_spread_oklab"] < 1 + assert 0 <= features["hue_concentration_01"] <= 1 + + +def test_geometry_logs_and_null_propagation_are_deterministic() -> None: + canonical = { + "smk_entity_id": "entity", "object_number": "A1", "colors": ["#ffffff"], + "net_width_cm": 10.0, "net_height_cm": 20.0, + "image_width": 100, "image_height": 200, "quality_flags": [], + } + first, _ = extract_artwork(canonical) + second, _ = extract_artwork(dict(canonical)) + assert first == second + assert first["net_area_cm2"] == 200.0 + assert first["log_net_area_cm2"] == pytest.approx(math.log(200.0)) + canonical["net_width_cm"] = None + missing, _ = extract_artwork(canonical) + assert missing["net_area_cm2"] is None + assert "PHYSICAL_GEOMETRY_MISSING" in missing["feature_quality_flags"] + + +def test_conservative_aspect_mismatch_flag() -> None: + canonical = { + "smk_entity_id": "entity", "object_number": "A1", "colors": ["#ffffff"], + "net_width_cm": 10.0, "net_height_cm": 10.0, + "image_width": 100, "image_height": 300, "quality_flags": [], + } + feature, _ = extract_artwork(canonical) + assert feature["aspect_mismatch_log"] == pytest.approx(math.log(3.0)) + assert "PHYSICAL_IMAGE_ASPECT_MISMATCH" in feature["feature_quality_flags"]