smk-playground/docs/SOURCE_DISAGREEMENTS.md

45 lines
2.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# DA/EN source disagreement audit
The detailed field-level report is `data/reports/source-disagreements.parquet`. It compares the
202,393 shared `object_number` identities while isolating 305 duplicate identity groups that first
need duplicate resolution.
## Findings
For comparable singleton records:
- `rights`: 50,425 differences, all the same localization pair: DA
`https://www.smk.dk/section/brug-af-museets-materiale/` versus EN
`https://www.smk.dk/en/section/use-of-smk-material/`;
- normalized dimensions: 156 differences;
- `has_image`: 7 differences;
- image width and height: 15 differences each (8 unequal, 5 missing in DA, 2 missing in EN);
- IIIF image IDs/info: 11 differences each (3 unequal, 6 missing in DA, 2 missing in EN);
- image thumbnail/native URLs: 15 differences each.
There were no comparable-singleton disagreements in `public_domain`, `colors`, brightness,
contrast, saturation, entropy or colortemp. Palette ordering therefore also agrees across the
language snapshots wherever both contain a comparable singleton record.
The image differences are not merely localization. Examples include `KKS1984-179` (2048×3072 DA
versus 5078×6618 EN with different IIIF IDs) and `KKSgb5300` (8180×5262 DA versus 999×643 EN).
The seven `has_image` disagreements go in both directions. Dimension differences include both
small numeric conflicts (`DEP977`) and measurements present in only one language.
## Source precedence recommendation
- Identity: match cross-language rows by SMK `id` where available and verify `object_number`; do
not use array position.
- Localized text: retain DA and EN independently with their explicit source language.
- `rights`: preserve both URLs, but map this exact localized pair to one policy concept. Use
`public_domain` plus the original rights URL for legal provenance.
- Palette and visual values: accept the shared value when equal. If a future snapshot disagrees,
flag it rather than apply global language precedence.
- Image metadata: choose a coherent image bundle (IIIF ID, width, height and URLs) at row level.
Prefer the bundle that is internally complete and consistent; if both differ and are complete,
retain both and require freshness/content verification. Never mix dimensions from one bundle
with an image ID from the other.
- Dimensions: compare localized canonical tuples, preserve both originals, and leave conflicting
canonical values unresolved unless an explicit part-aware rule decides them.
- Missing one language: retain the available source value with provenance.