Benchmark release checklist
Evidence checker benchmark: cases awaiting reproducible reports
This register defines 16 public-safe cases across provenance, metadata, processing, and C2PA integrity. It deliberately does not label them as results until each case has a releasable input, its actual SHA-256, dated report JSON, and a human review note.
4
Camera originals
Baseline, stripped, recompressed, and converted copies.
4
AI original exports
Marker and metadata conditions without a binary-origin promise.
4
Processed copies
Screenshots and platform-like transformations that weaken provenance.
4
C2PA integrity
Manifest, trust, binding, and marker-only states.
Publication gate
No case below is an evidence result yet. For every released case we will publish the source and license, actual input SHA-256, processing steps, run date, backend and c2patool versions, trust-policy state, a redacted report JSON, conclusion, and limitation. Assets without redistribution rights will expose only their source, hash, and steps.
Planned case register
Camera metadata baseline
Pending public report- Source and license plan
- Img2det-generated non-sensitive JPEG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Camera metadata removed
Pending public report- Source and license plan
- Img2det-generated non-sensitive JPEG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Camera image recompressed once
Pending public report- Source and license plan
- Img2det-generated non-sensitive JPEG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Camera image converted to WebP
Pending public report- Source and license plan
- Img2det-generated non-sensitive WebP fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
AI provenance-marker baseline
Pending public report- Source and license plan
- Img2det-generated non-sensitive PNG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
AI marker with metadata retained
Pending public report- Source and license plan
- Img2det-generated non-sensitive PNG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
AI-style XMP marker fixture
Pending public report- Source and license plan
- Img2det-generated non-sensitive JPEG fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
AI-style byte marker fixture
Pending public report- Source and license plan
- Img2det-generated non-sensitive WebP fixture · Project-owned test fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Compare the report’s strongest evidence category with the documented fixture condition.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Screenshot-style metadata loss
Pending public report- Source and license plan
- Derived from project-owned fixture · Project-owned derivative fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Treat missing provenance as evidence about this processed copy, not proof about the original.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Social upload-style recompression
Pending public report- Source and license plan
- Derived from project-owned fixture · Project-owned derivative fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Treat missing provenance as evidence about this processed copy, not proof about the original.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Metadata-stripped copy
Pending public report- Source and license plan
- Derived from project-owned fixture · Project-owned derivative fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Treat missing provenance as evidence about this processed copy, not proof about the original.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Resized and converted copy
Pending public report- Source and license plan
- Derived from project-owned fixture · Project-owned derivative fixture
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Treat missing provenance as evidence about this processed copy, not proof about the original.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Trusted manifest expectation
Pending public report- Source and license plan
- C2PA-compatible public conformance fixture · Source and license recorded with release
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Inspect manifest, signature, trust, and asset-binding fields separately.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Manifest present but trust unresolved
Pending public report- Source and license plan
- C2PA-compatible public conformance fixture · Source and license recorded with release
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Inspect manifest, signature, trust, and asset-binding fields separately.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
Asset-binding mismatch expectation
Pending public report- Source and license plan
- C2PA-compatible public conformance fixture · Source and license recorded with release
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Inspect manifest, signature, trust, and asset-binding fields separately.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
No manifest / marker-only expectation
Pending public report- Source and license plan
- C2PA-compatible public conformance fixture · Source and license recorded with release
- Input SHA-256
- Not published — case has not yet run
- Required processing
- Before publication, run the named input through the production evidence checker without changing the documented input recipe.
- Evidence to inspect
- Inspect manifest, signature, trust, and asset-binding fields separately.
Release condition: Publication is blocked until a real input hash, dated report JSON, and human review note are recorded.
Limit: This is a planned controlled case, not a completed benchmark result and not a measure of real-world prevalence or detector accuracy.
User uploads are never included. See the methodology and editorial policy for review rules.