Specification readiness
Every class has an observable rule, nearest confounder, positive and negative examples, and an uncertainty policy.
Illustrative delivery artifact
A quality plan should make acceptance a decision—not a feeling.
This example shows the structure Grasp uses to connect annotation work to a downstream model decision. The thresholds, sampling rates and reviewer profile are set for each project; they are never copied blindly from a generic benchmark.
Example program · multi-camera robotics video
Downstream decision
Improve failure detection and recovery analysis for manipulation episodes without teaching the model reviewer assumptions that the sensors do not support.
Unit of work
One synchronized episode containing camera streams, robot state, action timing, task outcome and any operator intervention.
Primary risks
Boundary drift, hidden-state inference, identity changes across views, missed recovery attempts and over-representation of clean successes.
Acceptance owner
A named Grasp operations lead and the customer’s model or data owner approve the specification, calibration evidence and final release.
1 · Evidence contract
Define what a label is allowed to claim.
For every label, the specification records the observable rule, its closest confounder and what to do when the evidence is insufficient. A state such as “secure grasp” cannot be inferred from a visually closed gripper alone when force or outcome evidence contradicts it.
- Positive, negative and boundary examples
- Required camera or sensor evidence
- Occlusion and uncertainty policy
- Temporal start and end rules
- Cross-view identity rules
- Escalation owner and adjudication record
2 · Calibration
Use a representative slice before production volume.
The calibration set intentionally contains ordinary examples, hard negatives, sensor problems, ambiguous boundaries and costly failure modes. Qualified annotators work independently before a reviewer compares disagreement by class and cause.
| Review question | Evidence retained |
|---|---|
| Can the rule be applied consistently? | Per-class agreement and adjudicated examples |
| Where does evidence become insufficient? | Uncertain cases grouped by cause |
| Which errors change the model decision? | Severity and deployment-slice analysis |
| Is the review policy economically useful? | Rework, escalation and review-effort distribution |
3 · Production controls
Place review where it changes the outcome.
Calibration agreement
Annotators and reviewers apply the rules to a representative slice; disagreement is investigated by class and failure mode.
Production control
Independent review and targeted sampling cover high-consequence classes, difficult slices and newly observed ambiguity.
Delivery acceptance
The customer receives the agreed format, versioned instructions, review evidence and a record of unresolved limitations.
An actual plan may include numeric gates, but only after the pilot establishes what is measurable and material. Grasp does not advertise a universal accuracy percentage that ignores class difficulty, ambiguity and error consequence.
4 · Release record
Deliver the decisions with the labels.
The release includes the agreed data format and the evidence needed to understand it later: specification version, ontology version, calibration outcome, review coverage, adjudications, known limitations and customer acceptance status.