You are building a structured data extraction system using Claude. The system extracts information from…

Question

You are building a structured data extraction system using Claude. The system extracts information from unstructured documents, validates the output using JavaScript Object Notation (JSON) schemas, and maintains high accuracy. It must handle edge cases gracefully and integrate with downstream systems.

After deployment, you find that 12% of extractions contain semantic errors that pass JSON schema validation (e.g., a duration like "30 minutes" incorrectly placed in an ingredient quantity field). Human reviewers have capacity to check only 20% of extractions.

Which approach most effectively allocates reviewer attention?

Accepted Answer

B. Have the model output field-level confidence scores, then calibrate review thresholds using a labeled validation set.

Answer

A. Prioritize review of all extractions where required fields are empty or explicitly marked as not found.

Answer

C. Review all extractions from documents with formatting anomalies such as unusual layouts or mixed content types.

Answer

D. Randomly sample 20% of extractions for review, using corrections to track accuracy and identify error patterns.

Question 114