Your automated extraction pipeline displays an impressive 96% aggregate accuracy on the dashboard. However,…

Question

Your automated extraction pipeline displays an impressive 96% aggregate accuracy on the dashboard. However, downstream teams complain about frequent errors when processing complex financial contracts. What monitoring methodology resolves this visibility gap?

Accepted Answer

A. Tracking accuracy per document type and field (stratified metrics) instead of relying solely on aggregate metrics.

Answer

B. Increasing the overall sampling size randomly to capture a broader dataset.

Answer

C. Switching entirely to the Message Batches API to ensure complete asynchronous processing.

Answer

D. Utilizing fork_session to isolate contract documents into separate evaluation workflows.

Question 140