After deploying the automated review, you notice high precision but low recall—real bugs are slipping…

Question

After deploying the automated review, you notice high precision but low recall—real bugs are slipping through undetected. Investigation reveals that your review prompt instructs Claude to “only report high-confidence issues you are certain about” and “err on the side of not commenting.” Developers appreciate the low noise, but a race condition that caused a production outage was visible in a reviewed pull request and went unreported. You need to substantially improve bug detection while keeping false-positive rates manageable. What is the most effective approach?

Accepted Answer

C. Split the review into a finding stage whose objective is comprehensive coverage—reporting every potential issue with confidence and severity metadata—and a separate stage that verifies and thresholds those findings.

Answer

A. Add detailed few-shot examples demonstrating bug categories Claude should flag—race conditions, null dereferences, and error-handling gaps—while retaining the high-confidence filtering instruction.

Answer

B. Remove the conservative instructions and have Claude report every potential issue, then apply a programmatic filter that deduplicates findings and suppresses historically noisy categories.

Answer

D. Expand the context to include related tests, recent Git history, and the module’s dependency graph so Claude has richer evidence for judging severity.

Question 281