Your pipeline processes 8,000 product listings daily using the Message Batches API. Each request uses the…

Question

Your pipeline processes 8,000 product listings daily using the Message Batches API. Each request uses the same 1,800-token system prompt (JSON schema definition + extraction instructions + normalization rules) alongside variable product description content. Without optimization, input token costs dominate your monthly bill. Which approach most effectively reduces input token costs for this workload?

Accepted Answer

B. Enable prompt caching on the system prompt so its tokens are billed at the reduced cache-read rate on repeated requests within the same batch.

Answer

A. Switch to a model with a larger context window to process multiple product listings in a single API call, reducing the number of requests.

Answer

C. Shorten the system prompt by removing normalization rules and schema documentation, relying on the model's base instruction-following capability.

Answer

D. Submit documents in fewer, larger batches to amortize the per-batch fixed overhead across more documents.

Question 248