Design efficient batch processing strategies

What the Message Batches API buys and what it costs

The trade is explicit, and it runs on more axes than cost.

	Synchronous Messages API	Message Batches API
Cost	standard pricing	50% cost savings
Latency	immediate response	up to 24 hours, no guaranteed SLA
Tool calling	client-side tool loops run turn by turn, and the response streams	same turn-by-turn loop, but no stream parameter — you retrieve a finished result, not a live one
Retrieval	the response is the return value	poll for completion; no push callback
Workload it fits	blocking work: pre-merge gates, interactive commands	non-blocking, latency-tolerant work: overnight quality reports, weekly compliance audits, nightly test generation, corpus backfills

Most batches finish far sooner than the window, but you cannot build a promise on that. The only number you may plan against is 24 hours, which is why a pre-merge check that gates a developer's merge cannot sit in that queue. One operational detail beyond the guide: at the 24-hour mark, unfinished requests come back marked expired, not late.⁵⁰ The ceiling is a deadline, not a delay.

The capability limit deserves reading precisely, because the exam tests its exact scope. The guide excludes the multi-turn case, so an agentic loop that needs to call a tool, read the result and continue belongs on the synchronous API. The wording implies that a one-shot request returning a tool_use block — the structured-extraction pattern from 4.3 — is not what the restriction is about, since no tool is executed mid-request. The API documentation settles that reading: tool use, including all server tools, and multi-turn conversations are both batchable.⁵⁰ What is testable is the guide's exclusion itself, so answer with it on the exam.

custom_id is the join key, and results are polled

Each request carries a custom_id and every result comes back with it. Results are not guaranteed to arrive in submission order, so custom_id — not array position — is how you correlate response to request, and how you identify precisely which documents failed.⁵¹ Keep the id short and plain: the API constrains its characters and its length.⁵² Each result also carries a type. succeeded and expired are the two that matter for the certification; the API also returns others specific to it.

Retrieval is a pull, not a push: you check the batch's status until it is done, then read the results. Nothing wakes your pipeline up when the work finishes, which is a second reason batch cannot sit under a blocking gate. Adding status polling does not rescue such a workflow either, because progress reporting shortens nothing.

Sizing submission cadence against an SLA

The guide's own worked example: you promise a 30-hour SLA and batch processing may take up to 24 hours. That leaves 6 hours of slack, which is the maximum time a document may wait in your queue before submission. Submitting every 4 hours therefore keeps the worst case at 4 + 24 = 28 hours, inside the promise with margin. That is the guide's arithmetic and the credited answer; it assumes the window delivers, which is what the expiry note above corrects. The general recipe: maximum queue window = promised SLA − 24 hours, then pick a submission interval at or below that.

Handling failures and de-risking the first pass

When a batch ends with partial failures, resubmit only the failed custom_ids, with a modification appropriate to the cause — chunk the documents that exceeded context limits, fix the ones that hit a validation problem. Resubmitting the whole batch pays a second time for work that already succeeded.

Because a wrong prompt is amplified by volume, refine the prompt on a small sample set before batch-processing large volumes. Every first-pass failure becomes an iterative resubmission cost, so a sample run that lifts the first-pass success rate is usually the cheapest step in the whole pipeline.

Pre-merge PR review        blocking, developer waiting   -> synchronous API
Post-merge deep analysis   non-blocking, results by 9am  -> Message Batches API
Nightly test generation    non-blocking, overnight       -> Message Batches API
Weekly compliance audit    non-blocking, 7-day cadence   -> Message Batches API
Interactive /explain cmd   blocking, human in the loop   -> synchronous API

Rule of thumb: if a person or a pipeline gate is waiting on the answer,
there is no latency SLA to rely on and batch is the wrong choice --
regardless of the 50% saving.

const failed: string[] = [];

for await (const r of client.messages.batches.results(batchId)) {
  if (r.result.type === 'succeeded') {
    save(r.custom_id, r.result.message);   // key by custom_id, not order
  } else {
    failed.push(r.custom_id);              // errored | expired | canceled
  }
}

// Resubmit only the failures, with a modification matched to the cause.
const retryRequests = failed.flatMap((id) =>
  exceededContextLimit(id)
    ? chunk(documents[id]).map((part, i) => buildRequest(id + '-part' + i, part))
    : [buildRequest(id, documents[id])]
);

if (retryRequests.length > 0) {
  await client.messages.batches.create({ requests: retryRequests });
}

Promised end-to-end SLA .................... 30 h
Batch processing worst case ................ 24 h
Remaining budget for queue wait ............  6 h

Submission interval chosen ..................  4 h
Worst case for any document ..... 4 h wait + 24 h processing = 28 h  <= 30 h  OK

Submitting once daily (24 h interval) gives 24 + 24 = 48 h -- breaks the SLA
even though most batches would finish in under an hour. Plan against the
documented window, not against observed latency.

The exam's arithmetic ends here. In production add margin: a request still
running at 24 h is returned as `expired` and must be resubmitted, so the real
worst case is 24 h + one more submission cycle.