If you’ve imported 10,000 Google Maps businesses, every one ships with a thin “Located in Bedford” placeholder description that does nothing for SEO. The traditional fix, running 10,000 sequential AI calls, costs around $12-15 in tokens and takes five hours on a healthy connection. Smart Directory Pro uses each provider’s Batch API (Gemini, OpenAI, Anthropic) to enrich the entire catalogue for $5-7 in five minutes. Same model, same content quality, half the price, 60× faster. This page explains how it works and where you can use it.
What the Batch API is and why we use it
Each major AI provider exposes a second flavour of their generation API: instead of submitting one prompt and waiting for one response, you submit a queue of thousands of prompts in a single HTTP call. The provider then processes them in parallel on their own infrastructure and returns the results when the batch completes. You pay half the per-token price as a trade for the asynchronous workflow.
- Gemini Batch API, submit up to ~10k inline prompts per batch, results returned as JSON inline. 50% discount.
- OpenAI Batches, submit a JSONL file, returns within 24h, 50% discount.
- Anthropic Message Batches, submit inline JSON, JSONL stream of results, 50% discount.
Smart Directory Pro speaks all three through one dispatcher (submit_*_batch / poll_*_batch), with prefixed batch IDs so the admin UI knows which provider to poll. When you click “Submit batch” from the Bulk Operations tab, the plugin picks the active provider, builds the prompt set, posts it once, and polls every 30 seconds until the provider returns the results, at which point each row writes back to the relevant listing / category / SEO page.
The numbers: sequential vs batch
| Workload | Sequential | With Batch API | Cost saving |
|---|---|---|---|
| 10,000 listing descriptions (500 words each) | ~5 hrs · $12-15 | ~5 min · $5-7 | ~50% |
| 1,200 category descriptions | 20-60 min · full price | 5-10 min · 50% off | ~50% |
| 1,200 category SEO metas | 20-60 min · full price | 5-10 min · 50% off | ~50% |
| 658 SEO landing pages × 2 calls each (TLDR + FAQ) | 30-60 min · full price | ~10 min · 50% off | ~50% |
| 10,000 FAQ generations | ~7 hrs · $18-22 | ~7 min · $9-11 | ~50% |
Where SDP uses Batch API today
- Listing description enrichment. Settings → AI → Bulk Operations → “Enrich descriptions” turns thin GMaps imports into 400-600 word listings.
- Listing SEO meta, title + meta description for every listing, in one batch.
- Listing FAQs, 7 Q&A pairs per listing, derived from listing facts + top reviews.
- Category descriptions + SEO meta, 1,200+ category terms enriched in a single submission.
- SEO landing pages. Tier-1 and Tier-2 programmatic SEO pages get their TLDR bullets and FAQ pairs generated in batches of all 658 pages × 2 batch types = 2 submissions total.
- Guide generation, multi-section guides use the same batch infrastructure for parallel section calls when supported (Gemini, OpenAI), falling back to sequential on Anthropic.
Under the hood: submit, poll, dispatch
The dispatch layer is intentionally provider-agnostic. Every batch path goes through a single entry point that:
- Builds the prompt array (one per item in the working set) using the same prompt templates the sequential flow uses.
- Calls the right per-provider
submit_{provider}_batch(, )method. - Prefixes the returned job ID (
gem-batch:<id>,oai-batch:<id>,ant-batch:<id>) and stores it in a transient + on the post so the UI can resume polling across page loads. - Polls every 30 seconds via
poll_{provider}_batch( )until the job reports completed, failed, cancelled, or expired. - On success, unpacks the responses, validates each one, and writes back to the relevant
_sdp_*postmeta or category description.
The poll loop is resumable: closing the browser tab doesn’t lose progress. The job ID is persisted; reopening the Bulk Operations tab picks back up wherever the poll left off. If the provider returns partial results (rare, but happens for Anthropic on individual prompt failures), the successful rows commit and the failures are surfaced as a downloadable error CSV, same row IDs as the input, with the error message per row.
How to trigger a batch run
- Go to Smart Directory → Settings → AI.
- Open the Bulk Operations subtab.
- Pick the operation (Enrich descriptions, SEO meta, FAQs, Category descriptions, SEO Pages TLDR/FAQ).
- Each card shows the working-set size, estimated cost, and estimated time. Click Submit batch.
- The UI shows a black status panel with live progress and current cost. You can close the tab, the poll resumes when you come back.
When sequential is still right
- One-off rewrites. Updating one listing’s description from the meta box editor, that’s a single sequential call, faster than the batch round-trip.
- Real-time AI semantic search. The “Search” mode on the frontend has to respond in <3s; batch is for offline work.
- Live FAQ generation triggered by a user action. When a visitor clicks “Generate FAQ for this listing” we go sequential so they get a response in seconds.
- Audit runs. Per-listing audits run sequentially because the audit prompt depends on the previous step’s output.
Provider matrix
| Provider | Discount | Typical turnaround | Max batch size | Notes |
|---|---|---|---|---|
| Gemini (Flash, Flash-Lite, Pro) | 50% | 5-15 min | ~10,000 per submission | Inline JSON in & out. Recommended default for SDP. |
| OpenAI (4o-mini, 4o) | 50% | 10 min – 24 hr | 50,000 per file | JSONL upload then batch create. Two HTTP calls to submit. |
| Anthropic (Claude Sonnet, Haiku) | 50% | 5-30 min | 100,000 per batch | Prompt caching applies on top of batch, additional 90% read discount on the cached system prompt. |
You pick the provider once in Settings → AI → Configuration. Batch operations route automatically; you don’t have to choose per-job.
FAQ
Is the content quality different in batch vs sequential mode?
No. Same model, same prompt, same parameters. The only difference is when you get the response and what you pay for it.
What if a batch fails partway?
Successful rows commit immediately. Failed rows are surfaced as a downloadable error CSV with the row ID + the provider’s error message. You can re-run only the failures by uploading that CSV back into the importer; the failure rate on properly configured batches is typically <0.5%.
Do I need to pay upfront?
No. You’re billed by the provider exactly as you would be for sequential calls (just at half the per-token rate). The cost estimate in the UI is based on the working set’s token count; actual billed amount can vary slightly depending on how the model generates.
Can I cancel a running batch?
Yes, click “Cancel batch” in the progress panel. Gemini and Anthropic cancel immediately and you’re charged only for completed prompts. OpenAI’s cancel is best-effort; depending on where the batch is in their queue, you may be charged for prompts that completed between your cancel request and the provider acknowledging it.
What happens if my server times out?
The submission is one fast HTTP call (1-3s). The polling happens client-side from the admin tab via REST, your PHP process isn’t held open. Worst case: close the tab. Reopen it; the poll resumes.