Benchmark image cost at 960×1472 without cooking the books

Label benchmark conditions clearly

The supplied fast-route figures for one 960×1472 portrait are $0.0006 on Runware, about $0.0037 on Together, and about $0.0045 on Leonardo. FLUX dev on Runware was supplied as $0.007. These numbers are the starting hypothesis for the router and calculator.

They should not be published as current universal provider pricing until the production team reruns them. The official provider pages may present different units or example settings. Account discounts, steps, output format, and failed requests can all change the billed result.

Record enough detail for another engineer to reproduce the run

Use a fixed prompt set covering the application’s real distribution, not one favorable prompt. Run requests in randomized order to reduce time-of-day bias. Capture provider request IDs, model identifiers, dimensions, steps, seeds where meaningful, result count, output format, region, and timestamps.

For every call, record HTTP outcome, generation outcome, billed cost when available, wall-clock duration, safety outcome, and whether the output passes a blinded acceptance rubric. Preserve raw logs privately and publish an aggregated dataset that does not expose sensitive prompts or credentials.

• At least several prompt categories and enough repetitions to see variance
• Warm-up runs separated from measured runs
• Same concurrency policy for each provider
• Explicit treatment of retries and rate limits
• Review date and commit hash for the harness

Report distributions and accepted-output economics

A mean hides tail latency and incident behavior. Report p50, p90, and p95 duration, success rate, timeout rate, moderation rejection rate, and provider cost. Then calculate cost per accepted image from blinded review.

If one route is cheaper but produces twice as many unusable outputs, the raw call price overstates its advantage. If the product needs fast previews, latency may matter more than final acceptance because users will iterate.

provider | model | n calls | success % | p50 ms | p95 ms
provider cost / call | acceptance % | cost / accepted image
benchmark date | dimensions | steps | known limitations

Publish updates instead of overwriting history

Give each benchmark run a date and preserve prior results. A short changelog explains whether cost moved because of provider pricing, model settings, route changes, or methodology corrections. This builds credibility and creates useful first-party SEO content over time.

Avoid ranking providers with an absolute ‘best’ label. State which route fits which workload under the measured conditions. Link comparison pages back to this methodology so every claim has one canonical evidence source.