Same model, different provider, wildly different speed

Output tokens per second — how fast the model generates text after it starts. This is the number you feel when a response streams in. It does not include time-to-first-token (prefill latency), which matters for short conversations but less for long generation tasks.

Hardware. Cerebras runs models on wafer-scale chips designed for inference throughput. Groq uses LPU (Language Processing Unit) chips. Cloud providers use GPU clusters with varying batch sizes, quantization, and serving software. The model weights are the same; the silicon and software stack are not.

Three types of source, ranked by reliability:

Measured: numbers from Artificial Analysis, which runs standardised tests against live API endpoints. These are the most comparable.

Announced: numbers from provider press releases or launch posts. These are real measurements but on the provider's own hardware under their own conditions, so they may not reflect what you see in practice.

Claimed: marketing numbers that have not been independently verified. Marked with a note.

Inference speeds are not stable. Providers upgrade hardware, change batching, add quantization, and adjust pricing. A number that is true today may be wrong next month. The date column shows when each measurement was taken.