How fast is your LLM API?
Benchmark local or remote LLM backends via an OpenAI-compatible streaming API. The gauge automatically adapts to the measured inference speed.
Live inference
0.0 tok/s
0150tok/s
LLM decode
— tok/s
Prompt ingest
— tok/s
First token · TTFT
— ms
Total generated
— tokens