Groq

Inference platform for large language models built on custom Language Processing Units (LPUs) instead of GPUs. Delivers low-latency text generation, speech-to-text, and vision inference at scale with an OpenAI-compatible API.

GroqCloud serves over two million developers and counts enterprises like the McLaren Formula 1 Team among its users. The LPU architecture, pioneered in 2016, uses on-chip SRAM and deterministic execution to achieve roughly 10x faster inference than GPU-based alternatives on comparable models. TruePoint numerics preserve model accuracy while enabling high throughput.

Key capabilities:

• OpenAI-compatible REST API with drop-in replacement via base URL swap
• Hosted models including Llama 3.3 70B, GPT OSS, Whisper Large v3, and vision-capable models
• Official Python and TypeScript SDKs with full type definitions
• Function calling, tool use, and multimodal support across text, speech, and vision
• Global data center deployment for low-latency inference

Ideal for real-time chat and streaming applications, voice transcription pipelines, and agents that need fast tool use. Developers integrate via the standard OpenAI client pattern or native Groq SDKs, with support for synchronous and asynchronous usage.