Faster vision-language and speech models
Document understanding, visual inspection and speech pipelines include image and audio encoders that most serving setups don't optimise. We tune the encoder, prefill and decode together.
What usually goes wrong
Encoders add latency
Each image adds thousands of prefill tokens and an encoder pass before the first output token.
Memory runs out
High-resolution images inflate the KV cache, so batch sizes drop.
Components tuned separately
OCR, layout, VLM and speech models are each tuned on their own rather than as one pipeline.
Products involved
VLM and speech model runs in Optimize
Quantisation lab
FP8, INT8, INT4 and NVFP4 rounding error on a layer with outlier channels.
Tell us what you need the model to do
Send us the model, the hardware and your target, whether that's tokens per second, latency, memory or cost. In a 30-minute call we'll tell you whether we think it's doable and roughly what it would take.
- ReviewA free 30-minute call. We look at your model, hardware and target and give you an honest estimate.
- SprintTwo weeks at a fixed fee, on your hardware. If we can't beat your current setup, you don't pay.
- RunWe run Serve and monitoring for you and keep re-tuning as traffic changes, under an agreed SLO.