Fixed-scope engineering work
We did this work by hand before we built the products, and we still do: two-week tuning sprints, custom kernels, speculative decoding, quantisation, edge deployment and capacity planning, all inside your environment.
Engineering services
We built the products from client work, and we still take on projects at a fixed scope and price for teams who'd rather have us do it.
Inference optimisation sprint
You give us the model, the hardware and the latency target. Two weeks later you get a tested configuration and a report. Fixed fee.
vLLM · SGLang · TensorRT-LLM · FP8 · INT4 · NVFP4 · speculative decoding · prefix cachingCustom kernels
Fused and specialised kernels for the cases where the standard ones are slow, tuned and checked at the SASS level.
Triton · CUDA · CUTLASS · wgmma · TMA · cp.async · Hopper · Blackwell · AdaSpeculative decoding
Draft models trained on your own traffic, so the speed-up holds up on your real prompts.
EAGLE-3 · Medusa · n-gram and prompt-lookup drafts · acceptance tuningQuantisation and distillation
Smaller models that keep the accuracy you approved, with eval results to show it.
AWQ · GPTQ · SmoothQuant · KV-cache quantisation · distillation · pruningEdge deployment and on-device training
Getting a model to fit on your device, run well, take updates and fine-tune in the field.
Jetson · Apple silicon · Qualcomm · Raspberry Pi · ExecuTorch · GGUF · MLX · QLoRA on deviceServing for agent workloads
Serving setups designed for long shared prompts, bursts of sub-agents and pauses during tool calls.
KV-aware routing · prefix sharing · capacity planningBenchmarking and capacity planning
Tokens per second per dollar on different GPUs and clouds, measured on your workload.
H100 · H200 · B200 · MI300X · L40S · consumer cards · buy-versus-rent modelsTraining throughput
Speeding up training runs: data loading, parallelism, kernels and memory use.
FSDP · DeepSpeed · Megatron · torch.compile · fused optimisers · MFU auditsHow a project usually goes
- Start here
Inference review Free · 30 minutes
Tell us about the model, the hardware you have or plan to buy, and the target you're aiming for. We'll give you an honest estimate of what's possible.
Book the review - Prove it
Optimisation sprint Fixed fee · 2 weeks
Our engineers use Optimize on your model and hardware. You get the tested configuration, any custom kernels, the eval results and a report.
If we don't beat your baseline, you don't pay. - Run it
Managed inference Monthly
We run Serve and its monitoring in your environment and keep re-tuning as your models and traffic change, under an agreed SLO.
Clients
“VitalOps helped us optimise our data pipelines for the finance databases, which directly translated into a smoother and more intuitive experience for our users.”
“The team is highly responsive, pragmatic, and product-minded, and the best part is that they understand what it takes to operationalize data, not just analyze it.”
What engineering teams ask us
Do we have to send you our weights or data?
No. Everything runs in your cloud account or data centre, and evals run on your data where it is. During sprints our engineers work inside your environment.
Which models can you speed up?
Open-weight models and your own fine-tunes, including Llama, Qwen, DeepSeek, Mistral, Gemma, GLM, Whisper, vision-language models, and embedding and reranker models. We can't speed up closed API models, but we can help with routing, caching and batching around them.
How do you make sure the faster model is still correct?
You set a minimum score on your eval set. Every candidate configuration is tested against it over several seeds and rejected if it falls short, however fast it is. Custom kernels are compared numerically with the reference and inspected at the SASS level.
Tell us what you need the model to do
Send us the model, the hardware and your target, whether that's tokens per second, latency, memory or cost. In a 30-minute call we'll tell you whether we think it's doable and roughly what it would take.
- ReviewA free 30-minute call. We look at your model, hardware and target and give you an honest estimate.
- SprintTwo weeks at a fixed fee, on your hardware. If we can't beat your current setup, you don't pay.
- RunWe run Serve and monitoring for you and keep re-tuning as traffic changes, under an agreed SLO.