Book a review
Platform

How the products work together

All VitalOps products use the same unit, called a bundle. It records the engine version, settings, quantised weights, kernels and draft model that were tested together. The configuration Plan sizes for and Optimize tests is the one Serve, Edge and Fleet run.

Book an inference review
1deployment format across all products
0weights or prompts sent outside your environment
4deployment options: cloud, data centre, air-gapped, devices
Architecture

Architecture

Each part can be used on its own. Used together, problems found in production can start new Optimize runs, and those results feed back into Plan.

  1. Your traffic, models and evals

    Prompts, weights and eval sets stay where they are, and VitalOps reads them there.

  2. Plan

    Sizes the hardware for your traffic and latency target.

  3. Optimize and Sim

    Find and test the best configuration, and compare kernels on CPU.

  4. The bundle

    Engine version, flags, quantised weights, kernels and draft model, measured together.

  5. Serve, Edge and Fleet

    Run the bundle on GPUs, devices or agent swarms.

  6. Monitoring

    Watches production and sends regressions back to Optimize.

Deployment

Where it runs

Your cloud account

AWS, Google Cloud, Azure, CoreWeave, Lambda or any Kubernetes cluster. Installed with Helm or Terraform. We don't hold your keys.

Your data centre

Bare metal or Kubernetes, Slurm for batch work. NVIDIA and AMD GPUs, InfiniBand or Ethernet.

Air-gapped

Offline installs and signed updates. Nothing connects back to us, and licence checks work offline.

Devices

Jetson, Apple, Android, Qualcomm, Raspberry Pi and x86. Signed exports and staged over-the-air updates.

Supported stack

Everything we work with

Serving and compilers

Hardware

Techniques

Models

Products

Products

PlanCapacity and hardware planningOptimizeAgentic inference optimisationSimKernel work without a GPUServeModel serving and monitoringEdgeOn-device models and trainingFleetAgent fleets on your GPUs
Get started

Tell us what you need the model to do

Send us the model, the hardware and your target, whether that's tokens per second, latency, memory or cost. In a 30-minute call we'll tell you whether we think it's doable and roughly what it would take.

Book an inference reviewhello@vitalops.ai
  1. ReviewA free 30-minute call. We look at your model, hardware and target and give you an honest estimate.
  2. SprintTwo weeks at a fixed fee, on your hardware. If we can't beat your current setup, you don't pay.
  3. RunWe run Serve and monitoring for you and keep re-tuning as traffic changes, under an agreed SLO.