Modular MAX

What is Modular MAX?

Modular MAX is a hardware-agnostic serving framework from Modular that optimizes kernels and request execution across accelerators. It is part of Modular's inference stack, which also includes the Mojo programming language and hosted or self-hosted deployment. Teams use it to serve open models and their own models through a single container and an OpenAI-compatible API, without tying production inference to one GPU vendor.

Key capabilities of Modular MAX

• Hardware portability: runs on NVIDIA, AMD, AWS Trainium, Google TPU, Qualcomm, Intel, ARM and Apple silicon.
• OpenAI-compatible serving: exposes models behind an OpenAI-style API so existing client code keeps working.
• Model library: runs 1000+ models out of the box, including DeepSeek and Kimi families.
• Mojo GPU kernels: ships hundreds of composable GPU kernels written in Mojo.
• Custom model support: lets teams port their own models using PyTorch-like modeling APIs.
• Flexible deployment: runs on Modular Cloud, in your own VPC or as a self-hosted container.

How Modular MAX works

You deploy MAX as a single container, or use Modular Cloud, and point an OpenAI-compatible client at the endpoint. MAX handles kernel selection and request execution for the accelerator underneath. The vendor reports roughly 2x the performance of vLLM on diverse hardware. Custom models are ported through its modeling APIs, which the vendor says can be done in minutes.

Who uses Modular MAX?

Modular MAX is aimed at teams running production AI inference who want to avoid GPU vendor lock-in, and at developers who want to run open models on infrastructure they control. Infrastructure and platform engineers evaluating NVIDIA versus AMD capacity are a natural audience.

Modular MAX pricing

Running MAX yourself is free, as is installing Mojo. On Modular Cloud, shared endpoints are priced per token, while dedicated and custom-model endpoints are priced per minute. Specific rates are on the vendor pricing page.

Modular MAX alternatives

Vast.ai is a GPU rental marketplace, so you rent raw capacity rather than adopt a serving framework. DeepInfra offers hosted inference APIs for open models, which suits teams that do not want to operate their own stack. Nebius is a cloud provider with GPU infrastructure, so it competes at the compute layer more than the software layer.