Inferless

What is Inferless?

Inferless is a serverless GPU platform for deploying machine learning models. You can deploy from Hugging Face, Git, Docker or the CLI, and it scales from zero to hundreds of GPUs. A banner on its site announces that Inferless is joining Baseten.

Key capabilities of Inferless

• Serverless deployment: deploy from Hugging Face, Git, Docker or CLI.
• Autoscaling: scales from zero to hundreds of GPUs with a custom load balancer.
• Custom runtimes: with writable volumes.
• CI/CD rebuilds: automated rebuilds, logs and monitoring.
• Dynamic batching: batches requests to raise throughput.
• Configurable endpoints: scale-down, timeouts, concurrency and webhooks.

How Inferless works

You point Inferless at a model, it builds a container, and it runs the model on GPUs that scale with traffic and shut down when idle. A customer testimonial on the site says billing follows hours used rather than a flat fee.

Who uses Inferless?

ML engineers and AI product teams that want to serve models without managing GPU clusters. The vendor cites SOC-2 Type II certification.

Inferless pricing

Inferless does not list prices on its homepage and links to a pricing page. Usage-based billing is described by a customer. Since the team is joining Baseten, ask the vendor how long the product will be offered.

Inferless alternatives

Cerebrium is another serverless GPU platform for deploying models. SaladCloud rents GPU capacity from a distributed network. Crusoe operates data centers and cloud GPU infrastructure at larger scale.