How the Best AI Cloud Providers Actually Run Their Infrastructure

Summary

• The AI cloud market now rewards providers that run their infrastructure like a hyperscaler: self-service, isolated tenants, and tooling customers already know.
• The best AI cloud providers share an operating model: bare metal GPUs with no hypervisor tax, tenant clusters that give every customer their own isolated control plane, and pre-validated AI stacks that ship in minutes instead of months.
• Named providers like Nscale and Nebius are useful as proof of that model. For a builder, the details that matter are what they did under the hood.
• For an AI cloud provider, inference provider, or enterprise AI factory building the same capability, vCluster Platform turns raw GPUs into every kind of cluster. CoreWeave and Nscale built on it.

There was a time when evaluating an "AI cloud provider" meant comparing a GPU catalog: how many H100s, how many regions, who had the newest Blackwell racks. Most lists you find still read that way, a ranking of brands by fleet size.

If you are renting a place to run your fine-tuning job, that list is roughly what you asked for. But if you are trying to understand why the leading AI clouds win deal after deal, the GPU catalog is the least interesting part of the story.

The providers pulling ahead deliver a hyperscaler experience on top of the GPUs they run: self-service environments, isolated tenants, and the tooling customers already know. This guide is about that operating model, what the best AI cloud providers do differently under the hood, and what it takes to replicate it.

The GPU Catalog Stopped Being the Differentiator

GPU specs are converging. Every serious AI cloud now runs NVIDIA Hopper or Blackwell, with similar networking and similar per-GPU-hour economics. When the hardware is the same, the buyer's decision moves up the stack.

The providers escaping the price war are the ones giving customers a reason to stay. Three reasons, none of them about rack density.

1. They sell environments instead of raw compute. Selling bare metal GPUs is a race to the bottom. Selling a managed, self-service experience is a margin business. The top AI clouds let a customer spin up an isolated environment in seconds, the way they would on AWS or GCP, instead of filing a ticket and waiting for a VM.

2. They isolate tenants properly. A paying customer expects their own cluster, their own API server, their own RBAC, and a hard boundary between themselves and the next customer sharing the hardware. Namespaces do not clear that bar. Neither does a shared control plane where one noisy tenant starves everyone else.

3. They ship turnkey AI tooling. Raw Kubernetes is an engine without a cockpit. The providers that keep customers longer offer Run:AI, Ray, PyTorch, Jupyter, or Slurm pre-configured, in minutes, not weeks of integration work.

When you look at the leading GPU clouds through this lens, the more useful question is what they built to deliver that experience.

How the Leading AI Clouds Actually Run

The most useful way to read the market is as proof of an operating model rather than a leaderboard. Two examples make the point concrete.

Nscale runs large-scale GPU supercluster infrastructure and presents its architecture publicly at KubeCon. The pattern is a bare metal Kubernetes underlay with vCluster Platform providing each customer a fully isolated tenant cluster on dedicated Private Nodes. They manage one platform that creates an isolated tenant cluster per customer in seconds, rather than maintaining a separate physical cluster for each customer.

Nebius uses vCluster to unify distributed inference and wire AI cloud suppliers into its "Token Factory" inference gateway. The pattern again is orchestration: one control plane that turns many compute sources into one consistent experience.

CoreWeave was an early design partner of vCluster, building their original managed Kubernetes offering on its tenant isolation technology. (Source: vCluster case studies)

What these three share is a stack that treats every cluster as a product they can ship, with isolation strong enough for paying customers and provisioning fast enough to close a deal.

The Operating Model, Layer by Layer

Strip away the brand names and the same four decisions show up in every winning GPU cloud. This is the part a builder needs to get right.

Bare metal over virtualization. The hypervisor tax is real: slower boot times from large GPU BARs, reduced throughput, complex passthrough. Top providers run workloads directly on bare metal. That means zero-touch provisioning (PXE boot, OS install, machine registration) so a racked GPU becomes bookable capacity instead of a ticket.

Tenant clusters instead of namespaces. Every customer gets their own isolated control plane: their own API server, etcd, RBAC, and CRDs. For production, the default is Private Nodes: dedicated worker nodes per tenant with their own CNI and storage, hardware-level isolation approaching a dedicated physical cluster. This is what separates a commercial GPU cloud from a shared hobby cluster.

Any cluster type, beyond Kubernetes. The newest shift is that the isolation model now extends to more than Kubernetes. Slurm, Run:AI, Ray, and inference clusters can each be delivered as an isolated, managed product. vCluster provides the isolation envelope for these cluster types; scheduling itself is delivered by SchedMD/Slinky, Ray, and Run:AI respectively. A provider that can offer a turnkey Slurm environment to one customer and a Ray environment to another is selling more than raw GPUs.

Pre-validated AI stacks. Certified Stacks ship pre-validated environments for partner tools like Run:AI, Ray, Jupyter, and Slurm (via Slinky), turning a bare cluster into a production AI platform in minutes. They deploy and work with the isolation model out of the box.

This is the part most builders underestimate. Assembling these four decisions yourself is measured in months and headcount, not days. The winning providers compressed that timeline by building on an orchestration platform instead of building one.

What This Means for a Builder

If you are an AI cloud provider, an inference provider, or an enterprise standing up an internal AI factory, the more important question is what orchestration stack to build on.

Long-term winners build the layer that makes those GPUs fast, safe, and cheap to share. That layer is exactly what vCluster Platform provides:

• vCluster Platform turns raw GPUs into every kind of isolated tenant cluster, including Kubernetes, Slurm, Run:AI, Ray, and inference. Private Nodes is the production default, giving each tenant hardware-level isolation without provisioning a separate physical cluster. Fleet management, self-service portals, auto-sleep, and Day 2 operations are included.
• vMetal handles zero-touch bare metal provisioning paired with vCluster Standalone, a CNCF-certified control plane that runs as a single binary with no k3s or kubeadm required.
• vNode adds kernel-native workload isolation using seccomp, cgroups, and namespaces. No VMs and no hypervisor tax; bare metal performance with breakout protection.
• Certified Stacks ship pre-validated environments for partner tools like Run:AI, Ray, Jupyter, and Slurm (via Slinky) that turn a bare cluster into a production AI platform in minutes.

Production scale: 100K+ GPUs powered, 40M+ tenant clusters created, validated in the NVIDIA DGX reference architecture. Boost Run launched a production managed Kubernetes service in under 45 days. Lintasarta launched Indonesia's leading GPU cloud in 90 days.

Whether you are selecting an AI cloud provider or building one, the infrastructure foundation matters more than the GPU catalog. The providers that win long-term turn racks into an experience their customers will not leave.

See how vCluster can help you build that: request a personalized demo.

Frequently Asked Questions

What actually separates the best AI cloud providers from the rest?

Specs have converged across the market, so the GPU catalog no longer separates providers. The providers pulling ahead deliver a hyperscaler experience on top of their GPUs: self-service environments, hard tenant isolation, and pre-validated AI tooling. They sell managed environments instead of raw compute, and that is the margin difference.

Do the leading AI clouds really run on the same kind of infrastructure?

Most of them share a recognizable pattern: bare metal GPUs with no hypervisor layer, tenant clusters that give each customer an isolated control plane, and turnkey AI stacks. Nscale, Nebius, and CoreWeave each built on vCluster Platform for tenant isolation, which points at the same operating model rather than a coincidence.

What is the difference between selling bare metal and selling an AI cloud?

Selling bare metal GPUs is a race to the bottom on price. Selling an AI cloud means adding self-service environments, tenant isolation, and turnkey tooling, which is where the margin lives. The buyer is willing to pay for the AWS-like experience: spin up an isolated environment in seconds, get the tools they already know, and never think about the hardware underneath.

Why does tenant isolation matter so much for an AI cloud provider?

Because a paying customer expects a hard boundary between their workloads and every other customer sharing the same hardware. Namespaces do not clear that bar, and neither does a shared control plane where one tenant can starve another. Tenant clusters with Private Nodes give each customer their own isolated environment, which is the difference between a commercial GPU cloud and a hobby cluster.

Do I need bare metal, or is hypervisor-based virtualization fine?

The hypervisor tax is measurable: slower boot times from large GPU BARs, reduced throughput, and complex passthrough configurations. For GPU workloads where every percentage point of utilization is margin, bare metal is the right default. That means zero-touch provisioning so hardware becomes programmable capacity instead of a manual process.

Can one platform handle more than Kubernetes clusters?

Yes, and this is the direction the market is moving. The isolation model that works for Kubernetes extends to Slurm, Run:AI, Ray, and inference clusters, so a provider can offer a turnkey Slurm environment to one customer and a Ray environment to another. Every cluster type becomes a product you can ship instead of a one-off integration.

How long does it take to build the same capability as the leading providers?

Assembling bare metal provisioning, tenant orchestration, workload isolation, and AI tooling yourself is measured in months and headcount. Building on an orchestration platform compresses that to weeks: Boost Run launched a production managed Kubernetes service in under 45 days, and Lintasarta launched Indonesia's leading GPU cloud in 90 days.

Is vCluster a GPU cloud itself?

No. vCluster is the infrastructure layer AI cloud providers build on. It turns raw GPUs into isolated tenant clusters, but it does not sell GPU hours to end users. Providers like Nscale and Nebius deliver the customer-facing experience.

‍