Training and running AI models, rendering and video encoding need GPUs. Providers offer them in several ways, from whole physical servers to slices of one card. This guide explains GPU servers for AI and how virtualization shares GPUs.
Before you start
- Know your workload: model size, training vs inference, batch sizes, and whether you need several GPUs working together.
Ways GPUs are offered
| Option | How it works | Good for |
|---|---|---|
| Dedicated GPU server | Bare metal with one or more GPUs | Long training runs, full control |
| GPU VM with passthrough | Whole GPU passed to a VM via VFIO | Near-native performance with VM flexibility |
| vGPU | Vendor software splits a GPU into time-shared virtual GPUs | VDI, light inference, many small users |
| MIG (Multi-Instance GPU) | Supported data-centre GPUs partitioned into isolated hardware slices | Predictable shared inference |
| Serverless/managed inference | Pay per request or token | Using models without managing GPUs |
Passthrough on your own KVM host is covered in PCI and GPU passthrough with VFIO.
Specs that matter
- VRAM: the model (plus activations, KV cache) must fit. This is often the deciding factor.
- Compute and tensor performance at the precision you use (FP16, BF16, FP8, INT8).
- Interconnect: NVLink or similar between GPUs, and fast networking between servers for multi-node training.
- CPU, RAM and NVMe: data loading can bottleneck a fast GPU.
- Software support: drivers, CUDA/ROCm versions and container images.
Cost considerations
- Hourly cloud GPUs suit experiments and bursts; dedicated servers usually win for constant use.
- Spot or preemptible instances are cheaper but can be interrupted; checkpoint often.
- Storage and data egress add up; see cloud egress costs.
Getting started on a GPU VM
- Choose an image with drivers preinstalled, or install the vendor driver matching your CUDA/ROCm version.
- Check the GPU:
nvidia-smi(NVIDIA) orrocm-smi(AMD). - Run workloads in containers with GPU support (for example the NVIDIA Container Toolkit).
- Secure it like any server: first steps on a new VPS.
Frequently asked questions
Can a normal VPS run AI models?
Small models can run on CPU, slowly. Most modern AI workloads need a GPU.
Is passthrough slower than bare metal?
Usually only marginally; the GPU itself runs natively.
Does MdmHost offer GPU servers?
MdmHost focuses on cPanel web, reseller and master reseller hosting. This guide is general background.