RTX 4090 & 5090 servers.
The best price per token.

For inference of models that fit in 24 to 48 GB, RTX cards deliver more tokens per dollar. Datacenter GPUs make sense when you need 80 GB on one card, HBM bandwidth for training, or NVLink between GPUs.

Yes. vLLM and llama.cpp split a model across both cards (tensor or layer parallelism), which gives you 64 GB for a 4-bit 70B model. The cards talk over PCIe, not NVLink, so scaling is good for inference and more limited for training.

If you need 48 GB on a single card with ECC memory, yes: it runs 32B models at 8-bit and 14B models at full precision without splitting them across GPUs. A 4-bit 70B model needs about 50 GB, so plan for two cards or 80 GB. For smaller models, the RTX 5090 is faster and cheaper.

Yes, on the Ubuntu 24.04 image. The RTX 5090 needs a recent CUDA build of PyTorch; the GPU images guide gives the exact commands.

GPU servers are not refundable once delivered, because the hardware is set aside for you. Check the specifications and the GPU images guide before you order.