A practitioner's guide for teams deciding whether AMD Instinct belongs in their AI infrastructure alongside, or instead of, NVIDIA. It starts from Lisa Su's December 2024 interview on Bloomberg's The Circuit and what AMD has shipped since, then covers the 2026 platforms layer by layer, the product lines buyers actually meet, the memory arithmetic that drives GPU counts, the CUDA-to-ROCm software map, a porting workflow with commands, running both vendors on one Kubernetes platform, a fair benchmarking method, a decision workflow and the operational checks for day two.
This technote accompanies our blog post Chipping away at Nvidia's lead: Lisa Su's plan, two years on, which summarises the interview. Vendor figures below are AMD's or NVIDIA's own published claims, not independent benchmarks; we say so where it matters. Facts are as of 14 November 2026.
1. From the interview to Helios
In "AMD's CEO Wants to Chip Away at Nvidia's Lead" (Bloomberg Originals, The Circuit with Emily Chang, 6 December 2024), Lisa Su, then ten years into the job, showed the Instinct MI300 running production AI work such as GPT-4o on Microsoft Azure and Meta's Llama, called AI "the most transformational technology" of her career, and argued there were "more than enough chips for everybody" even as hyperscalers designed their own. Bloomberg's chip reporter Ian King summed up AMD's position as "a solid number two". The two years since have tested that position.
AMD's third-quarter 2026 release reports total revenue of $13.4bn, Data Center revenue of a record $8.5bn (up 97% year on year), volume shipments of the MI450 Series, and Anthropic, Meta, Microsoft, OpenAI and Oracle Cloud Infrastructure running workloads on the first Helios racks. It guides to about $17bn of revenue in the fourth quarter. The same release notes ROCm 10, HPE's first Helios order (for Vultr), and EuroHPC's choice of Instinct MI430X GPUs for the LUMI-AI supercomputer, which matters for European buyers looking at sovereign AI capacity.
2. The 2026 platforms, layer by layer
| AMD Helios (AMD's published figures) | Value |
|---|---|
| GPUs per rack (one scale-up domain) | 72 × Instinct MI455X (CDNA 5) |
| Memory per GPU | 432 GB HBM4, up to 23.3 TB/s |
| Compute per GPU | up to 40 PF FP4 |
| Rack compute | 2.9 EF FP4, 1.4 EF FP8 |
| Rack memory | 31 TB HBM4, about 1.7 PB/s aggregate |
| Scale-up / scale-out bandwidth | 260 TB/s (UALink over Ethernet) / 43 TB/s (3 × 800G NICs per GPU) |
| Host CPU | 6th Gen EPYC "Venice" (9006 Series) |
| Rack standard | Open Rack Wide, contributed by Meta to the Open Compute Project |
AMD positions Helios directly against NVIDIA's Vera Rubin NVL72 and claims about 15% more AI compute, 50% more memory, 50% more scale-out bandwidth and up to 30% more tokens per dollar. Those are vendor claims measured by the vendor; treat them as a reason to benchmark, not as a result. The memory difference is the most structural: 432 GB per MI455X against 288 GB per Rubin GPU changes how many GPUs a large model needs (section 4).
3. The Instinct product lines you will meet
| Product | Memory | Where it fits |
|---|---|---|
| MI300X | 192 GB HBM3 | The 2024 generation in the interview; still common in clouds and the used market |
| MI325X | 256 GB HBM3E | Memory-uplifted MI300 platform, eight-GPU servers |
| MI350X / MI355X | 288 GB HBM3E | 2025 generation; eight-GPU servers, air or liquid cooled; the MI455X's comparison baseline |
| MI430X | see AMD | HPC and sovereign AI systems; chosen for EuroHPC's LUMI-AI |
| MI450X / MI455X | 432 GB HBM4 | Helios rack-scale systems; frontier training and large-scale inference |
| MI350P | see AMD | Used in AMD's Helion reference architecture for agentic systems and in the Threadripper Halo Station workstation |
Most UK organisations will meet Instinct as eight-GPU servers from Dell, HPE, Lenovo or Supermicro, or as cloud instances, rather than as full Helios racks. Helios matters to them indirectly: it is what makes ROCm a first-class target for the model and framework developers they depend on.
4. Memory arithmetic: why 432 GB changes GPU counts
Two numbers decide how many GPUs one copy of a model needs: the weights, and the key-value (KV) cache that grows with context length and concurrent users.
weights (GB) ≈ parameters (billions) × bytes per parameter
FP16/BF16 = 2, FP8 = 1, FP4 = 0.5
KV cache per token = 2 × layers × kv_heads × head_dim × bytes per value
# Example: a 70B model with 80 layers, 8 KV heads, head_dim 128, FP16 KV cache
KV per token = 2 × 80 × 8 × 128 × 2 bytes = 327,680 bytes ≈ 0.31 MiB
KV per 32k sequence ≈ 10 GiB KV per 128k sequence ≈ 40 GiB| Illustrative: 70B model, FP8 weights (~70 GB), FP16 KV, 10% headroom | 288 GB GPU | 432 GB GPU |
|---|---|---|
| Memory left for KV cache | ~189 GB | ~319 GB |
| Concurrent 32k-token sequences per GPU | ~18 | ~30 |
| Concurrent 128k-token sequences per GPU | ~4 | ~7 |
| ~400B-parameter model at FP8 (~400 GB weights) | needs 2+ GPUs | fits on 1, with little room for KV |
Fewer GPUs per model copy means less tensor parallelism, less inter-GPU traffic and more independent replicas from the same rack. Whether that becomes more tokens per pound depends on compute, interconnect and software efficiency too, which is why the benchmark in section 8 is the deciding step. Our GPU private cloud page has a VRAM sizer that runs the same arithmetic.
5. The software map: CUDA to ROCm
| Purpose | NVIDIA | AMD ROCm |
|---|---|---|
| Programming model | CUDA | HIP (CUDA-like C++; also compiles for NVIDIA) |
| Dense linear algebra | cuBLAS, cuBLASLt | hipBLAS, rocBLAS, hipBLASLt |
| Deep-learning primitives | cuDNN | MIOpen |
| Kernel templates | CUTLASS | Composable Kernel |
| Multi-GPU collectives | NCCL | RCCL (NCCL-compatible API) |
| Inference compiler | TensorRT, TensorRT-LLM | MIGraphX; most LLM serving uses vLLM or SGLang instead |
| Device management | nvidia-smi | amd-smi |
| Profiling | Nsight Systems / Compute | rocprofv3, ROCm Systems and Compute Profilers |
| Portable kernels | Triton (OpenAI), supported on both | |
| Frameworks and serving | PyTorch, JAX, vLLM, SGLang, Hugging Face: published builds for both | |
The practical rule: if your code stays at the framework level, moving to ROCm is mostly a matter of installing the ROCm build. The work concentrates in three places: custom CUDA kernels and C++ extensions, TensorRT engines, and libraries that ship CUDA-only binaries.
6. Porting workflow
6.1 Check the node and the framework
# GPUs visible to the driver, and their architecture (gfx target)
amd-smi list
rocminfo | grep -m1 -i "gfx"
# PyTorch on ROCm keeps the torch.cuda API, so most code is unchanged
python -c "import torch; print(torch.version.hip, torch.cuda.is_available(), torch.cuda.get_device_name(0))"6.2 Serve a model in a ROCm container
# AMD publishes ROCm images for PyTorch and vLLM; pick a tag matching your ROCm version
docker run --rm -it --ipc=host \
--device=/dev/kfd --device=/dev/dri --group-add video \
--security-opt seccomp=unconfined \
-p 8000:8000 -v ~/.cache/huggingface:/root/.cache/huggingface \
rocm/vllm:<tag> \
vllm serve <model> --tensor-parallel-size 2The OpenAI-compatible endpoint is identical to the one vLLM exposes on NVIDIA, so clients, gateways and tests do not change.
6.3 Custom kernels
# Mechanical CUDA → HIP translation, then build with the ROCm compiler
hipify-perl my_kernel.cu > my_kernel.hip.cpp # or hipify-clang for a full parse
hipcc -O3 --offload-arch=<gfx-target> -o my_kernel my_kernel.hip.cppTranslation is the easy part. Warp-size assumptions (AMD's CDNA wavefronts are 64 lanes, NVIDIA warps 32), hand-tuned shared-memory layouts and inline PTX need review by hand. Where a kernel is being rewritten anyway, write it in Triton once and run it on both vendors.
6.4 Validate before you tune
Run a fixed evaluation set with the same weights and precision on both vendors. Compare task scores (and, for kernels, logits or outputs within a stated tolerance). Different hardware gives slightly different floating-point results; what matters is that model quality on your tasks is unchanged. Only then tune batch sizes, parallelism and serving parameters.
7. One Kubernetes platform, two vendors
Both vendors expose GPUs to Kubernetes through an operator and a device plugin: nvidia.com/gpu on NVIDIA nodes and amd.com/gpu on AMD nodes, with AMD's GPU Operator installing the driver components, device plugin, node labeller and a metrics exporter for Prometheus. Install it from AMD's Helm chart following the current ROCm GPU Operator documentation.
Keep one manifest per workload and a small overlay per vendor that changes only the image build (ROCm or CUDA) and the GPU resource. With Kustomize:
# overlays/amd/kustomization.yaml (Workstation example)
resources: ["../../base"]
images:
- name: inference-server
newName: registry.example.com/inference-server
newTag: 2026.11-rocm
patches:
- target: {kind: Deployment, name: inference-server}
patch: |-
- op: replace
path: /spec/template/spec/containers/0/resources/limits
value: {"amd.com/gpu": 8}
- op: add
path: /spec/template/spec/nodeSelector
value: {"gpu-vendor": "amd"}Label and taint each node pool by vendor and generation (for example gpu-vendor=amd, gpu-model=mi355x) so schedulers and cost reports can tell them apart, and run the same CI tests against both overlays on every change.
8. Benchmark fairly
- Same model, same precision, same serving engine and version on both vendors, with each vendor's recommended build and settings.
- Measure at your latency target, not peak throughput: time to first token and inter-token latency at the p95 your users need, then throughput at that point.
vllm bench servedrives an OpenAI-compatible endpoint with realistic request mixes. - Use your traffic shape: prompt and output lengths, concurrency and the share of long-context requests.
- Convert to tokens per pound: hardware or rental cost per hour, plus power at your tariff, divided by tokens per hour at the latency target.
- Record accuracy on your evaluation set alongside speed.
- Cross-check with independent results such as MLPerf Inference where both vendors submit comparable systems.
9. Decide per workload
| Workload | Leans AMD when | Leans NVIDIA when |
|---|---|---|
| LLM inference, large models or long context | Memory per GPU cuts replicas' GPU count; vLLM or SGLang serving | You depend on TensorRT-LLM or NVIDIA NIM microservices |
| Training and fine-tuning in PyTorch | Standard PyTorch, FSDP or DeepSpeed; no custom CUDA | Heavy custom kernels, Megatron-specific CUDA paths, tight deadlines |
| Vision, speech and classic ML | Framework-level code, cost-sensitive inference | Libraries with CUDA-only binaries (some video and 3D pipelines) |
| HPC and simulation | FP64-heavy codes already ported to HIP; sovereign HPC programmes | CUDA-only commercial solvers |
10. Day-two operations
- Pin versions together: driver, ROCm release, framework build and container tag form one tested set, exactly as with CUDA. Upgrade them as a set, behind the golden-set tests.
- Monitor:
amd-smiexposes utilisation, memory, power, temperature and error counters; AMD's device metrics exporter publishes them to Prometheus next to DCGM metrics from NVIDIA nodes. Normalise label names so one dashboard covers both. - Health and RAS: alert on ECC and RAS error counts, thermal throttling and links dropping speed, and drain nodes automatically on repeated errors.
- Skills: train at least two engineers on ROCm tooling before production; most issues are version mismatches, which are quick to fix once someone knows where to look.
- Supply: keep both vendors qualified so a lead-time problem on one does not stop a project.
11. Risks to weigh honestly
- Ecosystem depth: CUDA still has more libraries, examples and people. Check every dependency, not just the framework.
- Vendor claims: both companies publish favourable comparisons. Independent and in-house measurements decide.
- Concentration: both vendors depend on the same leading-edge manufacturing in Taiwan, a point Su herself raised in the interview when she talked about supply-chain resilience. Dual-vendor reduces product risk, not geographic risk.
- Market momentum cuts both ways: AMD's large customer commitments make ROCm investment durable, but also compete with smaller buyers for early supply.
12. How Workstation can help
We design and run GPU infrastructure in the UK and help teams evaluate AMD alongside NVIDIA. For teams weighing Instinct we can run a like-for-like benchmark of your model on both vendors, assess your code for CUDA dependencies, set up a dual-vendor Kubernetes platform with shared monitoring, and size servers, power and cooling. Start with our GPU private cloud guide, compare leasing and buying at gpuleasing.co.uk, or talk to us.
Sources
- Bloomberg Originals, "AMD's CEO Wants to Chip Away at Nvidia's Lead | The Circuit with Emily Chang", 6 December 2024. Quotes are from the published video.
- AMD, "AMD Reports Third Quarter 2026 Financial Results", 3 November 2026 (revenue, Data Center growth, Helios customers, ROCm 10, LUMI-AI, Q4 outlook).
- AMD, Helios rack-scale solution and Instinct MI400 Series product pages, and "AMD Launches Helios" (2026): rack and GPU specifications and AMD's comparison with Vera Rubin NVL72.
- AMD and OpenAI agreement for 6 gigawatts of Instinct GPUs, announced 6 October 2025.
- Memory arithmetic and porting commands are Workstation examples; check model configurations and current ROCm documentation before relying on them.
Acknowledgement of rights: AMD, the AMD Arrow logo, Instinct, EPYC, Ryzen, Threadripper, CDNA, ROCm, Pensando, Helios and combinations thereof are trademarks of Advanced Micro Devices, Inc. NVIDIA, CUDA, NVLink, TensorRT, Spectrum-X, ConnectX and Vera Rubin are trademarks of NVIDIA Corporation. Bloomberg is a trademark of Bloomberg Finance L.P. Other names are the property of their respective owners. This article is independent and not endorsed by any of these companies.
