H100 PCIe vs SXM: What Is the Difference?

NVIDIA H100 is sold in more than one physical form, and the difference between PCIe and SXM matters once workloads move beyond a single GPU.

Both are Hopper-generation accelerators, both are designed for AI and high-performance computing, and both can run the same CUDA software stack. The important differences are power envelope, memory system, interconnect and the way the GPUs are deployed inside servers.

H100 PCIe in plain English

The PCIe version is designed to fit into conventional accelerator servers through the PCI Express bus. That makes it easier for system builders to integrate and can make single-GPU or smaller deployments more flexible.

PCIe systems are often a good fit when the workload does not need every GPU to exchange data with every other GPU at extremely high speed. Inference, fine-tuning, experimentation and many single-GPU jobs can fall into this category.

H100 SXM in plain English

SXM is a module format designed for tightly integrated GPU servers. In common HGX systems, multiple SXM GPUs are linked with high-bandwidth NVLink connections.

The higher power envelope and faster GPU-to-GPU fabric are the main reasons SXM systems are commonly used for large distributed training jobs. When a model is split across several GPUs, communication overhead can become a major part of total runtime. Faster interconnect helps reduce that penalty.

Why interconnect matters

A GPU can be very fast on its own and still scale poorly across a cluster if the GPUs spend too much time waiting for data from one another.

Training large language models often involves tensor parallelism, pipeline parallelism or other distributed strategies. These methods move large amounts of data between accelerators. In that environment, the network inside the server and between servers can matter almost as much as the GPU itself.

For a workload that fits comfortably on one accelerator, the interconnect advantage may matter far less.

Memory and workload fit

The H100 family is widely associated with 80 GB class configurations, but exact memory type and configuration depend on the form factor and system. For current hardware details, use the NVIDIA H100 product information and check the exact server specification offered by the cloud provider.

If your main constraint is memory capacity rather than raw compute, also compare the NVIDIA H200, which increases accelerator memory substantially while remaining in the Hopper generation.

PCIe or SXM for AI training?

For large multi-GPU training, SXM is usually the configuration to investigate first because high-bandwidth GPU-to-GPU communication is especially valuable.

That does not mean PCIe is unsuitable for training. Smaller models, fine-tuning jobs and workloads with less communication can run efficiently on PCIe systems. The real question is whether the workload is compute-bound, memory-bound or communication-bound.

PCIe or SXM for inference?

Inference can be very different from training. A service that runs one model replica per GPU may not benefit as much from a tightly connected multi-GPU fabric.

For very large models that must be split across several accelerators, however, interconnect becomes important again. Long-context workloads may also be limited by memory capacity, making H200 or newer accelerators relevant.

Price is not enough

Two listings that both say "H100" may not represent equivalent infrastructure. When comparing cloud offers, check:

• whether the GPU is PCIe, SXM or NVL
• GPU memory capacity
• number of GPUs in the instance
• NVLink or other GPU-to-GPU connectivity
• host CPU and RAM
• local and network storage
• networking between nodes
• region and data-transfer charges
• whether the price is on-demand, spot or reserved
• whether capacity is actually available

Use the GPU Price Tracker to compare published rates and the GPU cloud provider directory to compare providers. For a workload-level estimate, use the GPU cloud cost calculator.

The practical takeaway

Choose the infrastructure around the workload, not just the GPU name.

H100 PCIe can be a strong choice for flexible single-GPU and smaller-scale workloads. H100 SXM is designed for dense, high-performance systems where multiple GPUs need to work together with very high bandwidth.

For a broader generation comparison, see H100 vs H200. For the underlying accelerator profile, see NVIDIA H100.