NVIDIA B200 Architecture : Blackwell GPU Architecture Explained

NVIDIA B200 is a data-center GPU based on the NVIDIA Blackwell architecture, designed for large-scale artificial intelligence (AI), generative AI, high-performance computing (HPC), and inference workloads. Unlike a conventional monolithic GPU, Blackwell uses two reticle-limited GPU dies connected via a high-speed 10 TB/s chip-to-chip interconnect, operating as a unified GPU. The B200 combines this design with fifth-generation Tensor Cores, a second-generation Transformer Engine, HBM3e memory, fifth-generation NVLink, and specialized data-movement capabilities.

This article explains the NVIDIA B200 architecture from the GPU die and Streaming Multiprocessor (SM) level to Tensor Cores, memory hierarchy, HBM3e, NVLink, precision scaling, and multi-GPU operation. It also explains how B200 differs from H100 and how it relates to the GB200 Grace Blackwell Superchip.

What Is NVIDIA B200?

NVIDIA B200 is a Blackwell-generation data-center GPU designed primarily for AI training, inference, and accelerated computing. It is one of the principal GPU products in NVIDIA’s Blackwell platform and is designed to provide high compute throughput, large high-bandwidth memory capacity, and fast GPU-to-GPU communication. Please refer to our in-depth article on AI Chip Basics.

The B200 uses the Blackwell architecture’s dual-die design. NVIDIA describes Blackwell GPUs as containing two reticle-limited dies connected by a 10 TB/s chip-to-chip interconnect and operating as a unified GPU. The architecture contains 208 billion transistors and is manufactured using a custom TSMC 4NP process.

The B200 is intended for workloads such as:

• Large language model (LLM) training
• Generative AI
• Large-scale inference
• Recommendation systems
• Scientific computing
• High-performance computing
• Data analytics
• Multi-GPU AI workloads

B200 vs Blackwell vs GB200

These three terms describe different levels of NVIDIA’s Blackwell platform:

Term	What it means
Blackwell	NVIDIA’s GPU architecture generation
B200	A Blackwell-based data-center GPU
GB200	A Grace Blackwell Superchip combining one Grace CPU with two Blackwell GPUs

The B200 is therefore a GPU, while the GB200 is a larger compute module built around Blackwell GPUs and an NVIDIA Grace CPU. NVIDIA specifies the GB200 Grace Blackwell Superchip as one Grace CPU connected to two Blackwell GPUs.

For a detailed explanation of the GB200 Superchip, its Grace CPU, NVLink-C2C connection, and NVL72 rack-scale configuration, see our NVIDIA GB200 AI Chip guide.

NVIDIA B200 Architecture at a Glance

The major architectural blocks of B200 can be summarized as follows:

Architectural Component	Role
Blackwell architecture	Overall GPU architecture
Two reticle-limited dies	Form a unified GPU
Streaming Multiprocessors	Execute parallel workloads
Fifth-generation Tensor Cores	Accelerate AI and matrix operations
Second-generation Transformer Engine	Optimizes AI precision and transformer workloads
HBM3e	Provides high-capacity, high-bandwidth GPU memory
Cache and shared memory	Improves data locality and reduces memory traffic
Tensor Memory	Supports efficient tensor data movement
Fifth-generation NVLink	Enables high-speed GPU-to-GPU communication
Decompression engine	Accelerates decompression of supported compressed data

Together, these components allow B200 to operate as a large-scale AI accelerator rather than relying only on raw GPU arithmetic throughput.

NVIDIA B200 Dual-Die Architecture

One of the defining features of Blackwell is its dual-die GPU design.

A traditional monolithic GPU is limited by the maximum practical reticle size available during semiconductor manufacturing. Blackwell addresses this limitation by using two large GPU dies connected through NVIDIA’s high-speed chip-to-chip interconnect.

NVIDIA states that Blackwell GPUs contain two reticle-limited dies connected by a 10 TB/s interconnect and presented to software as a unified GPU.

Conceptually, the architecture can be viewed as:

GPU Die 1 + GPU Die 2 → 10 TB/s chip-to-chip interconnect → Unified Blackwell GPU

Why Does the Dual-Die Design Matter?

The dual-die design allows NVIDIA to build a GPU with a very large amount of compute and memory capability while remaining within semiconductor reticle-size constraints.

The approach provides:

• A larger overall compute domain
• Very high die-to-die bandwidth
• A unified programming model
• High-speed communication between the two dies
• Greater scalability for AI workloads

The important point is that the two dies are not intended to behave like two independent GPUs. Blackwell is designed so that they function as a unified accelerator.

NVIDIA B200 Streaming Multiprocessor Architecture

The Streaming Multiprocessor (SM) is one of the fundamental compute building blocks of the Blackwell GPU. An SM executes groups of threads in parallel and contains the execution resources and local memory structures required for CUDA workloads.

At a high level, the Blackwell SM includes resources for:

• General-purpose CUDA computation
• Tensor operations
• Instruction scheduling
• Register storage
• Shared memory
• Load and store operations
• Specialized functions

The SM is particularly important for AI workloads because modern neural networks rely heavily on matrix operations that can be accelerated by Tensor Cores.

CUDA Cores

CUDA cores handle general-purpose arithmetic and parallel computation.

They are useful for:

• FP32 computation
• Integer operations
• Address calculations
• General CUDA kernels
• HPC workloads

While Tensor Cores are specialized for matrix operations, CUDA cores remain important for the surrounding operations required to execute complete AI and scientific workloads.

Warp Scheduling and Instruction Execution

CUDA workloads are organized into threads and warps. The SM scheduler selects ready warps and dispatches their instructions to the appropriate execution resources.

This allows the GPU to hide memory latency by switching between available workloads when one group of threads is waiting for data.

The result is high utilization of the GPU’s parallel execution resources.

Fifth-Generation Tensor Cores

Tensor Cores are central to the NVIDIA B200 architecture.

They are specialized hardware units designed to accelerate matrix operations that form the computational foundation of modern AI models.

B200’s fifth-generation Tensor Cores support multiple numerical formats used across AI training and inference, including FP8 and lower-precision formats introduced with the Blackwell generation. NVIDIA’s Blackwell architecture adds new precision and microscaling capabilities intended to improve the balance between AI performance, memory usage, and numerical accuracy.

Tensor Cores accelerate operations such as:

• Matrix multiplication
• Neural-network layers
• Transformer computation
• Attention-related operations
• AI inference
• AI training

Why Lower Precision Matters?

AI models do not always need every calculation to use the same numerical precision.

Using a lower precision can reduce the amount of data that must be stored and moved while increasing the amount of computation that can be performed per unit of hardware.

Blackwell extends this approach with support for very low precision AI computation, including FP4-oriented workflows through its Transformer Engine and related software stack.

This is particularly useful for inference, where maximizing throughput while maintaining acceptable model accuracy is critical.

Second-Generation Transformer Engine

Blackwell introduces a second-generation Transformer Engine.

The Transformer Engine works with the Tensor Cores and NVIDIA’s AI software stack to optimize the precision used by transformer-based models.

NVIDIA describes the second-generation Transformer Engine as combining Blackwell Tensor Core technology with software technologies such as TensorRT-LLM and NeMo to accelerate training and inference for large language models and mixture-of-experts models.

The architecture is designed to help AI systems use different numerical precisions where they are most effective rather than forcing an entire model to operate at one fixed precision.

This can improve:

• AI throughput
• Memory efficiency
• Energy efficiency
• Model scalability
• Inference performance

Tensor Memory and Data Movement

Blackwell also introduces specialized mechanisms for handling tensor data efficiently.

Tensor workloads frequently involve moving large quantities of intermediate data between registers, shared memory, cache, and external HBM.

Reducing unnecessary data movement is important because data movement can become a limiting factor even when compute resources are available.

Tensor Memory is designed to provide specialized storage and data movement capabilities for tensor operations, helping keep matrix-processing hardware supplied with data.

The broader architectural goal is:

Load data → reuse data efficiently → perform tensor computation → minimize unnecessary movement

This is particularly important for transformer workloads, where large amounts of data are repeatedly processed.

NVIDIA B200 Memory Architecture

Memory bandwidth is critical for modern AI accelerators because models can require the GPU to continuously move large quantities of weights, activations, and intermediate tensors.

NVIDIA B200 uses HBM3e memory.

NVIDIA’s current technical material specifies B200 configurations with 180 GB of HBM3e memory and up to 8 TB/s of memory bandwidth.

B200 Memory Specifications

Feature	NVIDIA B200
Memory type	HBM3e
Memory capacity	Up to 180 GB
Memory bandwidth	Up to 8 TB/s
Primary purpose	AI, inference, training, HPC

The large memory capacity allows B200 to keep substantial portions of large models and datasets close to the GPU compute resources.

The high bandwidth helps feed Tensor Cores and other execution units with data.

B200 Memory Hierarchy

GPU memory is organized into several levels.

A simplified view is:

HBM3e → L2 cache → L1/shared memory → registers → execution units

Each level serves a different purpose.

HBM3e

HBM3e provides the large-capacity external memory used for model weights, activations, datasets, and other GPU data.

It has much greater capacity than the small memories located directly inside the SM.

L2 Cache

The L2 cache provides a shared high-speed cache between the compute resources and HBM.

Frequently reused data can be served from cache instead of repeatedly accessing HBM.

L1 Cache and Shared Memory

The SM includes fast local memory resources used to improve data reuse between threads.

Shared memory is particularly useful for algorithms where multiple threads repeatedly access the same data.

Registers

Registers provide very fast storage for values being actively processed by individual threads.

Efficient register use is important because moving intermediate values unnecessarily through slower memory levels can reduce performance.

NVIDIA B200 Cache and Data Locality

AI workloads often perform repeated operations on the same data.

For example, a matrix tile may be loaded once and reused by multiple calculations.

The cache and shared-memory hierarchy is therefore designed to minimize unnecessary trips to HBM.

The general optimization strategy is:

Bring data closer to compute → reuse it as much as possible → write results back efficiently.

This becomes increasingly important as AI models become larger and computational workloads become more memory-intensive.

NVIDIA B200 NVLink Architecture

Large AI models are frequently distributed across multiple GPUs.

For this reason, GPU-to-GPU communication is a critical part of the Blackwell platform.

B200 supports fifth-generation NVLink.

NVIDIA’s current HGX B200 platform specifications list fifth-generation NVLink with up to 1.8 TB/s of GPU-to-GPU bandwidth per GPU.

NVLink allows multiple GPUs to exchange data at much higher bandwidth than conventional host-based communication paths.

This is important for:

• Distributed AI training
• Model parallelism
• Tensor parallelism
• Large-model inference
• Multi-GPU HPC workloads

NVLink and NVSwitch

In multi-GPU systems, NVIDIA NVSwitch technology can provide high-bandwidth communication between GPUs.

For example, NVIDIA’s HGX B200 platform uses eight B200 GPUs connected through fifth-generation NVLink and NVLink Switch technology.

The important distinction is that B200 is the GPU, while the complete multi-GPU platform adds NVLink and NVSwitch infrastructure around those GPUs.

B200 Chip-to-Chip Interconnect vs NVLink

These two technologies should not be confused.

Blackwell chip-to-chip interconnect

The 10 TB/s chip-to-chip connection joins the two dies inside the Blackwell GPU and allows them to operate as one unified GPU.

NVLink

NVLink connects GPUs and other components at the system level.

Therefore:

10 TB/s chip-to-chip interconnect → connects the two dies inside the GPU

NVLink → connects GPUs within a multi-GPU system

This distinction is important when discussing B200 architecture.

Hardware Decompression Engine

Blackwell introduces a hardware decompression engine designed to accelerate data processing.

Compressed data can create a bottleneck when CPUs must repeatedly decompress information before the GPU can process it.

A dedicated hardware engine can perform decompression closer to the data-processing pipeline.

NVIDIA has described Blackwell’s decompression engine as supporting formats including LZ4, Deflate, and Snappy, with benefits for data analytics and memory-bound workloads.

This can help reduce CPU involvement in certain data-processing workloads and improve the flow of data into GPU computation.

How NVIDIA B200 Architecture Work?

The complete B200 processing path can be simplified into several stages. Please refer to our in-depth article on how AI GPUs Actually Work.

Step 1: Data is loaded

Model weights, input data, and other required information are stored in HBM3e.

Step 2: Data moves through the memory hierarchy

Frequently used information can move through cache and shared-memory structures so that compute units can access it efficiently.

Step 3: The SM schedules work

The SM schedules threads and warps and sends instructions to the appropriate execution units.

Step 4: Tensor Cores perform AI computation

Matrix-heavy operations are processed by the fifth-generation Tensor Cores.

Step 5: Transformer Engine optimizes precision

The Transformer Engine helps select efficient numerical representations for supported AI workloads.

Step 6: Results are stored and reused

Intermediate results can remain close to the compute units for reuse before being written to larger memory.

Step 7: NVLink enables multi-GPU scaling

When a workload spans multiple GPUs, NVLink provides high-speed GPU-to-GPU communication.

This architecture allows B200 to combine compute, memory bandwidth, precision optimization, and interconnect performance into one AI acceleration platform.

B200 Precision and AI Optimization

Different AI workloads require different numerical formats.

Precision	Typical role
FP64	Scientific and HPC workloads
FP32	General-purpose numerical computation
FP16/BF16	AI training
FP8	Efficient AI training and inference
FP4	Very low-precision AI inference and supported workloads

Blackwell’s architecture is particularly focused on making lower-precision AI computation practical at large scale.

FP4 can reduce the amount of data required to represent values compared with higher-precision formats. When model accuracy can be maintained, this can increase throughput and reduce memory requirements.

The Transformer Engine and associated NVIDIA software are important because simply lowering precision is not enough; the system must manage precision and numerical accuracy appropriately.

NVIDIA B200 vs H100 Architecture

B200 succeeds NVIDIA’s Hopper-generation H100 and introduces several architectural changes.

Feature	NVIDIA B200	NVIDIA H100
Architecture	Blackwell	Hopper
GPU design	Two reticle-limited dies	Monolithic GPU die
Memory	Up to 180 GB HBM3e	Up to 80 GB HBM3
Memory bandwidth	Up to 8 TB/s	Up to 3.35 TB/s on SXM
Tensor Cores	5th generation	4th generation
Transformer Engine	2nd generation	1st generation
NVLink	5th generation	4th generation
Primary focus	Large-scale AI and accelerated computing	AI and accelerated computing

The most important architectural differences are not simply the headline performance numbers. Please refer to learn more about Clash of AI Titans: NVIDIA vs AMD.

B200 increases:

• GPU scale
• Memory capacity
• Memory bandwidth
• Tensor Core capability
• Precision options
• GPU-to-GPU interconnect bandwidth
• Transformer acceleration

These improvements make B200 particularly suited to large AI models and multi-GPU systems.

B200 vs H100: Which is Better for AI?

The answer depends on the workload and deployment environment.

H100 remains a highly capable Hopper-generation accelerator with a mature ecosystem and broad deployment.

B200 is designed for newer AI workloads that benefit from greater memory capacity, higher memory bandwidth, newer Tensor Cores, lower-precision computation, and fifth-generation NVLink.

For new large-scale AI infrastructure, B200 provides substantially more architectural headroom than H100.

B200 vs GB200: What is the Difference?

The B200 and GB200 are related, but they are not the same product.

NVIDIA B200

B200 is a Blackwell GPU.

It provides:

• GPU compute
• Tensor Cores
• HBM3e memory
• NVLink
• Blackwell architectural features

NVIDIA GB200

GB200 is a Grace Blackwell Superchip.

It combines:

• One NVIDIA Grace CPU
• Two NVIDIA Blackwell GPUs
• NVLink-C2C connectivity

NVIDIA specifies the GB200 Grace Blackwell Superchip as one Grace CPU and two Blackwell GPUs.

GB200 NVL72

At the next level, GB200 NVL72 combines 36 Grace CPUs and 72 Blackwell GPUs in a rack-scale system.

Therefore:

B200 = GPU

GB200 = Grace CPU + 2 Blackwell GPUs

GB200 NVL72 = rack-scale system containing 72 Blackwell GPUs and 36 Grace CPUs

For the complete GB200 architecture, NVLink-C2C implementation, NVL72 design, and rack-scale system details, see our NVIDIA GB200 AI Chip article.

B200 Applications

The B200 architecture is designed for workloads where high compute throughput, memory bandwidth, and multi-GPU scaling are important.

Large Language Models

LLMs require enormous amounts of matrix computation and memory movement.

B200’s Tensor Cores, HBM3e memory, Transformer Engine, and multi-GPU interconnect capabilities are designed for these workloads.

Generative AI

Generative AI models can involve large neural networks and demanding inference workloads.

Lower-precision computation can improve throughput and memory efficiency when supported by the model and software stack.

AI Inference

Inference often benefits from high throughput and efficient memory usage.

Blackwell’s precision capabilities and Transformer Engine are designed to improve the efficiency of supported inference workloads.

High-Performance Computing

B200 can also accelerate scientific and engineering workloads that benefit from GPU parallelism.

Examples include:

• Scientific simulations
• Computational physics
• Climate modeling
• Engineering simulation
• Data analytics

Multi-GPU AI Systems

Large AI models often exceed the memory capacity or compute capability of a single GPU.

B200 can therefore be deployed in multi-GPU platforms where NVLink and NVSwitch provide high-speed communication between GPUs.

NVIDIA B200 Architectural Innovations

The most important architectural innovations in B200 can be summarized as:

1. Dual-die unified GPU design
   Two reticle-limited dies operate as a unified Blackwell GPU through a 10 TB/s chip-to-chip interconnect.
2. Fifth-generation Tensor Cores
   Specialized matrix-processing hardware accelerates AI workloads.
3. Second-generation Transformer Engine
   Helps optimize transformer training and inference across different numerical precisions.
4. HBM3e memory
   B200 provides up to 180 GB of HBM3e memory and up to 8 TB/s of memory bandwidth.
5. Fifth-generation NVLink
   Provides high-bandwidth communication for multi-GPU systems.
6. Tensor Memory and improved data movement
   Helps optimize the movement and reuse of tensor data.
7. Hardware decompression
   Accelerates supported compressed-data workloads and reduces data-processing bottlenecks.

Frequently Asked Questions about NVIDIA B200

What is NVIDIA B200?

NVIDIA B200 is a Blackwell-generation data-center GPU designed for AI training, inference, HPC, and other accelerated-computing workloads.

What architecture does NVIDIA B200 use?

B200 uses the NVIDIA Blackwell architecture. Blackwell GPUs use two reticle-limited dies connected through a 10 TB/s chip-to-chip interconnect that operates as a unified GPU.

How much memory does NVIDIA B200 have?

NVIDIA’s current technical material specifies the B200 with up to 180 GB of HBM3e memory and up to 8 TB/s of memory bandwidth.

What makes the B200 different from H100?

B200 belongs to the newer Blackwell generation and adds a dual-die architecture, fifth-generation Tensor Cores, second-generation Transformer Engine, HBM3e memory, and fifth-generation NVLink.

What is the difference between B200 and GB200?

B200 is a Blackwell GPU. GB200 is a Grace Blackwell Superchip that combines one Grace CPU with two Blackwell GPUs.

Does B200 use HBM3e?

Yes. NVIDIA’s B200 technical documentation identifies HBM3e as the GPU’s high-bandwidth memory technology, with up to 180 GB capacity and up to 8 TB/s bandwidth.

What is NVLink used for in B200 systems?

NVLink provides high-bandwidth communication between GPUs in multi-GPU systems. NVIDIA’s HGX B200 platform uses fifth-generation NVLink and NVLink Switch technology to connect eight B200 GPUs.

What is the 10 TB/s connection in Blackwell?

The 10 TB/s connection is the chip-to-chip interconnect that connects Blackwell’s two reticle-limited GPU dies so they can operate as a unified GPU. It is different from the NVLink connections used for system-level GPU communication.

Is B200 designed for gaming?

No. B200 is a data-center GPU designed primarily for AI, HPC, inference, and accelerated computing rather than consumer gaming.

What is the role of the Transformer Engine?

The second-generation Transformer Engine works with Blackwell Tensor Cores and NVIDIA’s AI software stack to optimize transformer workloads and support efficient precision scaling.

Is B200 used inside GB200?

Yes. NVIDIA’s GB200 Grace Blackwell Superchip combines one Grace CPU with two Blackwell GPUs, including B200-class GPUs.

Conclusion

The NVIDIA B200 architecture represents a major evolution in data-center GPU design.

Instead of relying solely on a larger monolithic GPU, Blackwell combines two reticle-limited dies into a unified accelerator through a high-speed chip-to-chip interconnect. Around this foundation, B200 combines fifth-generation Tensor Cores, a second-generation Transformer Engine, HBM3e memory, advanced data-movement capabilities, and fifth-generation NVLink.

The result is an architecture designed specifically for the demands of modern AI: large models, high-bandwidth memory access, low-precision computation, transformer workloads, and multi-GPU scaling.

The distinction between B200 and GB200 is also important. B200 is the GPU; GB200 combines Blackwell GPUs with an NVIDIA Grace CPU; and GB200 NVL72 scales those components into a rack-scale AI system.

For readers interested specifically in how Grace and B200 GPUs are combined into the GB200 Superchip and scaled into NVL72, see our NVIDIA GB200 AI Chip guide.