GIGABYTE AMD Epyc 7003 Series Processors User Manual

: June 6, 2024
: Gigabyte

Table of Contents

GIGABYTE AMD Epyc 7003 Series Processors
Introduction
Memory Bandwidth
Choosing the Right Configuration
Instance to Package Channel Mapping on an Ethanol-X board
References
Read User Manual Online (PDF format)
Download This Manual (PDF format)

GIGABYTE AMD Epyc 7003 Series Processors

Introduction

The introduction of AMD’s newest EPYC™ newest processor, the EPYC 7003 generation is designed to build on last generation’s industry-leading eight channels of DDR4 memory EPYC 7002 generation processor. While the EPYC 7003 brings additional performance capability, the memory subsystem is similar to EPYC 7002 apart from a new, 6-way interleave mode. EPYC 7003 processors have been designed to be a drop-in upgrade to EPYC 7002 systems with type 1, enhanced motherboards. However, systems with type 0 motherboards cannot be upgraded to use EPYC 7003. Supported DIMM types include registered DIMMs (RDIMMs) built with x4 and x8 devices, load-reduced DIMMs (LRDIMMs) built with dual die and stacked packages, threedimensional stacked DIMMs (3DS DIMMs), and non-volatile DIMMs (NVDIMMs, type N only).

Supported Standard DIMM Types on AMD EPYC Processors

DIMM Type	Ranks	Capacity **
RDIMM	1 (SR)	8 GB, 16 GB or 32 GB
RDIMM	2 (DR)	16 GB, 32 GB or 64 GB
LRDIMM	4	64 GB or 128 GB
LRDIMM	8	128 GB or 256 GB
3DS	4	64 GB or 128 GB
3DS	8	128 GB or 256 GB

This table represents a listing of DIMMs available on AMD EPYC processors at the time of writing. While the AMD EPYC 7003 family of processors is compatible with the listed DIMM configurations, you should consult with your platform vendor for a list of supported DIMMs.
See Appendix A “DIMM Population Rules” for guidance on mixing different DIMMs

Memory Bandwidth

EPYC 7003 processors have eight memory channels designated A, B, C, D, E, F, G, and H. Each channel supports up to two DIMMs. Systems can be built with one DIMM per channel (1 DPC), two DIMMs per channel (2 DPC), or a combination thereof.
The operating speed of the memory will depend on the number and types of DIMMs in the system. For EPYC 7002 processors, there were two types of motherboards defined: Motherboard Type-0 and Motherboard Type-1. While Type-0 motherboards cannot be upgraded to use EPYC 7003, Type-1 motherboards can be upgraded with no degradation of EPYC 7003 memory and I/O capability.
While a decreased operational frequency with two DIMMs populated may not seem ideal for memory-intensive workloads, the additional chip selects being used, or ranks of memory can outweigh the change in operating memory speed in certain workloads.

EPYC Memory Speed based on DIMM Population (One DIMM per Channel)

DIMM

Type

| DIMM Population| Max EPYC 7003

DDR Frequency (MHz)

---|---|---
DIMM 0

RDIMM

| 1R (1 Rank)| 3200
2R or 2DR (2 Ranks)| 3200

LRDIMM

| 4DR (4 Ranks)| 3200
2S2R (4 Ranks)| 3200
2S4R (8 Ranks)| 3200

3DS

| 2S2R (4 Ranks)| 3200
2S4R (8 Ranks)| 3200

EPYC Memory Speed based on DIMM Population (Two DIMM per Channel)

DIMM

Type

| DIMM Population| Max EPYC 7003

DDR Frequency (MHz)

---|---|---
DIMM 0| DIMM 1

RDIMM

| —| 1R| 3200
1R| 1R| 2933
—| 2R or 2DR| 3200
1R| 2R or 2DR| 2933
2R or 2DR| 2R or 2DR| 2933

LRDIMM

| —| 4DR| 3200
4DR| 4DR| 2933
—| 2S2R (4 Ranks)| 3200
—| 2S4R (8 Ranks)| 3200
2S2R (4 Ranks)| 2S2R (4 Ranks)| 2933
2S4R (8 Ranks)| 2S4R (8 Ranks)| 2933

3DS

Choosing the Right Configuration

AMD recommends that all eight memory channels per CPU socket be populated with all channels having equal capacity. This enables the memory subsystem to operate in eight-way interleaving mode, which should provide the best performance in most cases.
For a given processor model number, memory population, and NUMA node per socket (NPS) configuration, the pre-BIOS firmware chooses the optimal memory interleaving option. There are three NPS options available: NPS=1, NPS=2, and NPS=4. The following diagrams are examples of supported DIMM configurations in a system. Notice, however, that most configurations populating fewer than eight channels are supported, but not recommended. For a full list of population rules, consult Appendix A: DIMM Population Rules.

One DIMM Configuration (Not Recommended)
Interleave: None (NPS=1, 2 or 4) All DIMM has the same capacity

Two DIMM Configuration (Not Recommended)

Four DIMM Configuration (Conditionally recommended only with EPYC processors that have 128MB L3 or less Note1)

Six DIMM Configuration (Conditionally recommended if only 6 channels can be populated Note2)

Eight DIMM Configuration (Recommended)

Ten DIMM Configuration (Recommended)

Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type

Twelve DIMM Configuration (Recommended)

Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type

Fourteen DIMM Configuration (Recommended)

Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type

Sixteen DIMM Configuration (Recommended)

Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) All DIMM have the same capacity Two DIMMs occupying the same channel are of the same type

Appendix A: DIMM Population Rules

Supported device technology:
DDR4 8 Gbit, 16 Gbit.
Supported DIMM typesNote1:
1R and 2R RDIMM, built with x4 and x8 DDR4 devices.
4R and 8R LRDIMM, built with x4 DDR4 devices (4DR, 2S2R, 2S4R devices).
2S2R (= 4R) and 2S4R (= 8R) 3DS built with x4 DDR4 devices.
NVDIMMs, type N only.
Not supported
UDIMMs, 4R RDIMMs.
General population order guidelines
Populate open channels before populating two DIMMs on a given channel
In 2 DPC configurations where only one DIMM is populated on a channel, populate the DIMM socket physically farthest away from the processor
Balance memory capacity per channel pair on a given CPU
Balance memory capacity per CPU socket in a two-socket system
Though a system can be populated with a single DIMM as a minimum configuration, full memory bandwidth requires one DIMM per channel (A-H) to be populated
For best performance, AMD recommends populating all eight memory channels per socket, with every channel having the same capacity
If a customer chooses to populate only four channels in the system, AMD recommends limiting the processors in that system to those with 128MB or less of L3 and populating channels CDGH identically. This will enable four-way interleaving, which will generally provide the best performance with a four DIMM population
If a customer chooses to populate only six channels, AMD recommends populating channels ACDEGH with equal capacity, where all channels are less than 256GB each. This will enable six-way interleaving, which will generally provide the best performance with a six-channel population. Note that the memory interleaving size must be set to 2kB or 4kB for a six-way interleaving operation.
General population rules
RDIMM, LRDIMM, 3DS, and NVDIMM-N DIMM types can coexist in the system with restrictions.
DIMMs within the same channel must be of the same base DIMM module type (all RDIMM, LRDIMM, or 3DS). NVDIMM-N can be mixed with RDIMM
DIMMs within the same channel must be of the same DRAM width. No mixing DIMMs with x4 and x8 DIMMs on a channel.
DIMMs within the same channel must be of the same DRAM density. No mixing DIMMs with 8Gb and 16Gb DRAMs on a channel.
While DIMMs with different manufacturers can be populated on the same channel, platform developers are expected to validate these configurations.
General interleaving rules
Memory channels may be organized in four pairs: A+B, C+D, E+F, and G+H. These pairs may be interleaved together (two-way interleaving), a specific set of two pairs may be interleaved together (four-way interleaving like ABCD, CDGH, or EFGH), six specific channels may be interleaved together (six-way interleaving, ACDEGH), or all channels may be interleaved together (ABCDEFGH).
Eight-way interleaving is only possible when all eight channels are populated with equal size memory and NPS=1.
Six-way interleaving is only possible when channels ACDEGH are populated with equal size memory, all channels are <= 256GB, no other channels are populated, memory interleaving size is set to 2kB or 4kB, and NPS=1.
Four-way interleaving when NPS=1 is only possible when channels ABCD, EFGH, or CDGH are populated with equal size memory and no other channels are populated.
Four-way interleaving when NPS=2 is only possible when channels ABCD, EFGH, or CDGH are populated with equal size memory and no other channels are populated. It is possible to have
four-way interleaves of ABCD and EFGH simultaneously if channels ABCD are all equal size memory, and channels EFGH are all equal size memory. The memory sizes of ABCD and EFGH are independent of each other in this case and can be different.
Two-way interleaving is only possible for channels AB, CD, EF, and GH for all NPS values. For an optimized configuration, each channel should have identical capacity. While supported, unequal capacity configurations are suboptimal because the larger DIMM’s excess capacity will not be interleaved, resulting in only the matched memory capacity between the DIMMs being interleaved.
The resulting interleave is based on the processor model number, how the DIMMs are populated in the system, and the chosen NPS value. The highest allowable interleave is chosen by firmware for a given NUMA domain.
Volatile (standard DIMMs) and non-volatile (NVDIMM-N) memory types cannot be interleaved. If a channel is shared between a standard DIMM and an NVDIMM-N, that channel cannot be interleaved.
NVDIMM-N population rules
Contact your AMD representative for the BIOS version required for NVDIMM-N enablement and general support
All DIMM population rules in Appendix A still apply except where noted in this section
There must be at least one channel populated without NVDIMM-N in the system. A system with 100%NVDIMM-N is not supported.
For a given configuration, the populated channel with the lowest numbered UMC on the boot processor must contain only standard DIMMs.
For a single processor example, if only channels E and F are populated on the boot processor, channel E must be populated with only standard DIMMs.
For a multiprocessor example, if both processors have only channel B populated then the boot processor must be populated with only standard DIMMs.