Lenovo Mellanox ConnectX-6 HDR/200GbE VPI Adapters User Guide

: October 27, 2023
: Lenovo

Table of Contents

Lenovo Mellanox ConnectX-6 HDR/200GbE VPI Adapters
ThinkSystem Mellanox ConnectX-6 HDR/200GbE VPI Adapters
Supported transceivers and cables
Features
Technical specifications
Management and Control
NVIDIA Unified Fabric Manager
- Server support
Regulatory approvals
Warranty
Trademarks
References
Read User Manual Online (PDF format)
Download This Manual (PDF format)

Lenovo Mellanox ConnectX-6 HDR/200GbE VPI Adapters

Lenovo-Mellanox-ConnectX-6-HDR-200GbE-VPI-Adapters-product-
image

ThinkSystem Mellanox ConnectX-6 HDR/200GbE VPI Adapters

Product Guide
The ThinkSystem Mellanox ConnectX-6 HDR/200GbE VPI Adapters offer 200 Gb/s Ethernet and InfiniBand connectivity for high-performance connectivity when running HPC, cloud, storage and machine learning applications.
The following figure shows the ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter connected to the ThinkSystem Mellanox HDR/200GbE Aux Adapter (the standard heat sink has been removed in this photo).

Figure 1. ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter (right) and ThinkSystem Mellanox HDR/200GbE Aux Adapter (left)

Did you know?
Mellanox ConnectX-6 brings new acceleration engines for maximizing High Performance, Machine Learning, Storage, Web 2.0, Cloud, Data Analytics and Telecommunications platforms. ConnectX-6 HDR adapters support up to 200 Gb/s total bandwidth at sub-600 nanosecond latency, and NVMe over Fabric offloads, providing the highest performance and most flexible solution for the most demanding applications and markets. ThinkSystem servers with Mellanox adapters and switches deliver the most intelligent fabrics for High Performance Computing clusters.
ThinkSystem Mellanox ConnectX-6 HDR/200GbE VPI Adapters

CTO orders: For configure-to-order builds, these adapters are only available when you select one of the HPC & AI modes in the DCSC configurator. Not available in General Purpose mode of DCSC.

Table 1. Ordering information

Part number| Feature code| Mellanox equivalent| ****

Description

---|---|---|---
Primary adapter
4C57A15326| B4RC / BN38| MCX653105A- HDAT| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter
CTO only| B4RG| None| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter (SharedIO) DWC
CTO only| B951| None| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe 4 VPI Adapter (SharedIO) DWC
CTO only| B952| None| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe 4 VPI Adapter DWC
Shared I/O auxiliary adapter
4C57A14179| B4RB| MTMK0012| ThinkSystem Mellanox HDR/200GbE 2x PCIe Aux Kit

Part number 4C57A15326 includes the following:

One Mellanox adapter with full-height (3U) adapter bracket attached
Low-profile (2U) adapter bracket
Documentation

Part number 4C57A14179 includes the following:

One Mellanox adapter with full-height (3U) adapter bracket attached
Low-profile (2U) adapter bracket
350mm cable
Documentation

Note: 4C57A15326 was previously named ThinkSystem Mellanox ConnectX-6 HDR QSFP56 1-port PCIe 4 InfiniBand Adapter

Supported transceivers and cables

The adapter has an empty QSFP56 cage for connectivity. The following table lists the supported transceivers.
Table 2. Transceivers

Part number	Feature code	Description

100Gb transceivers
7G17A03539*| AV1D| Lenovo 100GBase-SR4 QSFP28 Transceiver
4M27A67042| BFH1| Lenovo 100Gb SR4 QSFP28 Ethernet Transceiver
Converters
4G17A10853| B306| Mellanox QSA 100G to 25G Cable Adapter

7G17A03539 also supports 40Gb when installed in a Mellanox adapter.
The following table lists the supported fiber optic cables and Active Optical Cables.

Table 3. Optical cables

Part number	Feature code	Description

QSFP28 EDR InfiniBand Optical Cables
00MP563| ASRN| 3m Mellanox EDR IB Optical QSFP28 Cable
00MP540| ASQZ| 5m Mellanox EDR IB Optical QSFP28 Cable
00MP544| ASR0| 10m Mellanox EDR IB Optical QSFP28 Cable
00MP548| ASR1| 15m Mellanox EDR IB Optical QSFP28 Cable
00MP552| ASR2| 20m Mellanox EDR IB Optical QSFP28 Cable
00MP556| ASR3| 30m Mellanox EDR IB Optical QSFP28 Cable
00MP566| ASRP| 50m Mellanox EDR IB Optical QSFP28 Cable
QSFP 40Gb Active Optical Cables
7Z57A04256| AX42| Lenovo 1m 40G QSFP+ Active Optical Cable
00YL652| ATZ3| Lenovo 3m 40G QSFP+ to QSFP+ Active Optical Cable
00YL655| ATZ4| Lenovo 5m 40G QSFP+ to QSFP+ Active Optical Cable
00YL658| ATZ5| Lenovo 7m 40G QSFP+ to QSFP+ Active Optical Cable
00YL661| ATZ6| Lenovo 15m 40G QSFP+ to QSFP+ Active Optical Cable
00YL664| ATZ7| Lenovo 20m 40G QSFP+ to QSFP+ Active Optical Cable
QSFP 40Gb Optical Cables (these cables require a transceiver)
00VX003| AT2U| Lenovo 10m QSFP+ MPO-MPO OM3 MMF Cable
00VX005| AT2V| Lenovo 30m QSFP+ MPO-MPO OM3 MMF Cable
QSFP28 100Gb Ethernet Active Optical Cables
4Z57A10844| B2UZ| Lenovo 1m 100G QSFP28 Active Optical Cable
7Z57A03546| AV1L| Lenovo 3m 100G QSFP28 Active Optical Cable
7Z57A03547| AV1M| Lenovo 5m 100G QSFP28 Active Optical Cable
7Z57A03548| AV1N| Lenovo 10m 100G QSFP28 Active Optical Cable
7Z57A03549| AV1P| Lenovo 15m 100G QSFP28 Active Optical Cable
7Z57A03550| AV1Q| Lenovo 20m 100G QSFP28 Active Optical Cable
100G MPO OM4 MMF Cables (these cables require a transceiver)
Part number| Feature code| Description
---|---|---
7Z57A03567| AV25| Lenovo 5m MPO-MPO OM4 MMF Cable
7Z57A03568| AV26| Lenovo 7m MPO-MPO OM4 MMF Cable
7Z57A03569| AV27| Lenovo 10m MPO-MPO OM4 MMF Cable
7Z57A03570| AV28| Lenovo 15m MPO-MPO OM4 MMF Cable
7Z57A03571| AV29| Lenovo 20m MPO-MPO OM4 MMF Cable
7Z57A03572| AV2A| Lenovo 30m MPO-MPO OM4 MMF Cable
QSFP56 HDR IB Optical Cables
4Z57A14188| B4QW| 3m Mellanox HDR IB Optical QSFP56 Cable
4Z57A14189| B4QX| 5m Mellanox HDR IB Optical QSFP56 Cable
4Z57A14190| B4QY| 10m Mellanox HDR IB Optical QSFP56 Cable
4Z57A14191| B4QZ| 15m Mellanox HDR IB Optical QSFP56 Cable
4Z57A14192| B4R0| 20m Mellanox HDR IB Optical QSFP56 Cable
4Z57A16016| B68P| 30m Mellanox HDR IB Active Optical QSFP56 Cable
4Z57A16017| B68N| 50m Mellanox HDR IB Active Optical QSFP56 Cable
4Z57A16018| B68M| 100m Mellanox HDR IB Active Optical QSFP56 Cable
4Z57A72553| BFXR| 3m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72554| BFXS| 5m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72555| BFXT| 10m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72556| BFXU| 15m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72557| BFXV| 20m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72558| BFXW| 30m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72559| BFXX| 50m Mellanox HDR IB Optical QSFP56 Low Latency Cable
4Z57A72560| BFXY| 100m Mellanox HDR IB Optical QSFP56 Low Latency Cable
QSFP56 HDR IB to 2x HDR100 Optical Splitter Cables
4Z57A14196| B4R4| 3m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Cable
4Z57A14197| B4R5| 5m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Cable
4Z57A14198| B4R6| 10m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Cable
4Z57A14199| B4R7| 15m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Cable
4Z57A14214| B4R8| 20m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Cable
4Z57A11490| B68K| 30m Mellanox HDR to 2x HDR100 IB Active Optical QSFP56 Splitter Cable
4Z57A72561| BFXZ| 3m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable
4Z57A72562| BFY0| 5m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable
4Z57A72563| BFY1| 10m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable
4Z57A72564| BFY2| 15m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable
4Z57A72565| BFY3| 20m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable
4Z57A72566| BFY4| 30m Mellanox HDR IB to 2x HDR100 Splitter Optical QSFP56 Low Latency Cable

The following table lists the supported direct-attach copper (DAC) cables.

Table 4. Copper cables

Part number	Feature code	Description

Features

Machine learning and big data environments
Data analytics has become an essential function within many enterprise data centers, clouds and hyperscale platforms. Machine learning relies on especially high throughput and low latency to train deep neural networks and to improve recognition and classification accuracy. ConnectX-6 offers an excellent solution to provide machine learning applications with the levels of performance and scalability that they require.
ConnectX-6 utilizes the RDMA technology to deliver low-latency and high performance. ConnectX-6 enhances RDMA network capabilities even further by delivering end-to-end packet level flow control.

Security

ConnectX-6 block-level encryption offers a critical innovation to network security. As data in transit is stored or retrieved, it undergoes encryption and decryption. The ConnectX-6 hardware offloads the IEEE AES-XTS encryption/decryption from the CPU, saving latency and CPU utilization. It also guarantees protection for users sharing the same resources through the use of dedicated encryption keys.
By performing block-storage encryption in the adapter, ConnectX-6 excludes the need for self-encrypted disks. This allows customers the freedom to choose their preferred storage device, including those that traditionally do not provide encryption. ConnectX-6 can support Federal Information Processing Standards (FIPS) compliance.
ConnectX-6 also includes a hardware Root-of-Trust (RoT), which uses HMAC relying on a device-unique key. This provides both a secure boot as well as cloning-protection. Delivering best-in-class device and firmware protection, ConnectX-6 also provides secured debugging capabilities, without the need for physical access.

Storage environments

NVMe storage devices offer very fast access to storage media. The evolving NVMe over Fabric (NVMe-oF) protocol leverages RDMA connectivity to remotely access NVMe storage devices efficiently, while keeping the end-to-end NVMe model at lowest latency. With its NVMe-oF target and initiator offloads, ConnectX-6 brings further optimization to NVMe-oF, enhancing CPU utilization and scalability.

Cloud and Web 2.0 environments

Telco, Cloud and Web 2.0 customers developing their platforms on software- defined network (SDN) environments are leveraging the Virtual Switching capabilities of server operating systems to enable maximum flexibility in the management and routing protocols of their networks.
Open V-Switch (OVS) is an example of a virtual switch that allows virtual machines to communicate among themselves and with the outside world. Software- based virtual switches, traditionally residing in the hypervisor, are CPU intensive, affecting system performance and preventing full utilization of available CPU for compute functions.
To address such performance issues, ConnectX-6 offers Mellanox Accelerated Switching and Packet
Processing (ASAP ) Direct technology. ASAP offloads the vSwitch/vRouter by handling the data plane in the NIC hardware while maintaining the control plane unmodified. As a result, significantly higher
vSwitch/vRouter performance is achieved minus the associated CPU load.
The vSwitch/vRouter offload functions supported by ConnectX-5 and ConnectX-6 include encapsulation and de-capsulation of overlay network headers, as well as stateless offloads of inner packets, packet headers re-write (enabling NAT functionality), hairpin, and more.
In addition, ConnectX-6 offers intelligent flexible pipeline capabilities, including programmable flexible parser and flexible match-action tables, which enable hardware offloads for future protocols.

Socket Direct

Mellanox’s Socket Direct technology improves the performance of dual-socket servers in numerous ways, such as by enabling each of their CPUs to access the network through a dedicated PCIe interface. As the connection from each CPU to the network bypasses the QPI (UPI) and the second CPU, Socket Direct reduces latency and CPU utilization. Moreover, each CPU handles only its own traffic (and not that of the second CPU), thus optimizing CPU utilization even further.
Socket Direct also enables GPUDirect RDMA for all CPU/GPU pairs by ensuring that GPUs are linked to the CPUs closest to the adapter card. Socket Direct enables Intel DDIO optimization on both sockets by creating a direct connection between the sockets and the adapter card.
Socket Direct technology is enabled by a main card housing the ConnectX-6 and an auxiliary PCIe card bringing in the remaining PCIe lanes. The ConnectX-6 Socket Direct card is installed into two PCIe x16 slots and connected using the supplied 350mm cable.
The two PCIe x16 slots may also be connected to the same CPU. In this case, the main advantage of the technology lies in delivering 200Gb/s to servers with PCIe Gen3-only support.

SharedIO
An implementation of Sockets Direct is SharedIO (Shared I/O), where a Mellanox VPI adapter is installed in one slot in one server and an auxiliary adapter is installed in a slot in second server in the same enclosure.
The result is that the two servers or processors share the network connection of the VPI adapter with significant savings both in the cost of the adapters but also the cost of switch ports.
The following figure shows the Mellanox SharedIO Adapter and Auxiliary Card installed in two ThinkSystem SD650 V2 servers in the same tray.

Figure 2. SharedIO adapters installed in the two SD650 V2 servers on a tray

Technical specifications

The adapters have the following technical specifications.

Form factor

Single-slot low-profile main adapter (6.6-inch x 2.71 in.)
Single-slot low-profile auxiliary adapter

PCI Express Interface

Supports PCIe 4.0 or PCIe 3.0
- In PCIe 4.0 servers, the ConnectX-6 adapter is used by itself to connect 16 PCIe lanes. For two-socket servers, the adapter can be used with the Aux adapter to enable the Socket
  Direct feature.
- In PCIe 3.0 servers, the ConnectX-6 adapter is used with the Aux adapter to connect 32 PCIe lanes. For two-socket servers, the adapter can be used with the Aux adapter to enable the Socket Direct feature.
PCIe Atomic
TLP (Transaction Layer Packet) Processing Hints (TPH)
PCIe switch Downstream Port Containment (DPC) enablement for PCIe hot-plug
Advanced Error Reporting (AER)
Access Control Service (ACS) for peer-to-peer secure communication
Process Address Space ID (PASID) Address Translation Services (ATS)
IBM CAPIv2 (Coherent Accelerator Processor Interface)
Support for MSI/MSI-X mechanisms

Connectivity

One QSFP56 port
Supports passive copper cables with ESD protection
Powered connectors for optical and active cable support

InfiniBand

Supports interoperability with InfiniBand switches (up to HDR, as 4 lanes of 50Gb/s data rate)
Total connectivity is 200 Gb/s:
- One port adapter supports a single 200 Gb/s link
HDR / HDR100 / EDR / FDR / QDR / DDR / SDR
IBTA Specification 1.3 compliant
RDMA, Send/Receive semantics
Hardware-based congestion control
Atomic operations
16 million I/O channels
256 to 4Kbyte MTU, 2Gbyte messages
8 virtual lanes + VL15

Ethernet (requires firmware 20.28.1002 or later)

Support interoperability with Ethernet switches (up to 200GbE, as 4 lanes of 50Gb/s data rate)
Total connectivity is 200 Gb/s:
- One port adapter supports a single 200 Gb/s link
Supports 200 GbE / 100GbE / 50GbE / 40GbE / 25GbE / 10GbE / 1GbE
IEEE 802.3bj, 802.3bm 100 Gigabit Ethernet
IEEE 802.3by, Ethernet Consortium 25, 50 Gigabit Ethernet, supporting all FEC modes
IEEE 802.3ba 40 Gigabit Ethernet
IEEE 802.3ae 10 Gigabit Ethernet
IEEE 802.3az Energy Efficient Ethernet
IEEE 802.3ap based auto-negotiation and KR startup
IEEE 802.3ad, 802.1AX Link Aggregation
IEEE 802.1Q, 802.1P VLAN tags and priority
IEEE 802.1Qau (QCN) – Congestion Notification
IEEE 802.1Qaz (ETS)
IEEE 802.1Qbb (PFC)
IEEE 802.1Qbg
IEEE 1588v2
Jumbo frame support (9.6KB)
IPv4 (RFQ 791)
IPv6 (RFC 2460)

Enhanced Features

Hardware-based reliable transport
Collective operations offloads
Vector collective operations offloads
PeerDirect RDMA (GPUDirect) communication acceleration
64/66 encoding
Enhanced Atomic operations
Advanced memory mapping support, allowing user mode registration and remapping of memory (UMR)
Extended Reliable Connected transport (XRC)
Dynamically Connected transport (DCT)
On demand paging (ODP)
MPI Tag Matching
Rendezvous protocol offload
Out-of-order RDMA supporting Adaptive Routing
Burst buffer offload
In-Network Memory registration-free RDMA memory access

CPU Offloads

RDMA over Converged Ethernet (RoCE)
TCP/UDP/IP stateless offload
LSO, LRO, checksum offload
RSS (also on encapsulated packet), TSS, HDS, VLAN and MPLS tag insertion/stripping, Receive flow steering
Data Plane Development Kit (DPDK) for kernel bypass applications
Open VSwitch (OVS) offload using ASAP 2
- Flexible match-action flow tables
- Tunneling encapsulation / de-capsulation
Intelligent interrupt coalescence
Header rewrite supporting hardware offload of NAT router

Storage Offloads

Block-level encryption: XTS-AES 256/512 bit key
NVMe over Fabric offloads for target machine
Erasure Coding offload – offloading Reed-Solomon calculations
T10 DIF – signature handover operation at wire speed, for ingress and egress traffic
Storage Protocols: SRP, iSER, NFS RDMA, SMB Direct, NVMe-oF

Overlay Networks

RoCE over overlay networks
Stateless offloads for overlay network tunneling protocols
Hardware offload of encapsulation and decapsulation of VXLAN, NVGRE, and GENEVE overlay networks

Hardware-Based I/O Virtualization

Single Root IOV
Address translation and protection
VMware NetQueue support
SR-IOV: Up to 512 Virtual Functions
SR-IOV: Up to 16 Physical Functions per host Virtualization hierarchies (network partitioning, NPAR)
- Virtualizing Physical Functions on a physical port
- SR-IOV on every Physical Function
Configurable and user-programmable QoS
Guaranteed QoS for VMs

HPC Software Libraries
HPC-X, OpenMPI, MVAPICH, MPICH, OpenSHMEM, PGAS and varied commercial packages

Management and Control

NC-SI, MCTP over SMBus and MCTP over PCIe – BMC interface
PLDM for Monitor and Control DSP0248
PLDM for Firmware Update DSP0267
SDN management interface for managing the eSwitch
I2C interface for device control and configuration
General Purpose I/O pins
SPI interface to Flash
JTAG IEEE 1149.1 and IEEE 1149.6

Remote Boot

Remote boot over InfiniBand
Remote boot over Ethernet
Remote boot over iSCSI
Unified Extensible Firmware Interface (UEFI)
Pre-execution Environment (PXE)

NVIDIA Unified Fabric Manager

NVIDIA Unified Fabric Manager (UFM) is InfiniBand networking management software that combines enhanced, real-time network telemetry with fabric visibility and control to support scale-out InfiniBand data centers.

The two offerings available from Lenovo are as follows:

UFM Telemetry for Real-Time Monitoring
The UFM Telemetry platform provides network validation tools to monitor network performance and conditions, capturing and streaming rich real-time network telemetry information, application workload usage, and system configuration to an on-premises or cloud-based database for further analysis.
UFM Enterprise for Fabric Visibility and Control
The UFM Enterprise platform combines the benefits of UFM Telemetry with enhanced network monitoring and management. It performs automated network discovery and
provisioning, traffic monitoring, and congestion discovery. It also enables job schedule provisioning and integrates with industry-leading job schedulers and cloud and cluster managers, including Slurm and Platform Load Sharing Facility (LSF).

The following table lists the subscription licenses available from Lenovo.

Table 5. NVIDIA Unified Fabric Manager subscriptions

Part number	Feature code (7S02CTO1WW)	Description

UFM Telemetry
7S02003HWW| S88D| UFM Telemetry 1-year License and Gold-Support for Lenovo clusters. Per node.
7S02003JWW| S88E| UFM Telemetry 3-year License and Gold-Support for Lenovo clusters. Per node.
7S02003KWW| S88F| UFM Telemetry 5-year License and Gold-Support for Lenovo clusters. Per node.
UFM Enterprise
7S02003LWW| S88G| UFM Enterprise 1-year License and Gold-Support for Lenovo clusters. Per node.
7S02003MWW| S88H| UFM Enterprise 3-year License and Gold-Support for Lenovo clusters. Per node.
7S02003NWW| S88J| UFM Enterprise 5-year License and Gold-Support for Lenovo clusters. Per node.

For more information, see the following web page:
https://www.nvidia.com/en-us/networking/infiniband/ufm/

Server support

The following servers offer a PCIe 4.0 host interface. All other supported servers have a PCIe 3.0 host interface.

ThinkSystem SR635
ThinkSystem SR655
ThinkSystem SR645
ThinkSystem SR665

The following tables list the ThinkSystem servers that are compatible.

Table 6. Server support (Part 1 of 2

Part Number| Description| Edge| 1S Intel V2| 2S Intel V2| AMD| Dense V2| 4S V2| 8S
---|---|---|---|---|---|---|---|---
SE350 (7Z46 / 7D1X)| SE450 (7D8T)| ST50 V2 (7D8K / 7D8J)| ST250 V2 (7D8G / 7D8F)| SR250 V2 (7D7R / 7D7Q)| ST650 V2 (7Z75 / 7Z74)| SR630 V2 (7Z70 / 7Z71)| SR650 V2 (7Z72 / 7Z73)| SR670 V2 (7Z22 / 7Z23)| SR635 (7Y98 / 7Y99)| SR655 (7Y00 / 7Z01)| SR645 (7D2Y / 7D2X)| SR665 (7D2W / 7D2V)| SD630 V2 (7D1K)| SD650 V2 (7D1M)| SD650-N V2 (7D1N)| SN550 V2 (7Z69)| SR850 V2 (7D31 / 7D32)| SR860 V2 (7Z59 / 7Z60)| SR950 (7X11 / 7X12)
4C57A15326| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI
Adapter| N| N| N| N| N| N| Y| Y| Y| Y| Y| Y| Y| Y| N| N| N| Y| Y| Y
B951| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe 4 VPI
Adapter (SharedIO) DWC| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| N| N| N| N| N
B952| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe 4 VPI

Adapter DWC

| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| N| N| N| N
4C57A14179| ThinkSystem Mellanox HDR/200GbE 2x PCIe Aux Kit| N| N| N| N| N| N| Y| Y| Y| N| N| Y| Y| N| Y| N| N| Y| Y| Y

Table 7. Server support (Part 2 of 2)

Part Number| Description| 1S Intel V1| 2S Intel V1| Dense V1| 4S V1
---|---|---|---|---|---
ST50 (7Y48 / 7Y50)| ST250 (7Y45 / 7Y46)| SR150 (7Y54)| SR250 (7Y52 / 7Y51)| ST550 (7X09 / 7X10)| SR530 (7X07 / 7X08)| SR550 (7X03 / 7X04)| SR570 (7Y02 / 7Y03)| SR590 (7X98 / 7X99)| SR630 (7X01 / 7X02)| SR650 (7X05 / 7X06)| SR670 (7Y36 / 7Y37)| SD530 (7X21)| SD650 (7X58)| SN550 (7X16)| SN850 (7X15)| SR850 (7X18 / 7X19)| SR850P (7D2F / 2D2G)| SR860 (7X69 / 7X70)
4C57A15326| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe
4 VPI Adapter| N| N| N| N| N| N| N| N| N| Y| Y| Y| N| N| N| N| N| N| N
B951| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe
4 VPI Adapter (SharedIO) DWC| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N
B952| ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-Port PCIe
4 VPI Adapter DWC| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N| N
4C57A14179| ThinkSystem Mellanox HDR/200GbE 2x PCIe Aux Kit| N| N| N| N| N| N| N| N| N| Y| Y| Y| N| Y| N| N| N| N| N

Operating system support
The adapters support the operating systems listed in the following tables.

ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter, 4C57A15326
ThinkSystem Mellanox HDR/200GbE 2x PCIe Aux Kit, 4C57A14179

Tip: These tables are automatically generated based on data from Lenovo ServerProven.

Table 8. Operating system support for ThinkSystem Mellanox ConnectX-6 HDR/200GbE QSFP56 1-port PCIe 4 VPI Adapter, 4C57A15326

Operating systems| SD630 V2| SR630 V2| SR650 V2| SR670 V2| SR850 V2| SR860 V2| SR635| SR645| SR655| SR665| SR630 (Gen 2)| SR650 (Gen 2)| SR670 (Gen 2)| SR950 (Gen 2)| SR630 (Gen 1)| SR650 (Gen 1)| SR950 (Gen 1)
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Microsoft Windows 10| N| N| N| N| N| N| N| N| Y 17| N| N| N| N| N| N| N| N
Microsoft Windows Server 2016| Y| Y| Y| Y| Y| Y| Y| Y| N| Y| Y| Y| N| Y| Y| Y| Y
Microsoft Windows Server 2019| Y| Y| Y| Y| Y| Y| Y| Y| N| Y| Y| Y| N| Y| Y| Y| Y
Microsoft Windows Server 2022| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Microsoft Windows Server version 1709| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| N| Y
Microsoft Windows Server version 1803| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| N| Y
Red Hat Enterprise Linux 6.10| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 6.9| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.3| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.4| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.5| N| N| N| N| N| N| N| N| N| N| N| N| Y| N| Y| Y| Y
Red Hat Enterprise Linux 7.6| N| N| N| N| N| N| Y 1| Y 11| N| Y 20| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.7| N| N| N| N| N| N| Y 2| Y 12| Y 12| Y 12| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.8| N| N| N| N| N| N| Y 3| Y 3| Y 3| Y 3| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.9| Y| Y| Y| Y| Y| Y| Y 4| Y 13| Y 13| Y 13| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.0| N| N| N| N| N| N| Y 5| N| Y 18| N| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.1| N| N| N| N| N| N| Y 6| Y 14| Y 14| Y 14| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.2| Y| Y| Y| Y| Y| Y| Y 7| Y 15| Y 15| Y 15| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.4| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.5| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.6| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 9.0| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 12 SP3| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
SUSE Linux Enterprise Server 12 SP4| N| N| N| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
SUSE Linux Enterprise Server 12 SP5| Y| Y| Y| Y| Y| Y| N| Y| N| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15| N| N| N| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP1| N| N| N| N| N| N| Y 8| Y 8| Y 8| Y 8| Y| Y| Y| Y| Y| Y| Y
Operating systems| SD630 V2| SR630 V2| SR650 V2| SR670 V2| SR850 V2| SR860 V2| SR635| SR645| SR655| SR665| SR630 (Gen 2)| SR650 (Gen 2)| SR670 (Gen 2)| SR950 (Gen 2)| SR630 (Gen 1)| SR650 (Gen 1)| SR950 (Gen 1)
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
SUSE Linux Enterprise Server 15 SP2| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP4| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Ubuntu 18.04.5 LTS| Y| Y| Y| Y| N| N| N| N| N| N| N| N| N| N| N| N| N
Ubuntu 20.04 LTS| N| Y| Y| N| N| N| N| N| N| N| N| N| N| N| N| N| N
Ubuntu 22.04 LTS| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7| N| N| N| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U1| N| N| N| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U2| N| N| N| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U3| Y| Y| Y| Y| N| N| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0| N| N| N| N| N| N| Y 9| Y 16| Y 16| Y 16| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U1| N| N| N| N| Y| Y| Y 10| Y| Y 19| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U2| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 8.0| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y

RHEL 7.6 Not support with EPYC 7003 processors.
RHEL7.7 Not support with EPYC 7003 processors.
RHEL7.8 Not support with EPYC 7003 processors
RHEL7.9 Not support with EPYC 7003 processors.
RHEL8.0 Not support with EPYC 7003 processors.
RHEL8.1 Not support with EPYC 7003 processors.
RHEL8.2 Not support with EPYC 7003 processors.
SLES15 SP1 Not support with EPYC 7003 processors
VMware 7.0 Not support with EPYC 7003 processors.
VMware 7.0 U1 Not support with EPYC 7003 processors.
RHEL7.6 Not support with EPYC 7003 processors
RHEL7.7 Not support with EPYC 7003 processors
RHEL7.9 Not support with EPYC 7003 processors
RHEL8.1 Not support with EPYC 7003 processors
RHEL8.2 Not support with EPYC 7003 processors
VMware 7.0 Not support with EPYC 7003 processors
ISG will not sell/preload this OS, but compatibility and cert only.
RHEL8.0 Not support with EPYC 7003 processors
VMware 7.0 U1 Not support with EPYC 7003 processors
RHEL7.6 Not support with EPYC 7003 processors.

Table 9. Operating system support for ThinkSystem Mellanox HDR/200GbE 2x PCIe Aux Kit, 4C57A14179

Operating systems| SR630 V2| SR650 V2| SR670 V2| SR850 V2| SR860 V2| SR645| SR665| SR630 (Gen 2)| SR650 (Gen 2)| SR670 (Gen 2)| SR950 (Gen 2)| SR630 (Gen 1)| SR650 (Gen 1)| SR950 (Gen 1)
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Microsoft Windows Server 2016| Y| Y| Y| Y| Y| Y| Y| Y| Y| N| Y| Y| Y| Y
Microsoft Windows Server 2019| Y| Y| Y| Y| Y| Y| Y| Y| Y| N| Y| Y| Y| Y
Microsoft Windows Server 2022| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Microsoft Windows Server version 1709| N| N| N| N| N| N| N| N| N| N| N| Y| N| Y
Microsoft Windows Server version 1803| N| N| N| N| N| N| N| N| N| N| N| Y| N| Y
Red Hat Enterprise Linux 6.10| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 6.9| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.3| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.4| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
Red Hat Enterprise Linux 7.5| N| N| N| N| N| N| N| N| N| Y| N| Y| Y| Y
Red Hat Enterprise Linux 7.6| N| N| N| N| N| Y 1| Y 9| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.7| N| N| N| N| N| Y 2| Y 2| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.8| N| N| N| N| N| Y 3| Y 3| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 7.9| Y| Y| Y| Y| Y| Y 4| Y 4| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.0| N| N| N| N| N| N| N| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.1| N| N| N| N| N| Y 5| Y 5| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.2| Y| Y| Y| Y| Y| Y 6| Y 6| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.4| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.5| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 8.6| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Red Hat Enterprise Linux 9.0| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 12 SP3| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
SUSE Linux Enterprise Server 12 SP4| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
SUSE Linux Enterprise Server 12 SP5| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP1| N| N| N| N| N| Y 7| Y 7| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP2| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
SUSE Linux Enterprise Server 15 SP4| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Ubuntu 18.04.5 LTS| Y| Y| Y| N| N| N| N| N| N| N| N| N| N| N
Ubuntu 20.04 LTS| Y| Y| N| N| N| N| N| N| N| N| N| N| N| N
Ubuntu 22.04 LTS| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
Operating systems| SR630 V2| SR650 V2| SR670 V2| SR850 V2| SR860 V2| SR645| SR665| SR630 (Gen 2)| SR650 (Gen 2)| SR670 (Gen 2)| SR950 (Gen 2)| SR630 (Gen 1)| SR650 (Gen 1)| SR950 (Gen 1)
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
VMware vSphere Hypervisor (ESXi) 6.7| N| N| N| N| N| N| N| N| N| N| N| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U1| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U2| N| N| N| N| N| N| N| Y| Y| N| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 6.7 U3| Y| Y| Y| N| N| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0| N| N| N| N| N| Y 8| Y 8| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U1| N| N| N| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U2| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 7.0 U3| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y
VMware vSphere Hypervisor (ESXi) 8.0| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y| Y

RHEL7.6 Not support with EPYC 7003 processors
RHEL7.7 Not support with EPYC 7003 processors
RHEL7.8 Not support with EPYC 7003 processors
RHEL7.9 Not support with EPYC 7003 processors
RHEL8.1 Not support with EPYC 7003 processors
RHEL8.2 Not support with EPYC 7003 processors
SLES15 SP1 Not support with EPYC 7003 processors
VMware 7.0 Not support with EPYC 7003 processors
RHEL7.6 Not support with EPYC 7003 processors.

Regulatory approvals

The adapters have the following regulatory approvals:

Safety: CB / cTUVus / CE
EMC: CE / FCC / VCCI / ICES / RCM / KC
RoHS: RoHS Compliant

Operating environment
The adapters have the following operating characteristics:

Typical power consumption (passive cables): 19.3W
Maximum power available through QSFP56 port: 5W
Temperature
- Operational: 0°C to 55°C
- Non-operational: -40°C to 70°C
Humidity: 90% relative humidity

Warranty

One year limited warranty. When installed in a Lenovo server, the adapter assumes the server’s base warranty and any warranty upgrades.

Related publications
For more information, refer to these documents:

Networking Options for ThinkSystem Servers:
https://lenovopress.com/lp0765-networking-options-for-thinksystem-servers
ServerProven compatibility:
http://www.lenovo.com/us/en/serverproven
Mellanox InfiniBand product page:
https://www.nvidia.com/en-us/networking/infiniband-adapters/
ConnectX-6 VPI user manual:
https://docs.nvidia.com/networking/display/ConnectX6VPI

Related product families
Product families related to this document are the following:

Ethernet Adapters
InfiniBand & Omni-Path Adapters

Notices
Lenovo may not offer the products, services, or features discussed in this document in all countries. Consult your local Lenovo representative for information on the products and services currently available in your area. Any reference to a Lenovo product, program, or service is not intended to state or imply that only that Lenovo product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any Lenovo intellectual property right may be used instead. However, it is the user’s responsibility to evaluate and verify the operation of any other product, program, or service. Lenovo may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

Lenovo (United States), Inc.
8001 Development Drive
Morrisville, NC 27560
U.S.A.

Attention: Lenovo Director of Licensing
LENOVO PROVIDES THIS PUBLICATION ”AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. Lenovo may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.
The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons. The information contained in this document does not affect or change Lenovo product specifications or warranties. Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of Lenovo or third parties. All information contained in this document was obtained in specific environments and is presented as an illustration. The result obtained in other operating environments may vary. Lenovo may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.
Any references in this publication to non-Lenovo Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this Lenovo product, and use of those Web sites is at your own risk. Any performance data contained herein was determined in a controlled environment. Therefore, the result obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

© Copyright Lenovo 2022. All rights reserved.
This document, LP1195, was created or updated on April 5, 2022.
Send us your comments in one of the following ways:

Use the online Contact us review form found at:
https://lenovopress.lenovo.com/LP1195
Send your comments in an e-mail to:
[email protected]

This document is available online at https://lenovopress.lenovo.com/LP1195.

Trademarks

Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at
https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other countries, or both:

Lenovo®
ServerProven®
ThinkSystem®

The following terms are trademarks of other companies:
Intel® is a trademark of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.