Lenovo Mellanox 2 Port 10Gb Ethernet Expansion Card for IBM BladeCenter User Guide
- June 3, 2024
- Lenovo
Table of Contents
Lenovo Mellanox 2 Port 10Gb Ethernet Expansion Card for IBM BladeCenter
Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM BladeCenter
(Withdrawn)
Product Guide (withdrawn product)
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM® BladeCenter® delivers the industry’s lowest latency performance of 10 Gigabit Ethernet connectivity with stateless offloads for converged fabrics in enterprise data centers, high-performance computing, and embedded environments. Financial services, cluster databases, web infrastructure, and cloud computing are just a few example applications that will achieve significant throughput and latency improvements resulting in faster access, real-time response, and increased number of users per server. The adapter is based on Mellanox ConnectX-2 EN technology, which improves network performance by increasing available bandwidth to the CPU, especially in virtualized server environments.
Figure 1 shows the adapter. Figure 1. Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM BladeCenter
Did you know?
New enhancements such as Data Center Ethernet (DCE) are being added to
Ethernet for better I/O consolidations. The InfiniBand Trade Association’s
(IBTA) RDMA over Converged Ethernet (RoCE) technology provides efficient low-
latency RDMA transport over Layer 2 Ethernet. The RoE software stack maintains
existing and future application compatibility for bandwidth and latency-
sensitive clustering applications. With link-level interoperability in
existing Ethernet infrastructure, network administrators can leverage existing
data center fabric management solutions with this adapter.
Part number information
The part number to order this card is shown in Table 1.
Table 1. Ordering part number and feature code
Description | Part number | Feature code |
---|---|---|
Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM BladeCenter | ||
90Y3570 | A1NW |
These part numbers include the following items:
- One Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh)
- Documentation CD that contains the Installation and User’s Guide
- Important Notices document
Features and benefits
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) has the features discussed below.
Performance
Based on ConnectX-2 EN technology, the Mellanox 2-port 10Gb Ethernet Expansion
Card (CFFh) provides a high level of throughput performance for all network
environments by removing I/O bottlenecks in mainstream servers that are
limiting application performance. Blade servers in the BladeCenter H and HT
chassis can fully utilize both 10 Gbps ports and achieve up to 40 Gbps duplex
bandwidth. Hardware-based stateless offload engines handle the TCP/UDP/IP
segmentation, reassembly, and checksum calculations that would otherwise
burden the host processor. These offload technologies are fully compatible
with Microsoft RSS and NetDMA.
RDMA over Converged Ethernet (RoCE) further accelerates application run time.
RoCE specification provides efficient data transfer with very low latencies on
lossless Ethernet networks. RoCE enables lowest latency memory transaction,
with less than 2 µs at full bandwidth with small message size. This allows
very high-volume transaction-intensive applications typical of financial
market firms and other industries where speed of data delivery is paramount to
take advantage. With the Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh),
high-frequency transaction applications are able to access trading information
with shorter times, making sure that the trading servers are able to respond
first to any new market data and market inefficiencies, while the higher
throughput enables higher volume trading, maximizing liquidity and
profitability.
In data mining or web crawl applications, RoCE provides the needed boost in
performance to search faster by solving the network latency bottleneck
associated with I/O cards and the corresponding transport technology in the
cloud. Various other applications that benefit from RoCE with ConnectX-2 EN
include Web 2.0 (Content Delivery Network), Business intelligence, database
transactions, and various Cloud computing applications. Mellanox ConnectX-2
EN’s low-power consumption (less than 3.5 W per port) provides customers with
high bandwidth and low latency at the lowest cost of ownership.
I/O virtualization
ConnectX-2 EN supports hardware-based I/O virtualization, providing dedicated
adapter resources and guaranteed isolation and protection for virtual machines
(VM) within the server. ConnectX-2 EN gives data center managers better server
utilization and LAN and SAN unification while reducing costs, power, and
complexity.
Quality of service
Resource allocation per application or per VM is provided by the advanced QoS
supported by ConnectX-2 EN. Service levels for multiple traffic types can be
based on IETF DiffServ or IEEE 802.1p/Q, along with the Data Center Bridging
enhancements, allowing system administrators to prioritize traffic by
application, virtual machine, or protocol. This powerful combination of QoS
and prioritization provides the ultimate fine-grained control of traffic,
ensuring that applications run smoothly in today’s complex environment.
Specifications
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) has the following
specifications:
- CFFh form-factor
- Two full-duplex 10 Gb Ethernet ports
- PCI Express x8 Gen 2 host interface
- Mellanox ConnectX-2 EN ASIC
- Connectivity to high-speed I/O module bays in BladeCenter H and BladeCenter HT chassis Features: Stateless Offload, Priority Flow Control, FCoE Ready
- Power consumption: 6.4 W (typical)
- Stateless iSCSI offload
- BladeCenter Open Fabric Manager support (available early 4Q/2011)
- PXE v2.1 boot support
- Wake on LAN support (available early 4Q/2011)
Ethernet support and standards are:
- IEEE Std 802.3ae 10 Gigabit Ethernet
- IEEE Std 802.3ad Link Aggregation and Failover
- IEEE Std 802.3x Pause
- IEEE 802.1Q, .1p VLAN tags and priority Multicast
- Jumbo frame support (10 KB)
- 128 MAC/VLAN addresses per port
Data Center Bridging (DCB) support:
- Enhanced Transmission Selection (ETS) (IEEE 802.1Qaz)
- Quantized Congestion Notification (QCN) (IEEE 802.1Qau)
- Priority-based Flow Control (PFC) (IEEE 802.1Qbb)
- Data Center Bridging Capability eXchange (DCBX) protocol
TCP/UDP/IP stateless offload:
- TCP/UDP/IP checksum offload
- TCP Large Send Offload (< 64 KB) or Giant Send Offload (64 KB – 16 MB) for segmentation
- Receive Side Scaling (RSS) up to 32 queues
- Line rate packet filtering
Additional CPU offloads:
- RDMA over Converged Ethernet support
- FC checksum offload
- VMDirect Path support
- Traffic steering across multiple cores
- Intelligent interrupt coalescence
- Compliant to Microsoft RSS and NetDMA
Hardware-based I/O virtualization:
- Address translation and protection
- Dedicated adapter resources and guaranteed isolation
- Multiple queues per virtual machine
- Hardware switching between guest OSs
- Enhanced QoS for vNICs
- VMware NetQueue support
Storage support:
- Compliant with T11.3 FC-BB-5 FCoE frame format
Operating environment
This card is supported in the following environment:
- Temperature: 0° C to 55° C (32° F to 131° F)
- Operating power: Approximately 6.4 watts
- Dimensions: Approximately 15.9 cm x 12.5 cm (6.26 in. x 4.92 in.)
Warranty
One-year limited warranty: When installed in a BladeCenter server, the card
assumes the system’s base warranty and any IBM ServicePac® upgrade.
Supported servers
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) is supported in the
IBM BladeCenter servers listed in Table 2.
Table 2. Supported servers
Figure 2 shows where the CFFh card is installed in a BladeCenter server.
Figure 2. Location on the BladeCenter server planar where the CFFh card is installed
Supported BladeCenter chassis and I/O modules
IBM BladeCenter chassis support is based on the blade server type in which the
expansion card is installed. Consult IBM ServerProven® to see which chassis
each blade server type is supported at:
http://ibm.com/servers/eserver/serverproven/compat/us/.
Table 3 lists the supported chassis and I/O module combinations that the card supports.
Table 3. I/O modules supported with the Mellanox 2-port 10Gb Ethernet
Expansion Card (CFFh) In BladeCenter H, the ports of CFFh cards are routed
through the midplane to I/O bays 7, 8, 9, and 10 (Figure 3). The BladeCenter
HT works in a similar way because the CFFh cards are also routed through the
midplane to I/O bays 7, 8, 9, and 10. Figure 3. IBM BladeCenter H I/O topology
showing the I/O paths from CFFh expansion cards
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) requires that two I/O
modules be installed in bays 7 and 9 of the BladeCenter H or HT chassis (Table
4).
Table 4. Locations of I/O modules required to connect to the expansion card
Expansion card| I/O bay 7| I/O bay 8| I/O bay 9| I/O
bay 10
---|---|---|---|---
Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh)| Supported I/O module| Not
used| Supported I/O module| Not used
Popular configurations
Figure 4 shows a configuration using the Mellanox 2-port 10Gb Ethernet
Expansion Card (CFFh) together with two BNT Virtual Fabric 10 Gb Switch
Modules. This solution enables two 10 Gbps Ethernet connections from each
blade server. Two BNT Virtual Fabric 10 Gb Switch Modules are installed in bay
7 and bay 9 of the BladeCenter H chassis. All connections between the
controller, the card, and the switch modules are internal to the chassis. No
internal cabling is needed.
Figure 4. 20 Gb solution using Intel 2-port 10 Gb Ethernet Expansion Card
(CFFh)
Table 5 lists the components used in this configuration.
Table 5. Components used when connecting Intel 2-port 10 Gb Ethernet Expansion
Card (CFFh) to two BNT Virtual Fabric 10 Gb Switch Modules
Diagram reference| Part number/machine type|
Description| Quantity
---|---|---|---
1| Varies| IBM BladeCenter HS22 or other supported server| 1 to 14
2| 90Y3570| Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh)| 1 per server
3| 8852 or 8740/8750| BladeCenter H or BladeCenter HT| 1
4| 46C7191| BNT Virtual Fabric 10 Gb Switch Module| 2
5| 44W4408| IBM 10GBase-SR SFP+ Transceiver| Up to 20*
*The BNT Virtual Fabric 10Gb Switch Module has 10 external ports. You must have one transceiver for every external port that you plan to utilize.
Operating system support
The Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) supports the following
operating systems:
- Microsoft Windows Server 2008 R2
- Microsoft Windows Server 2008, Datacenter x64 Edition
- Microsoft Windows Server 2008, Enterprise x64 Edition
- Microsoft Windows Server 2008, Standard x64 Edition
- Microsoft Windows Server 2008, Web x64 Edition
- Red Hat Enterprise Linux 5 Server with Xen x64 Edition
- Red Hat Enterprise Linux 5 Server x64 Edition
- Red Hat Enterprise Linux 6 Server x64 Edition
- SUSE LINUX Enterprise Server 10 for AMD64/EM64T
- SUSE LINUX Enterprise Server 10 with Xen for AMD64/EM64T SUSE LINUX Enterprise Server 11 for AMD64/EM64T
- SUSE LINUX Enterprise Server 11 with Xen for AMD64/EM64T VMware ESX 4.0
- VMware ESX 4.1
- VMware ESXi 4.0
- VMware ESXi 4.1
Support for operating systems is based on the combination of the expansion
card and the blade server in which it is installed. See IBM ServerProven for
the latest information about the specific versions and service packs supported
by going to the following address:
http://ibm.com/servers/eserver/serverproven/compat/us/.
Select the blade server and then select the expansion card to see the
supported operating systems.
Related publications
For more information, see the following documents:
-
Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM BladeCenter Installation Guide
http://ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5087563 -
IBM BladeCenter Interoperability Guide
http://ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5073016 -
IBM Redbooks® publication IBM BladeCenter Products and Technology, SG24-7523
http://www.redbooks.ibm.com/abstracts/sg247523.html
Related product families
Product families related to this document are the following:
- Blade Network Adapters
Notices
Lenovo may not offer the products, services, or features discussed in this
document in all countries. Consult your local Lenovo representative for
information on the products and services currently available in your area. Any
reference to a Lenovo product, program, or service is not intended to state or
imply that only that Lenovo product, program, or service may be used. Any
functionally equivalent product, program, or service that does not infringe
any Lenovo intellectual property right may be used instead. However, it is the
user’s responsibility to evaluate and verify the operation of any other
product, program, or service. Lenovo may have patents or pending patent
applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents.
You can send license inquiries, in writing, to:
Lenovo (United States), Inc.
8001 Development Drive
Morrisville, NC 27560
U.S.A.
Attention: Lenovo Director of Licensing
LENOVO PROVIDES THIS PUBLICATION ”AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors.
Changes are periodically made to the information herein; these changes will be
incorporated in new editions of the publication. Lenovo may make improvements
and/or changes in the product(s) and/or the program(s) described in this
publication at any time without notice.
The products described in this document are not intended for use in
implantation or other life support applications where malfunction may result
in injury or death to persons. The information contained in this document does
not affect or change Lenovo product specifications or warranties. Nothing in
this document shall operate as an express or implied license or indemnity
under the intellectual property rights of Lenovo or third parties. All
information contained in this document was obtained in specific environments
and is presented as an illustration. The result obtained in other operating
environments may vary. Lenovo may use or distribute any of the information you
supply in any way it believes appropriate without incurring any obligation to
you.
Any references in this publication to non-Lenovo Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this Lenovo product, and use of those Web sites is at your own risk. Any performance data contained herein was determined in a controlled environment. Therefore, the result obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.
© Copyright Lenovo 2022. All rights reserved.
This document, TIPS0820, was created or updated on June 14, 2011.
Send us your comments in one of the following ways:
-
Use the online Contact us review form found at:
https://lenovopress.lenovo.com/TIPS0820 -
Send your comments in an e-mail to:
comments@lenovopress.com
This document is available online at https://lenovopress.lenovo.com/TIPS0820.
Trademarks
Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo
in the United States, other countries, or both. A current list of Lenovo
trademarks is available on the Web at
https://www.lenovo.com/us/en/legal/copytrade/.
The following terms are trademarks of Lenovo in the United States, other
countries, or both:
Lenovo®
BNT®
BladeCenter Open Fabric
BladeCenter®
ServerProven®
The following terms are trademarks of other companies:
Intel® is a trademark of Intel Corporation or its subsidiaries.
Linux® is the trademark of Linus Torvalds in the U.S. and other countries.
Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft
Corporation in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of
others.
References
- Enterprise Business Server Solutions | IBM
- IBM Support
- IBM Support
- IBM BladeCenter Products and Technology
- Blade Network Adapters > Lenovo Press
- Mellanox 2-port 10Gb Ethernet Expansion Card (CFFh) for IBM BladeCenter (Withdrawn) Product Guide (withdrawn product) > Lenovo Press
- Copyright and Trademark Information | Lenovo US | Lenovo US
Read User Manual Online (PDF format)
Read User Manual Online (PDF format) >>