CISCO SE-NODE-G2 Nexus Dashboard Fabric Controller User Guide

SE-NODE-G2 Nexus Dashboard Fabric Controller

“`html

Specifications:

• Product Name: Cisco Nexus Dashboard Fabric Controller
• Release Version: 12.2.2
• Maximum ToR Switches: 32
• Node Types: SE-NODE-G2, ND-NODE-L4

Product Information:

The Cisco Nexus Dashboard Fabric Controller, Release 12.2.2,
allows for connecting a maximum of 32 ToR switches or 16 vPC-ToR
pairs per leaf-vPC pair. It provides verified scalability values
for various deployment types and system resources required to run
NDFC on top of Nexus Dashboard.

Server Resource Requirements:

NDFC can be deployed on different form factors with varying CPU,
memory, and storage requirements. The following are the
specifications for different node types:

• Physical Node (pND) – SE-NODE-G2: 16 vCPUs, 64
  GB Memory, 2 x 10-core 2.2GHz Intel Xeon Silver CPU

• Physical Node (pND) – ND-NODE-L4: 2.8GHz AMD
  CPU, 256 GB Memory

• SAN Controller: 16 vCPUs, 64 GB Memory, 128GB
  node (with SAN Insights)

Scale Limits for Fabric Discovery:

Product Usage Instructions:

1. Connecting ToR Switches:

To connect ToR switches, ensure that each leaf-vPC pair supports
a maximum of 32 ToR switches or 16 vPC-ToR pairs.

2. Running NDFC on Nexus Dashboard:

Refer to the Nexus Dashboard Capacity Planning guide to
determine the appropriate server resource requirements based on the
deployment type and node type.

3. Fabric Discovery Scale Limits:

When deploying Fabric Discovery personas on Nexus Dashboard,
ensure that you stay within the verified limits specified for each
deployment type to avoid scalability issues.

Frequently Asked Questions (FAQ):

Q: Can I exceed the verified scale limits for Fabric

Discovery?

A: Exceeding the verified limits may lead to performance issues
and is not recommended. Stick to the specified limits for optimal
operation.

Q: How many ToR switches can be connected per leaf-vPC

pair?

A: You can connect a maximum of 32 ToR switches or 16 vPC-ToR
pairs per leaf-vPC pair.

“`

Verified Scalability Guide for Cisco Nexus Dashboard Fabric Controller, Release 12.2.2
New and Changed Information 2 Verified Scale Limits for Release 12.2.2 2
Full Cisco Trademarks with Software License ?

Revised: August 12, 2024
New and Changed Information

The following table provides an overview of the significant changes to this guide for this current release. The table does not provide an exhaustive list of all changes made to the guide or of the new features in this release.

Feature

Description

Where Documented

Maximum of 32 ToR switches

You can connect a maximum of 32 ToR switches (or 16 vPC-ToR pairs) per leaf- vPC pair.

Scale Limits For Provisioning New Data Center VXLAN EVPN Fabrics (also referred to as “Greenfield” Deployment)

Verified Scale Limits for Release 12.2.2
This section provides verified scalability values for various deployment types for Cisco Nexus Dashboard Fabric Controller, Release 12.2.2.
The values are validated on testbeds that are enabled with a reasonable number of features and aren’t theoretical system limits for Cisco Nexus Dashboard Fabric Controller software or Cisco Nexus/MDS switch hardware and software. When you try to achieve maximum scalability by scaling multiple features at the same time, results might differ from the values that are listed here.

Nexus Dashboard System Resources
The following table provides information about Server Resource Requirements to run NDFC on top of Nexus Dashboard. Refer to Nexus Dashboard Capacity Planning to determine the number of switches supported for each deployment.
Cisco Nexus Dashboard can be deployed using number of different form factors. NDFC can be deployed on the following form factors:
· pND – Physical Nexus Dashboard
· vND – Virtual Nexus Dashboard

Table 1: Server Resource Requirements to run NDFC on top of Nexus Dashboard

Deployment Type

Node Type

CPUs

Memory

Fabric Discovery

Virtual Node (vND) ­ app node
Physical Node (pND)
(PID: SE-NODE-G2)

16 vCPUs

64 GB

2 x 10-core 2.2GHz Intel 256 GB of RAM Xeon Silver CPU

Physical Node (pND) (PID: ND-NODE-L4)

2.8GHz AMD CPU

256 GB of RAM

Storage (Throughput: 40-50 MB/s) 550 GB SSD
4 x 2.4 TB HDDs 400 GB SSD 1.2 TB NVME drive 4 x 2.4 TB HDDs 960 GB SSD 1.6 TB NVME drive

2

Deployment Type Fabric Controller

Node Type

CPUs

Memory

Virtual Node (vND) ­ app node
Physical Node (pND)
(PID: SE-NODE-G2)

16 vCPUs

64 GB

2 x 10-core 2.2GHz Intel 256 GB of RAM Xeon Silver CPU

Physical Node (pND) (PID: ND-NODE-L4)

2.8GHz AMD CPU

256 GB of RAM

SAN Controller

Virtual Node (vND) ­ app node
(with SAN Insights)

16 vCPUs
(with physical reservation)

64 GB
(with physical reservation)

Data Node (vND) ­ Data 32 vCPUs

node

(with physical

(with SAN Insights) reservation)

128GB
(with physical reservation)

Physical Node (pND) (PID: SE-NODE-G2)

2 x 10-core 2.2GHz Intel 256 GB of RAM Xeon Silver CPU

Physical Node (pND) (PID: ND-NODE-L4)

2.8GHz AMD CPU

256 GB of RAM

Scale Limits for NDFC Fabric Discovery
Table 2: Scale Limits for Fabric Discovery Persona and Nexus Dashboard

Profile Fabric Discovery Fabric Discovery Fabric Discovery Fabric Discovery Fabric Discovery

Deployment Type 1-Node vND (app node) 3-Node vND (app node) 5-Node vND (app node) 1-Node pND 3-Node pND

Verified Limit 100 switches 200 switches 1000 switches 100 switches 1000 switches

Storage (Throughput: 40-50 MB/s) 550 GB SSD
4 x 2.4 TB HDDs 400 GB SSD 1.2 TB NVME drive 4 x 2.4 TB HDDs 960 GB SSD 1.6 TB NVME drive 550 GB SSD
3 TB SSD
4 x 2.4 TB HDDs 400 GB SSD 1.2 TB NVME drive 4 x 2.4 TB HDDs 960 GB SSD 1.6 TB NVME drive

3

Scale Limits for NDFC Fabric Controller
Table 3: Scale Limits for Fabric Controller Persona and Nexus Dashboard

Profile

Deployment Type

Verified Limit

Fabric Controller

1-Node vND (app node)

50 switches

Fabric Controller Fabric Controller Fabric Controller

3-Node vND (app node) 5-Node vND (app node) 5-Node vND (app node)

100 switches 400 switches for Easy Fabrics1 1000 switches for External Fabrics2

Fabric Controller Fabric Controller Fabric Controller

1-Node pND 3-Node pND 3-Node pND

50 switches 500 switches for Easy Fabrics1 1000 switches for External Fabrics2

1 Easy Fabrics include Data Center VXLAN EVPN fabrics and BGP fabrics.
2External Fabrics include Flexible Network fabrics, Classic LAN fabrics, External Connectivity Network fabrics, and Multi-Site Interconnect Network fabrics. Both managed and monitored mode are supported.

Table 4: Scale Limits for Switches and Fabrics in Fabric Controller

Description

Verified Limit

Switches per fabric Physical Interfaces per NDFC instance1

200 30000

1 Supported scale for 1-node vND is 2500 physical interfaces.

Table 5: Scale Limits For Provisioning New Data Center VXLAN EVPN Fabrics (also referred to as “Greenfield” Deployment)

Description Fabric Underlay Overlay Switches per fabric Overlay Scale for VRFs and Networks1
VRF instances for external connectivity IPAM Integrator application

Verified Limit
200 500 VRFs, 2000 Layer-3 Networks or 2500 Layer-2 Networks 500 150 networks with a total of 4K IP allocations on the Infoblox server

4

Description

Verified Limit

ToR and Leaf devices
Endpoint Locator2 Endpoints VXLAN EVPN Multi-Site Domain

A Data Center VXLAN EVPN fabric can manage both Layer-2 ToR switches and leaf switches. Maximum scale for this sort of fabric is 40 leaf switches and 320 ToR switches. Maximum of 32 ToR switches (or 16 vPC-ToR pairs) can be connected per leaf-vPC pair.
100000

Sites

30

Virtual Machine Manager (VMM)3

Virtual Machines (VMs)

5500

VMware Center Servers

4

Kubernetes Visualizer application

Maximum of 160 namespaces with maximum of 1002 pods

1 Supported scale for 1-node vND is 250 VRFS and 1000 networks. 2 Supported scale for 1-node vND is 1 instance of endpoint locator with 10000 endpoints. 3 Supported scale for 1-node vND is 1 VMware Center Server and 1000 VMs.

Note

· Refer to the following table if you are transitioning a command line interface (CLI) configured Cisco Nexus 9000 series

switches based VXLAN EVPN fabric to NDFC.

· If you are performing a Brownfield migration, where you transition an existing Data Center VXLAN EVPN fabric management to NDFC, note that the scale limts provided below no longer apply once that Brownfield migration is completed and you should be using the scale limits provided in the Greenfield table above instead.

Table 6: Scale Limits For Transitioning Existing Data Center VXLAN EVPN Fabric Management to NDFC (also referred to as “Brownfield Migration”)

Description
Fabric Underlay and Overlay Switches per fabric Physical Interfaces VRF instances Overlay networks VRF instances for external connectivity Endpoint Locator

Verified Limit
200 11500 400 1050 400

5

Description Endpoints IPAM Integrator application
Virtual Machine Manager (VMM) Virtual Machines (VMs) VMware Center Servers Kubernetes Visualizer application

Verified Limit 50000 150 networks with a total of 4K IP allocations on the Infoblox server
5500 4 Maximum of 160 namespaces with maximum of 1002 pods

Scale Limits for Cohosting NDFC and Other Services
Table 7: Scale Limits for Cohosting Nexus Dashboard Insights and NDFC (NDFC 12.1.3/ND 3.0.1 and earlier)

Profile
Nexus Dashboard Insights and Nexus Dashboard Fabric Discovery

Deployment Type 3-Node pND

Verified Limit · 50 switches · 10,000 flows/sec

Nexus Dashboard Insights and Nexus Dashboard Fabric Controller

3-Node pND

· 50 switches · 10,000 flows/sec

Table 8: Scale Limits for Cohosting Nexus Dashboard Insights and NDFC (NDFC 12.2.1/ND 3.1 and later)

Profile
Nexus Dashboard Insights and Nexus Dashboard Fabric Discovery (NX-OS without controller mode1)
Nexus Dashboard Insights and Nexus Dashboard Fabric Controller

Deployment Type 3-Node pND
3-Node pND

Verified Limit · 250 switches · 10,000 flows/sec
· 250 switches · 10,000 flows/sec

1 NX-OS Discovery mode is required when you deploy Nexus Dashboard Insights for NX-OS fabrics without using NDFC.

Scale Limits for IPFM Fabrics
Table 9: Scale Limits for Nexus Dashboard and IPFM Fabrics

Profile Fabric Controller

Deployment Type 1-Node vND

Verified Limit
35 switches (2 spine switches and 33 leaf switches)

6

Profile Fabric Controller Fabric Controller Fabric Controller

Deployment Type 3-Node vND 1-Node pND 3-Node pND

Verified Limit
120 switches (2 spine switches, 100 leaf switches, and 18 Tier-2 leaf switches)
35 switches (2 spine switches and 33 leaf switches)
120 switches (2 spine switches, 100 leaf switches, and 18 Tier-2 leaf switches)

Table 10: Scale Limits for IPFM Fabrics

Description

Verified Limit
NBM Active Mode Only NBM Passive Mode Only

Switches

120

Number of flows

32000

Number of End Points 5000 (Discovered Hosts)

VRFs

16

Host Policy – Sender 8000

Host Policy – Receiver 8000

Host Policy – PIM

512

(Remote)

Flow Policy

2500

NBM ASM group-range 20

Host Alias

2500

Flow Alias

2500

NAT Flows

3000

RTP Flow Monitoring 8000

PTP Monitoring

120 switches

32 32000 1500
16 NA NA NA
NA NA NA NA 3000 8000 32 switches

Mixed Mode

NBM Active VRF

NBM Passive VRF

32

32

32000

32000

3500

1500

16

16

8000

NA

8000

NA

512

NA

2500 20 2500 2500 3000 8000 32 switches

NA NA NA NA 3000 8000 32 switches

7

Scale Limits for NDFC SAN Controller
Table 11: Scale Limits for SAN Zones
Description Zone sets Zone

Verified Limits 1000 16000

Table 12: Scale Limits for Nexus Dashboard and SAN Controller Persona

Profile

Deployment Type

Verified Limit

SAN Controller

1-Node vND (app node)1

Without SAN Insights 80 switches, 20K ports

With SAN Insights
40 switches, 10K ports, and 40K ITs

1-Node vND (data node)

80 switches, 20K ports

80 switches, 20K ports, and 1M ITLs/ITNs 2

1-Node pND (SE)

80 switches, 20K ports

80 switches, 20K ports, and 120K ITLs/ITNs

SAN Controller

3-Node vND (app node)

160 switches, 40K ports

80 switches, 20K ports, and 100K ITs

3-Node vND (data node)

160 switches, 40K ports

160 switches, 40K ports, and 240K ITLs/ITNs

3-Node pND

160 switches, 40K ports

160 switches, 40K ports, and 500K ITLs/ITNs

1 App nodes have fewer features than data nodes. For example, the lun and fc- scsi.scsi_initiator_itl_flow features are not supported in the app ova, whereas those features are supported in the data ova. Therefore, you would have to install the data ova in order to use the lun or fc- scsi.scsi_initiator_itl_flow features.
2 1 million flows is the maximum number supported. If other features are enabled that consume resources, 1 million flows will not be stable in all situations. NDFC consumes more resources per flow when processing telemetry from a larger number of devices. Watch flow counts and node memory usage (1 minute averages above ~105GB starts to show instability).

Note ITLs – Initiator-Target-LUNs ITNs – Initiator-Target-Namespace ID ITs – Initiator-Targets

8

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE- NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.
Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/c/en/us/about/legal/trademarks.html. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)
© 2024 Cisco Systems, Inc. All rights reserved.

Americas Headquarters Cisco Systems, Inc. San Jose, CA 95134-1706 USA

Asia Pacific Headquarters CiscoSystems(USA)Pte.Ltd. Singapore

Europe Headquarters CiscoSystemsInternationalBV Amsterdam,TheNetherlands

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.

Read User Manual Online (PDF format)

Read User Manual Online (PDF format)  >>

Download This Manual (PDF format)

Download this manual  >>