CISCO Nexus 3600 Series NX-OS Verified Scalability Guide Release 9.3 User Guide
- May 15, 2024
- Cisco
Table of Contents
- CISCO Nexus 3600 Series NX-OS Verified Scalability Guide Release 9.3
- Specifications:
- Product Information:
- Verified Scalability Limits (Unidimensional):
- Verified Scalability Limits (Multidimensional)
- CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
- Americas Headquarters
- Asia Pacific Headquarters CiscoSystems(USA)Pte.Ltd. Singapore
- Europe Headquarters CiscoSystemsInternationalBV Amsterdam,TheNetherlands
- FAQ
- Read User Manual Online (PDF format)
- Download This Manual (PDF format)
CISCO Nexus 3600 Series NX-OS Verified Scalability Guide Release 9.3
USER GUIDE
Specifications:
- Product Name: Cisco Nexus 3600 Series NX-OS Verified Scalability Guide
- Release: 9.3(12)
Product Information:
This document provides information on the Cisco NX-OS configuration limits for the Cisco Nexus 3600 1U Top of Rack platforms, specifically N3K-C3636C-R and N3K-C36180YC-R. The values in this guide are validated limits by Cisco and may increase over time with further testing and validation.
Verified Scalability Limits (Unidimensional):
The verified scalability limits focus on the scalability of individual features at a time. These limits are specific to Cisco NX-OS Release 9.3(12) and may vary based on hardware capabilities and future software releases.
Table 1: Interfaces Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
DHCP servers/relay IPs per switch
|
5 (IPv4) + 5 (IPv6)1
MAC address – table limit per port
|
2,000 – (Nexus 3636C-R and 36180YC-R switches)
MAC address – table system, VLAN limit
|
2,000 – (Nexus 3636C-R and 36180YC-R switches)
Port channel member links
|
32
SVIs
|
3967
vPCs
|
48
1. This limit has not been tested
Table 2: Label Switching Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
LDP sessions
|
200
Forwarding Equivalence Classes (FECs)
|
1,000
Equal-cost multipaths (ECMPs)
|
8
Feature
|
Verified Limit
---|---
FECs ECMPs
|
4,000
IAS option B labels
|
450,000
Layer 3 VPN routes
|
100,000
ECMPs
|
2,000
Table 3: Layer 2 Switching Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
MAC addresses (default template)
|
196,000
MAC addresses (L2-scale template)
|
384,000
MST instances
|
64
MST virtual ports
|
218,185
RPVST virtual ports
|
13,750
VLANs
|
3,967
VLANs in RPVST mode
|
250
Table 4: Layer 3 Multicast Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
IPv4 multicast routes
|
32,000 (Layer 3)
Outgoing interfaces (OIFs)
|
16 OIFs for 32K mroutes or 287 OIFs for 1000 mroutes
PIM neighbors
|
500
SVI
|
50-60
MVPN- unidimensional
Multicast VRFs
|
100 (N3K-C3636C-R and N3K-C36180YC-R)
Default MDT groups
|
100 (N3K-C3636C-R and N3K-C36180YC-R)
MVPN Peers (PIM neighbors) per device
|
100 (N3K-C3636C-R and N3K-C36180YC-R)
Table 5: Security Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
IPv4 ingress access control entries (ACEs)
|
RACL-2000, PACL-1024 (without TCAM Carving)
IPv6 ingress access control entries (ACEs)
|
RACL-1000, PACL-1024 (without TCAM Carving)
Feature
|
Verified Limit
---|---
ACL
|
12,000 (with TCAM Carving)
Egress ACLs
|
20,000 (N3K-C3636C-R and N3K-C36180YC-R)
RACLs
|
4,000 (N3K-C3636C-R and N3K-C36180YC-R)
System ACLs
|
4,000 TCAM entries in internal TCAM and 64,000 TCAM entries in external TCAM (N3K-C3636C-R and N3K-C36180YC-R)
Table 6: System Management Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
SPAN and ERSPAN
Configurable SPAN or ERSPAN sessions
|
32
Active SPAN or ERSPAN sessions
|
32
Active localized SPAN or ERSPAN session per line card
|
32 sessions across ports on single line card
Active localized SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)
|
32 sessions, 128 sources and 1 destination
Destination interfaces per SPAN session
|
1
Source VLANs per SPAN or ERSPAN
|
6
Table 7: Layer 3 Unicast Routing Verified Scalability Limits (Unidimensional) – For Default system routing template
Feature
|
Verified Limit
---|---
BFD sessions (echo mode)
|
288
100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)
BGP neighbors
|
256
HSRP groups
|
498
IPv4 ARP
|
75,000
IPv4 host routes
|
750,000
IPv6 host routes
|
62,000
IPv6 ND
|
32,000
IPv4 unicast routes (LPM)
|
192,000
IPv6 unicast routes (LPM)
|
62,000
OSPFv2 neighbors
|
1,000
Feature
|
Verified Limit
---|---
OSPFv3 neighbors
|
1,000
OSPF/OSPFv3 LSA/LSDB size
|
250,000
OSPF/OSPFv3 areas
|
15
VRFs
|
3,967
VRRP
VRRP groups per interface or I/O module
|
15
Table 8: Layer 3 Unicast Routing Verified Scalability Limits (Unidimensional) – For Internet-peering system routing template
Feature
|
Verified Limit
---|---
Routes (internet-peering mode)
|
852000
IPv4 routes (internet-peering mode)
|
781000
IPv6 routes (internet-peering mode)
|
71000
Table 9: HSRP Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit
---|---
Groups with default timers (3s/10s) and multiple group optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]
|
7,928
Groups with aggressive timers (1s/3s) and multiple groups optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]2
|
7,928
Groups per interface or I/0 module
|
Maximum 16 (Because 16 is the unique virtual MAC address limit)
2 If the user has Multi-protocol configuration, user should configure appropriate COPP policies so as to avoid any control plane traffic drops.
Table 10: VXLAN Verified Scalability Limits (Unidimensional)
Feature
|
Verified Limit 3
---|---
IGMP snooping over VXLAN
VXLAN VLANs
|
1,000
VTEP peers
|
256
Underlay multicast groups
|
128
Feature
|
Verified Limit 3
---|---
VXLAN Flood and Learn
Virtual network identifiers (VNIs) or VXLAN-mapped VLANs
|
Not applicable
Virtual network identifiers (VNIs) or VXLAN-mapped VLANs
|
Not applicable
Underlay multicast groups.
|
Not applicable
Overlay MAC addresses
|
Not applicable
Remote VXLAN tunnel endpoints (VTEPs)
|
Not applicable
Ingress replication peers
|
Not applicable
Ingress replication Layer 2 VNIs
|
Not applicable
MAC addresses for ingress replication
|
Not applicable
Port VLAN translations under an interface
|
Not applicable
Port VLAN translations in a switch
|
Not applicable
Static MAC addresses pointing to a remote VTEP
|
Not applicable
VXLAN VLAN logical port VP count
|
Not applicable
VXLAN VLANs per FEX port (host interface)
|
Not applicable
Layer 2 routed VNIs for vPC-centralized gateway
|
Not applicable
IGMP groups
|
Not applicable
VXLAN BGP eVPN
Layer 2 VNIs
|
2,000
Xconnect VLANs
|
Not applicable
SVI with Distributed Anycast Gateway; Layer 2 VNI extended
|
2,000
Layer 3 VNIs / VRFs
|
900
Underlay multicast groups
|
128
VTEPs
|
256
MAC addresses
|
90,000
IPv4 host routes
|
350,000
IPv6 host routes
|
48,000
Overlay IPv4 LPM routes
|
180,000
Overlay IPv6 LPM routes
|
48,000
Feature
|
Verified Limit 3
---|---
VXLAN VLAN logical port VP count
|
Not applicable
VXLAN VLANs per FEX port (host interface)
|
Not applicable
IGMP groups
|
8192
VXLAN BGP eVPN Ingress Replication
Layer 2 VNIs
|
Not applicable
Xconnect VLANs
|
Not applicable
SVI with Distributed Anycast Gateway; Layer 2 VNI extended
|
Not applicable
Layer 3 VNIs / VRFs
|
Not applicable
VTEPs
|
Not applicable
MAC addresses
|
Not applicable
IPv4 host routes
|
Not applicable
IPv6 host routes
|
Not applicable
Overlay IPv4 LPM routes
|
Not applicable
Overlay IPv6 LPM routes
|
Not applicable
VXLAN VLAN logical port VP count
|
Not applicable
VXLAN VLANs per FEX port (host interface)
|
Not applicable
IGMP groups
|
Not applicable
3. For Cisco Nexus 3636C-R and Cisco Nexus 36180YC-R switches
Verified Scalability Limits (Multidimensional)
The tables in this section list the verified scalability limits for Cisco NX- OS Release 9.3(12). These limits are validated with a multidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.
Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.
Attention
These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the “Verified Scalability Limits” section.
Table 11: MSDC Profile Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
Number of 100G ports
|
6
36 (N3K-C3636C-R)
vPC port channels
|
10
ISIS IPv4 /32 unicast routes
|
1,291
ISIS IPv6 /128 unicast routes
|
1,291
Multicast IPv4 SSM
|
10,000
VRF IPv4/IPv6
|
100
PIM neighbors
|
100
IGMP snooping database entries
|
240
VRRP v4 and v6
|
1,000 vlans
Multicast SSM
|
10,000
HSRP v4 and v6
|
1,000 vlans
SVI
|
100 (N3K-C3636C-R)
Sub-interfaces
|
100 (N3K-C3636C-R)
MAC
|
1000 (N3K-C3636C-R)
BGP IPv4/IPv6 VLSM routes
|
1000 (N3K-C3636C-R)
BGP IPv4/IPv6 Unicast routes
|
10,000 (N3K-C3636C-R)
ECMP
|
16-way Upstream (N3K-C3636C-R)
SPAN sessions
|
1 local SPAN session (N3K-C3636C-R)
Table 12: MPLS Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
MPLS Layer 3 VPN
|
3,715
VPE
|
3,715
PE nodes
|
Nil
PE routes
|
Nil
ACL (IPv4)
|
1,100
ACL (IPv6)
|
440
Feature
|
Verified Limit
---|---
HSRP and IPv6 VIP
|
3.715 each for v4 and v6
vPC uRPF
|
Nil
Strict uRPF
|
Yes
VRF
|
3,715
SVI
|
3,715
Layer 3 VPN routes IP ECMP
|
<500
MPLS LSR ECMP
|
<500
VPN IPv4 routes
|
65,000
VPN IPv6 routes
|
25,000
EBGP neighbors
|
Nil
Table 13: Layer 2/Layer 3 TOR Boundary Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
ECMP
|
16-way (Upstream)
vPC port channels
|
44
OSPFv2 neighbors
|
16
OSPFv3 neighbors
|
16
OSPF IPv4 /32 unicast routes
|
45,000
OSPF IPv4 VLSM unicast routes
|
1,000
OSPF IPv6 /128 unicast routes
|
25,000
OSPF IPv6 VLSM unicast routes
|
1,000
BFD sessions
|
230
100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)
VLAN
|
1,250
SVI
|
1,000
1250 (N3K-C3636C-R)
Sub-interfaces
|
250 per interface and 500 across the system (N3K-C3636C-R)
VRRP IPv4 groups
|
1,000 VRRS / 8 VRRPv3
Feature
|
Verified Limit
---|---
VRRP IPv6 groups
|
1,000 VRRS / 8 VRRPv3
PIM neighbors
|
230
IPv4 (*,G) multicast routes
|
300
IPv4 (S,G) multicast routes
|
2,320
IGMP snooping database entries
|
6,300
Sflow enabled interfaces
|
63
45 (N3K-C3636C-R)
UDLD enabled interfaces
|
65
48 (N3K-C3636C-R)
SPAN sessions
|
1 local SPAN session
MVR VLANs
|
250
MVR receiver ports
|
10
MVR multicast groups
|
1,000
MAC
|
20,000 (N3K-C3636C-R)
Q-in-Q tunnel ports
|
26
RSTP VLANS (tunneled over L2PT)
|
3,960
Table 14: Layer 2/Layer 3 Spine Boundary (for N3K-C3636C-R) Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
Number of 100G ports
|
36
Number of 10G ports
|
36 x 4 (Breakout)
ECMP
|
16-way (Upstream)
vPC port channels
|
40
OSPFv2 neighbors
|
100
OSPFv3 neighbors
|
100
OSPF IPv4 /32 unicast routes
|
45,000
OSPF IPv4 VLSM unicast routes
|
1,000
OSPF IPv6 /128 unicast routes
|
25,000
OSPF IPv6 VLSM unicast routes
|
1,000
Feature
|
Verified Limit
---|---
BFD sessions
|
280
100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)
VLAN
|
3,967
SVI
|
3,967
Sub-interfaces
|
250 per interface and 511 across system
VRRP IPv4 groups
|
1,996 VRRS / 4 VRRPv3
VRRP IPv6 groups
|
1,996 VRRS / 4 VRRPv3
HSRP IPv4
|
1,743 Secondary Groups / 7 Primary Groups
HSRP IPv6
|
1,743 Secondary Groups / 7 Primary Groups
PIM neighbors
|
230
IPv4 (*,G) multicast routes
|
2,000
IPv4 (S,G) multicast routes
|
30,000
IGMP snooping database entries
|
6,300
sFlow enabled interfaces
|
45
UDLD enabled interfaces
|
48
SPAN sessions
|
1 local SPAN session
MAC
|
50,000
Table 15: Segment Routing Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
LACP
|
26
LACP members
|
1 or 4
eBGP IPv6 neighbors
|
25
eBGP IPv4 LU neighbors
|
24
IPv4 (LU) routes
|
1,537
IPv4 (LU) paths
|
6,987
IPv6 routes
|
1,486
IPv6 paths
|
6,915
Feature
|
Verified Limit
---|---
SR ECMP (max)
|
18
MPLS HW entries
|
6,868
Table 16: Segment Routing (for N3K-C3636C-R) Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
VLAN
|
100
SVI
|
100
MAC entries
|
10,000
ARP entries
|
70
HSRPv4, HSRPv6 VIPs
|
100, 100
LACP
|
3
LACP members
|
4
eBGP IPv6 neighbors
|
2
eBGP IPv4 neihbors
|
2
IPv4 (LU) routes
|
6,848
IPv4 (LU) paths
|
8,187
IPv6 routes
|
6,640
IPv6 paths
|
7,975
SR ECMP
|
2
MPLS HW entries
|
2,682
Table 17: VXLAN Profile Verified Scalability Limits (Multidimensional)
Feature
|
Verified Limit
---|---
Number of ports
|
16
ECMP
|
8-way (Upstream)
BGP neighbors
|
2
BGP EVPN Layer 2 VPN host routes
|
60,000
BGP IPv4 VLSM unicast routes or ospf
|
10,000
BGP IPv6 VLSM unicast routes or ospf
|
2,000
Feature
|
Verified Limit
---|---
BFD sessions
|
10
PIM neighbors
|
10
IPv4 (*,G) multicast routes (co-existing)
|
4,000
IPv4 (S,G) multicast routes (co-existing)
|
2,000
Layer 3 VNI
|
100
Layer 2 VNI
|
400
Local VTEP
|
1
Remote VTEPs
|
205
VLAN
|
400
SVI
|
100
MAC
|
80,000
vPC hosts
|
1
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE- NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
This product includes cryptographic software written by Eric Young ([email protected]).
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (https://www.openssl.org/) This product includes software written by Tim Hudson ([email protected]).
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:
https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)
© 2023 Cisco Systems, Inc. All rights reserved.
Americas Headquarters
Cisco Systems, Inc.
San Jose, CA 95134-1706 USA
Asia Pacific Headquarters CiscoSystems(USA)Pte.Ltd. Singapore
Europe Headquarters CiscoSystemsInternationalBV Amsterdam,TheNetherlands
FAQ
Q: Can I exceed the verified scalability limits mentioned in the guide?
A: It is not recommended to exceed the verified limits as it may impact the performance and stability of the system. Future software releases may increase these limits based on further testing.
Q: Are the limits provided applicable to both N3K-C3636C-R and N3K-
C36180YC-R platforms?
A: The limits specified in this guide are primarily for the N3K-C36180YC-R TOR platform. Specific values for N3K-C3636C-R are indicated where applicable.