cisco HyperFlex Hyper Converged Infrastructure Instructions
- June 5, 2024
- Cisco
Table of Contents
- cisco HyperFlex Hyper-Converged Infrastructure
- Recommended Upgrade Method
- Upgrading Your HyperFlex Cluster Using the HX Connect UI
- Post vCenter Upgrade Tasks
- Online Upgrade Process Workflow
- Offline Upgrade Process Workflow
- Offline Upgrade Guidelines
- Offline Upgrade Using CLI
- stcli cluster upgrade –components ucs-fw, hxdp, hypervisor –location/tmp/
- stcli cluster upgrade –components hxdp,hypervisor –location
- stcli cluster upgrade –components hxdp,ucs-fw –location/tmp/ <storfs package
- stcli cluster upgrade –components hxdp,ucs-fw –location/tmp/ <storfs package
- Start Cluster and Power On VMs
- stcli cluster start
- Read User Manual Online (PDF format)
- Download This Manual (PDF format)
cisco HyperFlex Hyper-Converged Infrastructure
Recommended Upgrade Method
For both combined upgrade and split upgrade, Cisco recommends upgrading the HyperFlex components in the following order for optimizing the upgrade time:
Note
Make sure to upgrade the vCenter to the desired version based on the ESXi
version and recommendation from VMware before upgrading ESXi.
As part of the Server Firmware upgrade operation initiated from HX Connect, some of the UCS policies may be updated to be compatible with the new HXDP version. These changes are applied only to the nodes that are part of the cluster is upgraded. It is highly recommended to use HX Connect to initiate the Server Firmware upgrade to avoid any policy drift.
- Upgrade Cisco UCS Infrastructure
- Upgrade Cisco HX Data Platform
- Upgrade Cisco customized VMware ESXi
- Upgrade Cisco UCS firmware
Upgrading Your HyperFlex Cluster Using the HX Connect UI
Note
Hypercheck Health Check Utility— Cisco recommends running this proactive
health check utility on your HyperFlex cluster prior to upgrade. These checks
provide early visibility into any areas that may need attention and will help
ensure a seamless upgrade experience. For more information see the Hyperflex
Health & Pre-Upgrade Check Tool TechNote for full instructions on how to
install and run Hypercheck.
Important
Use the HX Connect UI when upgrading from a current HX Data Platform
version of 2.5(1a) or later releases.
Procedure
Step 1
If UCSM (A-bundle) or UCS Server Firmware (C-bundle) upgrade is required,
download Cisco UCS Infrastructure A, blade bundle B, and rack bundle C. See
Downloading Software for more details.
Step 2
Ensure that the hx-storage-data and vMotion upstream switches are
configured for full network failover capability before proceeding forward.
Otherwise, the HyperFlex Cluster becomes offline and all datastores unmount
from the ESXi hosts. See Test Upstream Network Connectivity for more details.
Step 3
Upgrade Cisco UCS Infrastructure bundle as required. See Upgrading Cisco
UCS Infrastructure Using Cisco UCS Manager for more details.
Note
It is important that you manually upgrade the UCS infrastructure first
before initiating the upgrade sequence of the HyperFlex components as outlined
in Recommended Upgrade Method, on page 1. The upgrade feature of the HX
Platform software will not upgrade the UCS infrastructure bundle. This upgrade
is a manual process.
Step 4
Bootstrap to upgrade Cisco HX Data Platform.
Note
If you are running HyperFlex release 3.5(1a) or later, you can upgrade the
Cisco HX Data Platform by performing the auto-bootstrap process from the HX
Connect UI (Auto Bootstrap Upgrade Process from HX Connect UI). However, if
you are running a HyperFlex release that is earlier than release 3.5(1a) you
must run the manual bootstrap process to upgrade the Cisco HX Data Platform
(Manual Bootstrap Upgrade Process).
Step 5
Log in to HX Connect.
- Enter the HX Storage Cluster management IP address in a browser. Navigate to https://storage-cluster-management-ip.
- Enter the administrative username and password.
- Click Login.
Step 6
In the Navigation pane, select Upgrade.
Step 7
Choose the type of upgrade from the Select Upgrade Type page.
Caution
After manual bootstrap, validation will fail if you perform UCS only, ESXi
only, or UCS and ESXi combined upgrade. For a successful upgrade, Cisco
recommends the following upgrade types:
- HX Data Platform only upgrade, followed by UCS Firmware and/or Hypervisor Software upgrade
- HX Data Platform and UCS Firmware
- HX Data Platform and Hypervisor Software
- HX Data Platform, UCS Firmware, and Hypervisor Software.
Step 8
Depending on the type of upgrade you want to perform, complete the
following fields on the Enter Credentials tab.
UCS Server Firmware
Field | Essential Information |
---|---|
UCS Manager Hostname field | Enter the Cisco UCS Manager FQDN or IP |
address. Example:
10.193.211.120.
User Name field| Enter the Cisco UCS Manager _< admin> username.
Admin Password field| Enter the Cisco UCS Manager < admin> _password.
Discover button| Click Discover to view the current UCS firmware
package version, in the Current Version field.
HX Data Platform
UI Element | Essential Information |
---|---|
Drag the HX file here or click to browse | Upload the latest _Cisco |
HyperFlex Data Platform Upgrade Bundle for upgrading existing clusters with previous release.tgz_ package file from Download Software – HyperFlex HX Data Platform.
Sample file name format: storfs-packages-3.5.2a-31601.tgz.
Current version| Displays the current HyperFlex Data Platform version.
Current cluster details| Lists the HyperFlex cluster details like the
HyperFlex version and
Cluster upgrade state.
Bundle version| Displays the HyperFlex Data Platform version of the
uploaded bundle.
(Optional) Checksum field| The MD5 Checksum number is stored in a
separate text file at the /tmp
directory where the upgrade package was downloaded.
This is an optional step that helps you verify the integrity of the uploaded upgrade package bundle.
ESXi
Note
The ESXi upgrade option is supported in the HyperFlex Connect UI for
HyperFlex release 3.5(1a) or later.
UI Element | Essential Information |
---|---|
Drag the ESXi file here or click to browse field | Upload the latest |
Cisco HyperFlex Custom Image Offline Bundle for upgrading existing ESXi hosts from Download Software – HyperFlex HX Data Platform.
Example:
HX-ESXi-6.5U2-10884925-Cisco-Custom-6.5.2.4-upgrade-bundle.zip.
Current version field| Displays the current ESXi version.
Current hypervisor details field| Lists the HyperFlex cluster details
like the Hypervisor version and
Cluster upgrade state.
Bundle details field| Displays the ESXi version of the uploaded bundle.
vCenter Credentials
UI Element | Essential Information |
---|---|
User Name field | Enter the vCenter _< admin> _username. |
Admin Password field | Enter the vCenter _< admin> _password. |
Step 9
Click Upgrade to begin the cluster upgrade process.
Step 10
The Validation Screen on the Upgrade Progress page displays the progress of
the checks performed. Fix validation errors, if any. Confirm that the upgrade
is complete. When the upgrade is in progress, you may see an error message,
‘Websocket connection failed. Automatic refresh disabled’. You can either
refresh the page or log out and log back in to clear the error message. You
can safely ignore this error message. During cluster upgrade, if the
orchestration node reboots or power cycles due to a power issue, the cluster
upgrade will be stuck. Once the node is up, restart the cluster upgrade
process after cleaning the cluster system using the following command:
stcli cluster upgrades –components hxdp –clean.
If the clean-up command fails, restart the stMgr service on all control VMs
(ctrlVM) by running the following command:
restart stMgr
Then, clean the cluster system by rerunning the following command:
stcli cluster upgrade –components hxdp –clean
Here is a sample code:
- root@ucs-stctlvm-385-1:~# stcli cluster upgrade –clean –components hxdp
- Forcefully cleaned up upgrade progress
- root@ucs-stctlvm-385-1:~# stcli cluster upgrade –status
- No active upgrade was found. Upgrade progress is available after triggering an upgrade.
Post vCenter Upgrade Tasks
If the extension is not working, and HyperFLex and vCenter are upgraded to compatible versions, perform the following steps:
Note
If you have more than one HyperFlex cluster, then you must first update all HX
clusters to compatible HX versions for the corresponding vCenter versions
prior to attempting to reregister. Do not unregister
com.springpath.sysmgmt unless all clusters are removed from the vCenter.
Before you begin
Verify the extension is working. If so, then there is no need to perform any post-upgrade tasks.
Procedure
Step 1
Try to re-register the extension. If the extension still does not work, then
continue with the next steps.
Step 1
Unregister the extension.
Example:
com.spring path.sysmgmt.domain-
com.spring path.sysmgmt Use the mob browser https://
Note
We recommend removing the cluster before unregistering extensions.
Step 3
Re-register the Springpath plug-in using:
Example:
stcli cluster reregister
Note
You can use stcli cluster reregister –-for help and then continue with the
reregistration.
Online Upgrade Process Workflow
Attention
If you are running HyperFlex release 3.5(1a) or later, you can upgrade the
Cisco HX Data Platform by performing the auto-bootstrap process from the HX
Connect UI (Auto Bootstrap Upgrade Process from HX Connect UI). However, if
you are running a HyperFlex release that is earlier than release 3.5(1a) you
must run the manual bootstrap process to upgrade the Cisco HX Data Platform
(Manual Bootstrap Upgrade Process).
When using the online upgrade process workflow, consider the following:
- First, upgrade Cisco UCS infrastructure to the latest version and then use the automated upgrade workflow for a combined upgrade of Cisco UCS firmware and Cisco HX Data Platform. The online upgrade uses host firmware packages to upgrade all server endpoints.
- During the online upgrade, as one node is being upgraded (placed into maintenance mode), the number of tolerated node failures is reduced based on the Data Replication Factor and the Access Policy settings. See Entering Cisco HyperFlex Maintenance Mode for the procedures on how to access Cisco HyperFlex Maintenance Mode.
- If upgrading both HXDP and UCS firmware, the combined upgrade can be selected through HX Connect depending on the length of the maintenance window.
- Do not use the Firefox browser. It is not supported due to an outdated version of flash that is bundled with the browser.
Note
Cisco UCS Manager infrastructure upgrade is only supported using
AutoInstall and the direct server firmware upgrade should be performed only
through the upgrade orchestration framework provided by the HX Data Platform
Plug-in.
Note
During the online upgrade process, do not acknowledge server reboot pending
activities from UCS Manager. Doing so will interrupt the upgrade process and
can cause a storage outage. HyperFlex will automatically reboot each node.
The following table summarizes the online upgrade workflow:
Step | Description | Reference |
---|---|---|
1. | If UCSM (A-bundle) or UCS Server Firmware |
(C-bundle) the upgrade is required, download Cisco UCS Infrastructure A, blade bundle B, and rack bundle C.
| Downloading Software
2.| Ensure that the hx-storage-data and vMotion upstream switches are
configured for full network failover
capability before proceeding forward. Otherwise, the HyperFlex Cluster becomes offline and all datastores unmount from the ESXi hosts.
| Test Upstream Network Connectivity
3.| Upgrade Cisco UCS Infrastructure bundle as required.
Note **** It is important that you manually upgrade the UCS infrastructure first before initiating the upgrade sequence of the HyperFlex
components as outlined in Recommended Upgrade Method, on page 1 . The upgrade feature of the HX Platform software will not upgrade the UCS infrastructure bundle. This upgrade is a manual process.
| Upgrading Cisco UCS
Infrastructure Using Cisco UCS Manager
Step | Description | Reference |
---|---|---|
4. | Bootstrap to upgrade Cisco HX Data Platform. | HX Release 3.5(1a) and |
later:
• Auto bootstrap is supported beginning with HX release 3.5(1a). Auto Bootstrap
Upgrade Process from HX Connect UI
HX Release earlier than 3.5(1a):
• Manual cluster bootstrap is required HX releases earlier than 3.5(1a). Manual Bootstrap Upgrade Process
5.| Disable the snapshot schedule, on the bootstrapped storage controller VM.
Note **** It is enough to run this script on one of the controller nodes.
| Run the command stcli snapshot-schedule –disable.
6.| Log in to HX Connect with administrator credentials.|
7.| Start combined upgrade of:
• HX Data Platform and UCS Firmware
• HX Data Platform and Hypervisor Software
| Upgrading Your HyperFlex Cluster Using the HX Connect UI, on page 2
Attention To perform a split upgrade, you must upgrade HX Data Platform
first. After HX Data Platform is upgraded to 3.5(1x), you can perform a split
upgrade of UCSM only and/or ESXi only.
When only UCS firmware is being upgraded, you may see the upgrade process pause at the validation screen after the fabric interconnect discovery. It may be a network connectivity failure issue, however, in most cases, it just requires waiting for the process to finish.
| Upgrading Cisco HyperFlex Data Platform Using HX Connect
Upgrading Cisco UCS Server Firmware Using the HX Connect UI
Upgrading ESXi Using the HX Connect UI
8.| Confirm that the upgrade task is complete.| Post Upgrade Tasks
9.| Dynamic certificate creation.| Starting with 4.0(2a) release, Dynamic
self-signed certificate are generated rather than static
certificates.
10.| On the same controller VM, enable snapshot schedule.| Run the command stcli snapshot-schedule -–enable.
Offline Upgrade Process Workflow
The following table summarizes the offline upgrade workflow:
Step | Description | Reference |
---|---|---|
1. | If UCSM (A-bundle) or UCS Server Firmware (C-bundle) upgrade is required, | |
download Cisco UCS Infrastructure A, blade bundle B, and rack bundle C. |
Downloading Software
2.| Ensure that the hx-storage-data and vMotion upstream switches are
configured for full network failover capability before proceeding
forward. Otherwise the HyperFlex Cluster becomes offline and all datastores unmount from the ESXi hosts.
| Test Upstream Network Connectivity
3.| Upgrade Cisco UCS Infrastructure bundle as required.
Note **** It is important that you
manually upgrade the UCS infrastructure first before initiating the upgrade sequence of the HyperFlex components as outlined in Recommended Upgrade Method, on page 1.
The upgrade feature of the HX Platform software will not upgrade the UCS infrastructure bundle. This upgrade is a manual process.
| Upgrading Cisco UCS
Infrastructure Using Cisco UCS Manager
Step | Description | Reference |
---|---|---|
4. | Launch the vSphere Web Client and power down all user VMs |
residing on HX servers and all user VMs running on HX datastores.
This includes VMs running on computer-only nodes. After the VMs have been shut down, verify the health state of the cluster and perform a graceful shutdown.
Important HyperFlex controller
VMs (stCtlVMs) must remain powered on.
| Graceful Shutdown of a HX Cluster
5.| Manually stage the correct firmware version before starting the upgrade
process.| Modifying Host Firmware Package Using Cisco UCS Manager
6.| Shutdown the HyperFlex Controller VMs (stCtlVMs).| In vCenter, right-click
on each HX Controller VM (stCtlVM) and select Power > Shut Down Guest
OS.
7.| Once the Controller VMs are shutdown, place the ESXi hosts into
Maintenance Mode.| In vCenter, right-click on each ESXi host select
Maintenance Mode > Enter Maintenance Mode.
8.| Acknowledge the pending reboot on the servers that comprise your HX
cluster nodes, including both converged nodes and computer-only nodes
connected to the cluster.
Wait until all nodes are upgraded. Confirm that correct firmware packages have been installed before proceeding.
|
9.| Once the ESXi hosts have booted, take them out of Maintenance Mode. Now
the controller VM should come back online.| In vCenter, right-click on each
ESXi host select Maintenance
Mode > Exit Maintenance Mode.
Step | Description | Reference |
---|---|---|
10. | Bootstrap to upgrade the Cisco HX Data Platform Plug-in. |
Important **** • Be sure to copy the
bootstrap file to the controller VM
/tmp directory.
• Ensure that you confirm the version of the plug-in in the vCenter Administration > Client Plug-Ins page.
| Manual Bootstrap Upgrade Process
11.| Disable snapshot schedule, on the bootstrapped storage controller
VM.
Note **** It is enough to run this
script on one of the controller nodes.
| Run the command stcli snapshot-schedule –disable.
12.| From the same controller VM, begin the upgrade.| Upgrading Your HyperFlex
Cluster Using the HX Connect UI, on page 2
Offline Upgrade Using CLI, on page 11
13.| Confirm that upgrade is complete.| Post Upgrade Tasks
14.| After the upgrade is complete, start the cluster and power on VMs.| Start
Cluster and Power On VMs , on page 12
15.| On the same controller VM, enable snapshot schedule.| Run the command
stcli snapshot-schedule -–enable.
Offline Upgrade Guidelines
Important
ucsm-host and ucsm-user parameters are required when you are upgrading from 1.7x to 1.8x. These parameters must not be used when moving up from 1.8(1a)/1.8(1b) to 2.0(1a) as we are not changing the Cisco UCS server firmware version. Cisco recommends using HX Connect UI to upgrade HyperFlex clusters over CLI for ease of use and better reporting. You can safely ignore Cluster Invalid State alerts in HX Connect.
Before you proceed, with either a combined or split upgrade consider the following guidelines:
- The package name must match the file that you uploaded to the controller VM.
- Enter passwords when prompted.
- Nodes are upgraded with the new version of the Cisco HX Data Platform software and rebooted one at a time.
- Offline cluster upgrades with nested vCenter is not supported.
Offline Upgrade Using CLI
Important
If you need to perform a split upgrade, you must upgrade HX Data Platform
first. After HX Data Platform is upgraded to Release 3.5(1x), you can perform
a split upgrade of UCSM only and/or ESXi only.
Combined Upgrade of Cisco HX Data Platform, ESXi and Cisco UCS Firmware
M5 Servers
stcli cluster upgrade –components ucs-fw, hxdp, hypervisor –location/tmp/
<storfs package name,ESXi package name> –ucsm-host <IP/FQDN of UCSM> ucsm-user
Example for M5 Servers:
~# stcli cluster upgrade –components ucs-fw, hxdp, hypervisor –location /tmp
/storfs-packages-3.5.1a-19712.tgz –ucsm-host eng-fi16.eng.storvisor.com –ucsm-
user admin –ucs5fw-version ‘3.1(2g)’
M4 Servers
stcli cluster upgrade –components ucs-fw, hxdp, hypervisor –location/tmp/
<storfs package name, ESXi package name> –ucsm-host <IP/FQDN of UCSM> –ucsm-
user
Example for M4 Servers:
~# stcli cluster upgrade –components ucs-fw, hxdp, hypervisor –location
/tmp/storfs-packages-3.5.1a-19712.tgz
–ucsm-host eng-fi16.eng.storvisor.com –ucsm-user admin –ucsfw-version
‘3.1(2g)’
Combined Upgrade of Cisco HX Data Platform and ESXi
M5 Servers
stcli cluster upgrade –components hxdp,hypervisor –location
/tmp/hxupgrade_bundle.tgz –hypervisor-bundle /tmp/esxiupgrade_bundle.zip
Example for M5 Servers:
stcli cluster upgrade –components hxdp,hypervisor –location
/tmp/hxupgrade_bundle.tgz
- hypervisor-bundle /tmp/esxiupgrade_bundle.zip
M4 Servers
stcli cluster upgrade –components hxdp,hypervisor –location
/tmp/hxupgrade_bundle.tgz –hypervisor-bundle /tmp/esxiupgrade_bundle.zip
Example for M4 Servers:
~# stcli cluster upgrade –components hxdp,hypervisor –location /tmp/hxupgrade_bundle.tgz
- hypervisor-bundle /tmp/esxiupgrade_bundle.zip
Combined Upgrade of Cisco HX Data Platform and Cisco UCS Firmware
M5 Servers
stcli cluster upgrade –components hxdp,ucs-fw –location/tmp/ <storfs package
name> –vcenter-user
M4 Servers
stcli cluster upgrade –components hxdp,ucs-fw –location/tmp/ <storfs package
name> –vcenter-user
Example for M4 Servers:
~# stcli cluster upgrade –components hxdp,ucs-fw –location /tmp/storfs-
packages-1.8.1c-19712.tgz –vcenter-user
administrator@vsphere.local –ucsm-host
eng-fi16.eng.storvisor.com –ucsm-user admin –ucsfw-version ‘3.1(2b)’.
Start Cluster and Power On VMs
After the upgrade is complete and the cluster has been upgraded, log out and log back in to vCenter to see upgrade changes.
Procedure
Step 1
After the upgrade is complete, start your cluster.
Step 2
Login to any controller VM through SSH.
stcli cluster start
Example:
HyperFlex StorageController 1.8(1c) Last login: Wed Sept 21 23:54:23 2016 from
pguo-dev.eng.storvisor.com root@ucs-stclivm – 384 -1;~# stcli cluster upgrade-
status Cluster upgrade succeeded. Cluster version: 1.8(1c) root@ucs-
stctlvm-384;~# stcli cluster start waiting for Cluster to start on nodes:
[ucs-383, ucs-384, ucs-385, ucs-386]
This will start the cluster and mount the HX datastores. Wait for the
cluster to come online. You will see the prompt:
Started cluster on nodes; [ucs-383, ucs-384, ucs-385, ucs-386] Cluster is
online root@ucs-stctlvm-384-1;~#.
Step 3
Wait for cluster to become healthy before starting the VMs. Run command:
~# stcli clustr info| grep health
Example:
root@SpringpathControllerZRVF040451;~# stcli cluster info | grep health state:
healthy.
state: healthy
storage cluster is healthy
Step 4
After the cluster is healthy, launch vSphere Web Client or Thick Client,
navigate to Hosts and Cluster > Datacenter > Cluster > . Right-click, select
Power> Power On to start the VMs.
Read User Manual Online (PDF format)
Read User Manual Online (PDF format) >>