Single Node Upgrade

The following instructions will guide you through upgrading a high-availability Rancher server installation.

Prerequisites

Review the Known Upgrade Issues and Caveats
Air Gap Installs Only: Collect and Populate Images for the new Rancher server version

Follow the guide to populate your private registry with the images for the Rancher version that you want to upgrade to.

Placeholder Review

During upgrade, you’ll enter a series of commands, filling placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (<EXAMPLE>).

Here’s an example of a command with a placeholder:

docker stop <RANCHER_CONTAINER_NAME>

In this command, <RANCHER_CONTAINER_NAME> is the name of your Rancher container.

Cross reference the image and reference table below to learn how to obtain this placeholder data. Write down or copy this information before starting the upgrade.

^{Terminal docker ps Command, Displaying Where to Find <RANCHER_CONTAINER_TAG> and <RANCHER_CONTAINER_NAME>}

Placeholder	Example	Description
`<RANCHER_CONTAINER_TAG>`	`v2.1.3`	The rancher/rancher image you pulled for initial install.
`<RANCHER_CONTAINER_NAME>`	`festive_mestorf`	The name of your Rancher container.
`<RANCHER_VERSION>`	`v2.1.3`	The version of Rancher that you’re creating a backup for.
`<DATE>`	`2018-12-19`	The date that the data container or backup was created.

You can obtain <RANCHER_CONTAINER_TAG> and <RANCHER_CONTAINER_NAME> by logging into your Rancher Server by remote connection and entering the command to view the containers that are running: docker ps. You can also view containers that are stopped using a different command: docker ps -a. Use these commands for help anytime during while creating backups.

Upgrade Outline

During upgrade, you create a copy of the data from your current Rancher container and a backup in case something goes wrong. Then you deploy the new version of Rancher in a new container using your existing data. Follow the steps to upgrade Rancher server:

A. Create a copy of the data from your Rancher server container
B. Create a backup tarball Get the options set from your current Rancher install
C. Upgrade Rancher
D. Verify the Upgrade
E. Clean up your old Rancher server container

A. Create a copy of the data from your Rancher server container

Using a remote Terminal connection, log into the node running your Rancher Server.
Stop the container currently running Rancher Server. Replace <RANCHER_CONTAINER_NAME> with the name of your Rancher container.
```
docker stop <RANCHER_CONTAINER_NAME>
```
Use the command below, replacing each placeholder, to create a data container from the Rancher container that you just stopped.
```
docker create --volumes-from <RANCHER_CONTAINER_NAME> --name rancher-data rancher/rancher:<RANCHER_CONTAINER_TAG>
```

B. Create a backup tarball

From the data container that you just created (rancher-data), create a backup tarball (rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz).

This tarball will serve as a rollback point if something goes wrong during upgrade. Use the following command, replacing each placeholder.
```
docker run --volumes-from rancher-data -v $PWD:/backup busybox tar zcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
```
Step Result: When you enter this command, a series of commands should run.
Enter the ls command to confirm that the backup tarball was created. It will have a name similar to rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz.
```
[rancher@ip-10-0-0-50 ~]$ ls
rancher-data-backup-v2.1.3-20181219.tar.gz
```
Move your backup tarball to a safe location external from your Rancher Server.

C. Upgrade Rancher

Pull the image of the Rancher version that you want to upgrade to.

Placeholder Description

<RANCHER_VERSION_TAG> The release tag of the Rancher version that you want to upgrade to.
```
docker pull rancher/rancher:<RANCHER_VERSION_TAG>
```
Start a new Rancher server container using the data from the rancher-data container. Remember to pass in all the environment variables that you had used when you started the original container.
Note: After upgrading Rancher Server, data from your upgraded server is now saved to the rancher-data container for use in future upgrades.

Important: Do not stop the upgrade after initiating it, even if the upgrade process seems longer than expected. Stopping the upgrade may result in database migration errors during future upgrades.

Did you…
- Use a proxy? See HTTP Proxy Configuration
- Configure custom CA root certificate to access your services? See Custom CA root certificate
- Record all transactions with the Rancher API? See API Auditing
Choose from the following options:
- Single Node Upgrade
- Single Node Upgrade for Air Gap Installs

Select which option you had installed Rancher server

Option A-Default Self-Signed Certificate

If you have selected to use the Rancher generated self-signed certificate, you add the –volumes-from rancher-data to the command that you had started your original Rancher server container.

Placeholder	Description
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.

 docker run -d --volumes-from rancher-data \
  --restart=unless-stopped \
    -p 80:80 -p 443:443 \
    rancher/rancher:<RANCHER_VERSION_TAG>

Option B-Bring Your Own Certificate: Self-Signed

If you have selected to bring your own self-signed certificate, you add the –volumes-from rancher-data to the command that you had started your original Rancher server container and need to have access to the same certificate that you had originally installed with.

Reminder of the Cert Prerequisite: The certificate files must be in PEM format. In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see SSL FAQ / Troubleshooting.

Placeholder	Description
`<CERT_DIRECTORY>`	The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>`	The path to your full certificate chain.
`<PRIVATE_KEY.pem>`	The path to the private key for your certificate.
`<CA_CERTS>`	The path to the certificate authority’s private key.
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.

  docker run -d --volumes-from rancher-data \
    --restart=unless-stopped \
    -p 80:80 -p 443:443 \
    -v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
    -v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
    -v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
    rancher/rancher:<RANCHER_VERSION_TAG>

Option C-Bring Your Own Certificate: Signed by Recognized CA

If you have selected to use a certificate signed by a recognized CA, you add the –volumes-from rancher-data to the command that you had started your original Rancher server container and need to have access to the same certificates that you had originally installed with. Remember to include –no-cacerts as an argument to the container to disable the default CA certificate generated by Rancher.

Reminder of the Cert Prerequisite: The certificate files must be in PEM format. In your certificate file, include all intermediate certificates provided by the recognized CA. Order your certificates with your certificate first, followed by the intermediates. For an example, see SSL FAQ / Troubleshooting.

Placeholder	Description
`<CERT_DIRECTORY>`	The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>`	The path to your full certificate chain.
`<PRIVATE_KEY.pem>`	The path to the private key for your certificate.
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.

  docker run -d --volumes-from rancher-data \
    --restart=unless-stopped \
    -p 80:80 -p 443:443 \
    -v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
    -v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
    rancher/rancher:<RANCHER_VERSION_TAG> \
    --no-cacerts

Option D-Let's Encrypt Certificate

Remember: Let’s Encrypt provides rate limits for requesting new certificates. Therefore, limit how often you create or destroy the container. For more information, see Let’s Encrypt documentation on rate limits.

If you have selected to use Let’s Encrypt certificates, you add the --volumes-from rancher-data to the command that you had started your original Rancher server container and need to provide the domain that you had used when you originally installed Rancher.

Reminder of the Cert Prerequisites:

Create a record in your DNS that binds your Linux host IP address to the hostname that you want to use for Rancher access (rancher.mydomain.com for example).

Open port TCP/80 on your Linux host. The Let’s Encrypt http-01 challenge can come from any source IP address, so port TCP/80 must be open to all IP addresses.

Placeholder	Description
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.
`<YOUR.DNS.NAME>`	The domain address that you had originally started with

  docker run -d --volumes-from rancher-data \
    --restart=unless-stopped \
    -p 80:80 -p 443:443 \
    rancher/rancher:<RANCHER_VERSION_TAG> \
    --acme-domain <YOUR.DNS.NAME>

For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.

Did you…

Configure custom CA root certificate to access your services? See Custom CA root certificate.

Record all transactions with the Rancher API? See API Auditing.

For Rancher versions from v2.2.0 to v2.2.x, you will need to mirror the system-charts repository to a location in your network that Rancher can reach. Then, after Rancher is installed, you will need to configure Rancher to use that repository. For details, refer to the documentation on setting up the system charts for Rancher prior to v2.3.0.

Choose from the following options:

Option A-Default Self-Signed Certificate

If you have selected to use the Rancher generated self-signed certificate, you add the –volumes-from rancher-data to the command that you had started your original Rancher server container.

Placeholder	Description
`<REGISTRY.YOURDOMAIN.COM:PORT>`	Your private registry URL and port.
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to to upgrade to.

  docker run -d --volumes-from rancher-data \
      --restart=unless-stopped \
      -p 80:80 -p 443:443 \
      -e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
      -e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
      <REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>

Option B-Bring Your Own Certificate: Self-Signed

Reminder of the Prerequisite: The certificate files must be in PEM format. In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see SSL FAQ / Troubleshooting.

Placeholder	Description
`<CERT_DIRECTORY>`	The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>`	The path to your full certificate chain.
`<PRIVATE_KEY.pem>`	The path to the private key for your certificate.
`<CA_CERTS>`	The path to the certificate authority’s private key.
`<REGISTRY.YOURDOMAIN.COM:PORT>`	Your private registry URL and port.
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.

  docker run -d --restart=unless-stopped \
      -p 80:80 -p 443:443 \
      -v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
      -v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
      -v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
      -e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
      -e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
      <REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>

Option C-Bring Your Own Certificate: Signed by Recognized CA

Reminder of the Prerequisite: The certificate files must be in PEM format. In your certificate file, include all intermediate certificates provided by the recognized CA. Order your certificates with your certificate first, followed by the intermediates. For an example, see SSL FAQ / Troubleshooting.

Placeholder	Description
`<CERT_DIRECTORY>`	The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>`	The path to your full certificate chain.
`<PRIVATE_KEY.pem>`	The path to the private key for your certificate.
`<REGISTRY.YOURDOMAIN.COM:PORT>`	Your private registry URL and port.
`<RANCHER_VERSION_TAG>`	The release tag of the Rancher version that you want to upgrade to.

Note: Use the --no-cacerts as argument to the container to disable the default CA certificate generated by Rancher.

  docker run -d --volumes-from rancher-data \
      --restart=unless-stopped \
      -p 80:80 -p 443:443 \
      --no-cacerts \
      -v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
      -v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
      -e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
      -e CATTLE_SYSTEM_CATALOG=bundled \ #Available as of v2.3.0, use the packaged Rancher system charts
      <REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>

D. Verify the Upgrade

Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window.

Having network issues in your user clusters following upgrade?

See Restoring Cluster Networking.

E. Clean up your old Rancher server container

Remove the previous Rancher Server container. If you only stop the previous Rancher Server container (and don’t remove it), the container may restart after the next server reboot.

Rolling Back

If your upgrade does not complete successfully, you can roll back Rancher server and its data back to its last healthy state. For more information, see Single Node Rollback.

Illumina