Skip to content

Commit

Permalink
More docs adjustments based on recent feedback (#437)
Browse files Browse the repository at this point in the history
  • Loading branch information
@akrzos
akrzos authored Feb 19, 2024
1 parent de7ccec commit 5bb9aff
Show file tree
Hide file tree
Showing 5 changed files with 62 additions and 30 deletions.
37 changes: 31 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Three separate layouts of clusters can be deployed:
* RWN - Remote Worker Node - 3 control-plane/worker nodes, X number of remote worker nodes
* SNO - Single Node OpenShift - 1 OpenShift Master/Worker Node "cluster" per available hardware resource

Each cluster layout requires a bastion machine which is the first machine out of your lab "cloud" allocation. The bastion machine will host the assisted-installer and serve as a router for clusters with a private machine network. BM and RWN layouts produce a single cluster consisting of 3 control-plane nodes and X number of worker or remote worker nodes. SNO layout creates an SNO cluster per available machine after fulfilling the bastion machine requirement. Lastly, BM/RWN cluster types will allocate any unused machines under the `hv` ansible group which stands for hypervisor nodes. The `hv` nodes can host vms for additional clusters that can be deployed from the hub cluster. (For ACM/MCE testing)
Each cluster layout requires a bastion machine which is the first machine out of your lab "cloud" allocation. The bastion machine will host the assisted-installer service and serve as a router for clusters with a private machine network. BM and RWN layouts produce a single cluster consisting of 3 control-plane nodes and X number of worker or remote worker nodes. The worker node count can also be 0 such that your bare metal cluster is a compact 3 node cluster with schedulable control-plane nodes. SNO layout creates an SNO cluster per available machine after fulfilling the bastion machine requirement. Lastly, BM/RWN cluster types will allocate any unused machines under the `hv` ansible group which stands for hypervisor nodes. The `hv` nodes can host vms for additional clusters that can be deployed from the hub cluster. (For ACM/MCE testing)

_**Table of Contents**_

Expand All @@ -24,7 +24,7 @@ _**Table of Contents**_

## Tested Labs/Hardware

The listed hardware has been used for cluster deployments successfully. Potentially other hardware has been tested but just not documented here.
The listed hardware has been used for cluster deployments successfully. Potentially other hardware has been tested but not documented here.

**Alias Lab**

Expand Down Expand Up @@ -60,22 +60,23 @@ Versions:
* Ansible 4.10+ (core >= 2.11.12) (on machine running jetlag playbooks)
* ibmcloud cli => 2.0.1 (IBMcloud environments)
* ibmcloud plugin install sl (IBMcloud environments)
* RHEL 8.6 / Rocky 8.6 (Bastion)
* RHEL >= 8.6 (Bastion)
* podman 3 / 4 (Bastion)

Update to RHEL 8.7
Update to RHEL 8.9

```console
[root@xxx-xxx-xxx-r640 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.2 (Ootpa)

[root@xxx-xxx-xxx-r640 ~]# ./update-latest-rhel-release.sh 8.7
[root@xxx-xxx-xxx-r640 ~]# ./update-latest-rhel-release.sh 8.9
...
[root@xxx-xxx-xxx-r640 ~]# dnf update -y
...
[root@xxx-xxx-xxx-r640 ~]# reboot
...
[root@xxx-xxx-xxx-r640 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.7 (Ootpa)
Red Hat Enterprise Linux release 8.9 (Ootpa)
```

Installing Ansible via bootstrap (requires python3-pip)
Expand Down Expand Up @@ -159,6 +160,30 @@ Single Node OpenShift:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook -i ansible/inventory/cloud99.local ansible/sno-deploy.yml
```

Interact with your cluster from your bastion machine:

```console
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# export KUBECONFIG=/root/bm/kubeconfig
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# oc get no
NAME STATUS ROLES AGE VERSION
xxx-h02-000-r650 Ready control-plane,master,worker 73m v1.25.7+eab9cc9
xxx-h03-000-r650 Ready control-plane,master,worker 103m v1.25.7+eab9cc9
xxx-h05-000-r650 Ready control-plane,master,worker 105m v1.25.7+eab9cc9
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cat /root/bm/kubeadmin-password
xxxxx-xxxxx-xxxxx-xxxxx
```

And for SNO

```console
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# export KUBECONFIG=/root/sno/xxx-h02-000-r650/kubeconfig
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# oc get no
NAME STATUS ROLES AGE VERSION
xxx-h02-000-r650 Ready control-plane,master,worker 30h v1.28.6+0fb4726
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cat /root/sno/xxx-h02-000-r650/kubeadmin-password
xxxxx-xxxxx-xxxxx-xxxxx
```

## Quickstart guides

* [Deploy a Bare Metal cluster via jetlag from a Scale Lab Bastion Machine quickstart](docs/bastion-deploy-bm.md)
Expand Down
32 changes: 17 additions & 15 deletions docs/bastion-deploy-bm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,27 +32,27 @@ and check to make sure that only the key(s) you wanted were added.
[user@fedora ~]$
```

3. Update the version of RHEL that came on the bastion machine and reboot
3. Update the version of RHEL on the bastion machine and reboot

```console
[user@fedora ~]$ ssh [email protected]
...
[root@xxx-h01-000-r650 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.2 (Ootpa)

[root@xxx-h01-000-r650 ~]# ./update-latest-rhel-release.sh 8.7
Changing repository from 8.2 to 8.7
[root@xxx-h01-000-r650 ~]# ./update-latest-rhel-release.sh 8.9
Changing repository from 8.2 to 8.9
Cleaning dnf repo cache..

-------------------------
Run dnf update to upgrade to RHEL 8.7
Run dnf update to upgrade to RHEL 8.9

[root@xxx-h01-000-r650 ~]# dnf update -y
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
rhel87 AppStream 245 MB/s | 7.8 MB 00:00
rhel87 BaseOS 119 MB/s | 2.4 MB 00:00
rhel89 AppStream 245 MB/s | 7.8 MB 00:00
rhel89 BaseOS 119 MB/s | 2.4 MB 00:00
Extra Packages for Enterprise Linux 8 - x86_64 14 MB/s | 14 MB 00:00
Last metadata expiration check: 0:00:01 ago on Tue 02 May 2023 06:58:15 PM UTC.
Dependencies resolved.
Expand All @@ -65,10 +65,10 @@ Connection to xxx-h01-000-r650.rdu2.scalelab.redhat.com closed.
[user@fedora ~]$ ssh [email protected]
...
[root@xxx-h01-000-r650 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.7 (Ootpa)
Red Hat Enterprise Linux release 8.9 (Ootpa)
```

4. Install some additional tools to help after reboot
4. Install additional tools to help after reboot

```console
[root@xxx-h01-000-r650 ~]# dnf install tmux git python3-pip sshpass -y
Expand All @@ -77,7 +77,7 @@ Updating Subscription Management repositories.
Complete!
```

5. Setup ssh keys on the bastion and copy to itself to permit local ansible interactions
5. Setup ssh keys on the bastion to permit local ansible interactions

```console
[root@xxx-h01-000-r650 ~]# ssh-keygen
Expand Down Expand Up @@ -159,9 +159,9 @@ Change `lab_cloud` to `lab_cloud: cloud99`

Change `cluster_type` to `cluster_type: bm`

Set `worker_node_count` if you desire to limit the number of worker nodes from your scale lab allocation. Set it to `0` if you want a 3 node compact cluster.
Set `worker_node_count` to limit the number of worker nodes from your scale lab allocation. Set it to `0` if you want a 3 node compact cluster.

Change `ocp_release_image` to the desired image if the default (4.14.7) is not the desired version.
Change `ocp_release_image` to the intended image if the default (4.14.7) is not the desired version.
If you change `ocp_release_image` to a different major version (Ex `4.14`), then change `openshift_version` accordingly.

Only change `networktype` if you need to test something other than `OVNKubernetes`
Expand Down Expand Up @@ -190,12 +190,12 @@ Here you can see a network diagram for the bare metal cluster on Dell r650 with
Double check your nic names with your actual bastion machine.
** If you desire to use a *different network* than "Network 1" for your controlplane network then you will have to append some additional overrides to the extra vars portion of the all.yml vars file.
See [tips and vars](https://github.com/redhat-performance/jetlag/blob/main/docs/tips-and-vars.md#Other-Networks) for more information
** If you desire to use a *different network* than "Network 1" for your controlplane network then you will have to append additional overrides to the extra vars portion of the all.yml vars file.
See [tips and vars](tips-and-vars.md#using-other-network-interfaces) for more information
### OCP node vars
The same chart provided by the scale lab for the bastion machine, is used to identify the nic names for `controlplane_lab_interface`.
The same chart provided by the scale lab for the bastion machine, is used to identify the nic name for `controlplane_lab_interface`.

* `controlplane_lab_interface` should always be set to the nic name under "Public Network" for the specific system type

Expand Down Expand Up @@ -420,7 +420,7 @@ Finally run the `bm-deploy.yml` playbook ...

It is suggested to monitor your first deployment to see if anything hangs on boot or if the virtual media is incorrect according to the bmc. You can monitor your deployment by opening the bastion's GUI to assisted-installer (port 8080, ex `xxx-h01-000-r650.rdu2.scalelab.redhat.com:8080`), opening the consoles via the bmc of each system, and once the machines are booted, you can directly ssh to them and tail log files.

If everything goes well you should have a cluster in about 60-70 minutes. You can interact with the cluster from the bastion.
If everything goes well you should have a cluster in about 60-70 minutes. You can interact with the cluster from the bastion via the kubeconfig or kubeadmin password.

```console
(.ansible) [root@xxx-h01-000-r650 ~]# export KUBECONFIG=/root/bm/kubeconfig
Expand All @@ -429,4 +429,6 @@ NAME STATUS ROLES AGE VERSION
xxx-h02-000-r650 Ready control-plane,master,worker 73m v1.25.7+eab9cc9
xxx-h03-000-r650 Ready control-plane,master,worker 103m v1.25.7+eab9cc9
xxx-h05-000-r650 Ready control-plane,master,worker 105m v1.25.7+eab9cc9
(.ansible) [root@xxx-h01-000-r650 jetlag]# cat /root/bm/kubeadmin-password
xxxxx-xxxxx-xxxxx-xxxxx
```
13 changes: 9 additions & 4 deletions docs/deploy-bm-ibmcloud.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Deploy a Bare Metal cluster on IBMcloud via jetlag quickstart

To deploy a bare metal OpenShift cluster, order 6 machines from the [IBM bare metal server catalog](https://cloud.ibm.com/gen1/infrastructure/provision/bm).
For guidance on how to order hardware on IBMcloud, see [order-hardware-ibmcloud.md](../docs/order-hardware-ibmcloud.md) in [docs](../docs) directory.
For guidance on how to order hardware on IBMcloud, see [order-hardware-ibmcloud.md](order-hardware-ibmcloud.md) in [docs](../docs) directory.
The machines used to test this are of Server profile E5-2620 in DAL10 datacenter with automatic port redundancy. One machine will become the bastion, 3 machines will become control-plane nodes, and the remaining 2 nodes will be worker nodes. Ensure that you order either CentOS or RHEL machines with a new enough version (8.6) otherwise podman will not have host networking functionality. The bastion machine should have a public accessible ip and will NAT traffic for the cluster to the public network. The other machines can have a public ip address but it is not currently in use with this deployment method.

Once your machines are delivered, login to the ibmcloud cli using the cut and paste link from the cloud portal. You should be able to list the machines from your local machine, for example:
Expand Down Expand Up @@ -33,11 +33,16 @@ Resolving deltas: 100% (704/704), done.
[user@fedora ~]$ cd jetlag
```

## Review Prerequisites and set pull-secret
## Prerequisites

Review the Ansible prerequisites on the [README](https://github.com/redhat-performance/jetlag#prerequisites).
Source bootstrap to install/setup Ansible

Recommended: run ansible inside virtual environment: ```source bootstrap.sh```
```console
[user@fedora jetlag]$ source bootstrap.sh
Collecting pip
...
(.ansible) [user@fedora jetlag]$
```

Set your pull secret file `pull_secret.txt` in the base directory of the cloned jetlag repo. The contents should resemble this json:

Expand Down
2 changes: 1 addition & 1 deletion docs/deploy-sno-quickstart.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ dns2=10.1.36.2
# Unused
```
** If your bastion machine is not running RHEL 8.6, you will have to upgrade following [this short procedure](troubleshooting.md#upgrade-rhel-to-86-in-scalelab).
** If your bastion machine is not running RHEL 8.6 or newer, you will have to upgrade following [this short procedure](troubleshooting.md#scalelab---upgrade-rhel).
Next run the `setup-bastion.yml` playbook ...
Expand Down
8 changes: 4 additions & 4 deletions docs/troubleshooting.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ Substitute the user/password/hostname to perform the reset on a desired host. No

## Dell - Unable Mount Virtual Media

In some cases, the Dell iDRAC is unable to mount Virtual Media due to the Virtual Console Plug-in Type being set as eHTML5 instead of HTML5.
In some cases, the Dell iDRAC is unable to mount Virtual Media due to the Virtual Console Plug-in Type being set as eHTML5 instead of HTML5.
To change this, navigate to Configuration -> Virtual Console -> Plug-in Type and select HTML5 instead of eHTML5.

## Scalelab - Fix boot order of machines
Expand Down Expand Up @@ -159,12 +159,12 @@ The values in *config/idrac_interfaces.yml* are first of all for the SCALE lab.
On the bastion machine:

```console
[root@f16-h11-000-1029p ~]# ./update-latest-rhel-release.sh 8.7
Changing repository from 8.2 to 8.7
[root@f16-h11-000-1029p ~]# ./update-latest-rhel-release.sh 8.9
Changing repository from 8.2 to 8.9
Cleaning dnf repo cache..

-------------------------
Run dnf update to upgrade to RHEL 8.7
Run dnf update to upgrade to RHEL 8.9

[root@f16-h21-000-1029p ~]# dnf update -y
...
Expand Down

0 comments on commit 5bb9aff

Please sign in to comment.