mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-08-25 01:06:37 +00:00
Merge branch 'bulk-provision' into 'master'
Bulk provision Rebased with master and performed a force push per our discussion so we get a clean merge See merge request !154
This commit is contained in:
Rodrigo Caballero
Binary file not shown.
|
After Width: | Height: | Size: 36 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 59 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 33 KiB |
+129
-110
@@ -1,146 +1,165 @@
|
||||
.. _bulk_provisioning:
|
||||
|
||||
Bulk Provisioning
|
||||
=================
|
||||
#################
|
||||
|
||||
Data centers tend to land and install new machines in bulk. When dealing with
|
||||
large quantities of new machines or hardware upgrades, data center admins may need
|
||||
a way to install and configure with a minimal amount of effort. To support this
|
||||
use case, the Clear Linux team has developed some tooling and instructions for
|
||||
how to bulk-load machines with Clear Linux using iPXE.
|
||||
The |CLOSIA| can be automatically provisioned in bulk using a combination of
|
||||
the |CL| installer, **Ister**, and :abbr:`ICIS (Ister Cloud Init Service)`.
|
||||
This guide covers how to perform a bulk provision of |CL| using **Ister**
|
||||
and **ICIS**.
|
||||
|
||||
This bulk provisioning scenario is a variation on the :ref:`network_boot` scenario,
|
||||
with some alterations to facilitate installing and configuring many hosts. The Clear
|
||||
Linux installer (``Ister``) was augmented to detect and find ``cloud-init`` configuration
|
||||
data during the install. When this data is found, a :file:`cloud-init` file is fetched,
|
||||
and the host, after being rebooted by ``ister``, uses micro-config drive (``ucf``) to
|
||||
configure the host as directed by the :file:`cloud-init` configuration file.
|
||||
To configure a bulk provision, Ister configuration files and cloud-init
|
||||
files must be defined. Hosting the configuration files in **ICIS** allows
|
||||
**Ister** to use them during the installation. The Ister configuration
|
||||
files allow us to customize the installation process. The cloud-init\*
|
||||
files allow us to customize the instance of the installation.
|
||||
|
||||
In our pilot of this workflow, we used ``clear-cloud-init`` (which will become ucf - micro config drive)
|
||||
to install the trust relationships for the host to be further managed by Ansible. The
|
||||
:file:`cloud-init` configuration file was served by a simple little web application
|
||||
that ``Ister`` knows how to query for host configuration data. This allowed us to
|
||||
simultaneously power-on one hundred machines, each of which
|
||||
* picked up the Clear Linux installer via PXE,
|
||||
* installed Clear Linux,
|
||||
* rebooted, and
|
||||
* entered into a mode that can be managed through Ansible.
|
||||
Figure 1 depicts the flow of information between a PXE server and a PXE
|
||||
client that needs to be set up to perform a bulk provision.
|
||||
|
||||
The web application that serves cloud-init configurations to ``Ister`` is uninspiringly named
|
||||
"Ister Cloud Init Service" (ICIS), and is available at github - https://github.com/clearlinux/ister-cloud-init-svc
|
||||
.. figure:: _static/images/bulk-provision-flow.png
|
||||
:alt: Bulk provision information flow
|
||||
|
||||
One nice attribute of this system is that once the iPXE bits are created, many installer
|
||||
behaviors can be configured without having to regenerate the iPXE bits.
|
||||
Figure 1: Bulk provision information flow
|
||||
|
||||
System Architecture
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
The following diagrams illustrate the system architecture.
|
||||
Before performing a bulk provision, verify you have a PXE server capable
|
||||
of performing network boots of |CL|. Please refer to our
|
||||
:ref:`guide on how to perform an iPXE boot<network_boot>` using
|
||||
:abbr:`NAT (network address translation)` for details.
|
||||
|
||||
.. image:: _static/images/icis_pxe_diagram.png
|
||||
:align: center
|
||||
:alt: pxe-server-overview
|
||||
Because a bulk provision relies on a reboot, ensure the following
|
||||
preparations have been made:
|
||||
|
||||
As can be seen, ICIS is a web application (specifically a Flask application, hence managed by ``uwsgi``
|
||||
under nginx) that can run **anywhere** visible to hosts booting from PXE. In our example, we simply
|
||||
hosted it on the same system serving the pxe content.
|
||||
* No existing disks are bootable.
|
||||
* The network boot option must come immediately after the disk boot option
|
||||
on any computer performing the installation.
|
||||
|
||||
The following diagram illustrates the flow of information between the pxe server and a host booting
|
||||
into the installer via pxe. This diagram is intended to show the logical flow of information, and
|
||||
is not a literal depiction of the protocol-level exchanges.
|
||||
Configuration
|
||||
=============
|
||||
|
||||
.. image:: _static/images/icis_installer_workflow.png
|
||||
:align: center
|
||||
:alt: pxe-icis-install-workflow
|
||||
#. Install **ICIS** by following the getting started guide on the
|
||||
`ICIS GitHub repository`_.
|
||||
|
||||
Configuration Overview
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
#. Create an Ister installation file and save it to the
|
||||
:file:`static/ister` directory within the web hosting directory for
|
||||
**ICIS**. The installation file is a JSON block and provides **Ister**
|
||||
with the steps it needs to perform an installation. The file outlines
|
||||
what partitions, file systems, and mount points **Ister** should set
|
||||
up. Lastly, the file outlines which bundles to install. See our
|
||||
:ref:`bundles_overview` for the list of available bundles. The
|
||||
following example shows the contents of an Ister installation file:
|
||||
|
||||
Here is a high-level overview describing set up and configuration of a bulk-provisioning solution.
|
||||
.. code-block:: json
|
||||
|
||||
* First, stand up an iPXE server as described in the :ref:`network_boot` docs. Note that you will
|
||||
be generating your own artifacts to be served by PXE in a later step.
|
||||
{
|
||||
"DestinationType":"physical",
|
||||
"PartitionLayout":[
|
||||
{"disk":"sda", "partition":1, "size":"512M", "type":"EFI"},
|
||||
{"disk":"sda", "partition":2, "size":"512M", "type":"swap"},
|
||||
{"disk":"sda", "partition":3, "size":"rest", "type":"linux"}
|
||||
],
|
||||
"FilesystemTypes":[
|
||||
{"disk":"sda", "partition":1, "type":"vfat"},
|
||||
{"disk":"sda", "partition":2, "type":"swap"},
|
||||
{"disk":"sda", "partition":3, "type":"ext4"}
|
||||
],
|
||||
"PartitionMountPoints":[
|
||||
{"disk":"sda", "partition":1, "mount":"/boot"},
|
||||
{"disk":"sda", "partition":3, "mount":"/"}
|
||||
],
|
||||
"Version":"latest",
|
||||
"Bundles":[
|
||||
"kernel-native",
|
||||
"os-core",
|
||||
"os-core-update",
|
||||
"os-cloudguest"
|
||||
],
|
||||
"IsterCloudInitSvc":"http://192.168.1.1:60000/icis/"
|
||||
}
|
||||
|
||||
* Land `Ister Cloud Init Service <https://github.com/clearlinux/ister-cloud-init-svc>`_ on the
|
||||
pxe server. The README in the ICIS github repo has directions on how to install and configure.
|
||||
.. important::
|
||||
|
||||
* Generate the installer that will load and boot over PXE. Copy the relevant files into the
|
||||
appropriate location on the iPXE server.
|
||||
Every Ister installation file hosted on **ICIS** must contain the
|
||||
the ``IsterCloudInitSvc`` parameter as well as the ``os-cloudguest``
|
||||
bundle. These entries allow **Ister** to customize an instance of of an
|
||||
install.
|
||||
|
||||
A pxe installer is generated for every release of Clear Linux OS. It can be found alongside the
|
||||
published images in https://download.clearlinux.org/releases/XXXXX/clear/clear-XXXXX-pxe.tar.xz
|
||||
#. Create an Ister configuration file to define the location of the
|
||||
Ister installation file. Save it to the :file:`static/ister` directory
|
||||
within the web hosting directory of **ICIS**. The following example shows
|
||||
an Ister configuration file:
|
||||
|
||||
Alternatively, use the `create_pxe.sh <https://github.com/bryteise/ister/blob/master/create_pxe.sh>`_
|
||||
script to roll your own. Note this creates the pxe-installer from the "provisioning" installer in
|
||||
Clear Linux. This is a touchless installer that takes all installation information from a config
|
||||
file. You can get this image from a given Clear Linux release by looking
|
||||
in https://download.clearlinux.org/releases/XXXX/clear/ where XXXX is a Clear Linux release number.
|
||||
Just download the image alongside the ``create_pxe.sh`` script and name it ``provision.img``.
|
||||
.. code-block:: json
|
||||
|
||||
* Stage config files for ``ister`` that will govern Ister's behavior. This includes modifying the
|
||||
script for ipxe boot, getting it to pass an additional parameter to the kernel.
|
||||
template=http://192.168.1.1:60000/icis/static/ister/ister.json
|
||||
|
||||
The magic that ties all of this together is that the pxe script conveys to ister the location of its configuration files via the kernel command line of the installer it kicks off. The kernel preserves its command line precisely, and ister inspects it via ``/proc/cmdline``.
|
||||
#. Modify the iPXE boot script by adding a kernel parameter to the command line
|
||||
for booting the network image. Add the kernel parameter ``isterconf`` with
|
||||
the location of the Ister configuration file hosted on **ICIS** as the
|
||||
kernel parameter value. The following example shows an iPXE boot script
|
||||
with the ``isterconf`` parameter:
|
||||
|
||||
Here is an example pxe script:
|
||||
.. code-block:: json
|
||||
|
||||
.. code-block:: console
|
||||
#!ipxe
|
||||
kernel linux quiet init=/usr/lib/systemd/systemd-bootchart initcall_debug tsc=reliable no_timer_check noreplace-smp rw initrd=initrd isterconf=http://192.168.1.1:60000/icis/static/ister/ister.conf
|
||||
initrd initrd
|
||||
boot
|
||||
|
||||
#!ipxe
|
||||
kernel linux quiet rdinit=/usr/lib/systemd/systemd-bootchart initcall_debug tsc=reliable
|
||||
no_timer_check noreplace-smp rw initrd=initrd isterconf=http://192.168.1.1/icis/static/ister/ister.conf
|
||||
initrd initrd
|
||||
boot
|
||||
.. note::
|
||||
|
||||
When the pxe installer kicks off ``ister``, it will make note of the location of the ``conf`` file
|
||||
that was given on the kernel command line, and fetch the file. This file then tells ``ister`` where
|
||||
to get the json template file that describes partition schemes, and which version of Clear Linux to
|
||||
install. This means that so long as the contents of a release are compatible with the version of
|
||||
software update (``swupd``) in the installer, this pxe installer can be told to install a newer
|
||||
version of Clear Linux simply by tweaking the json on the web server, rather than rolling an
|
||||
entirely new installer.
|
||||
After the network image of |CL| boots, **Ister** inspects the
|
||||
parameters used during boot in :file:`/proc/cmdline` to find the
|
||||
location of the Ister configuration file.
|
||||
|
||||
One other important piece of configuration data in the json configuration file is the location of
|
||||
an ICIS configuration service. Ister will query ICIS for a role using the MAC address of the network
|
||||
interface being used to communicate with the ICIS service. Ister will then fetch that specific
|
||||
:file:`cloud-init` file and ``configure ucd`` to run on first-boot against that config file.
|
||||
#. Write a cloud-init document to customize the instance of the installation
|
||||
according to your requirements. The `cloud-init Read the Docs`_ provides a
|
||||
guide on how to write a cloud-init document. The guide covers the
|
||||
customization options provided by cloud-init after an installation.
|
||||
|
||||
The `Ister Cloud Init Service <https://github.com/clearlinux/ister-cloud-init-svc>`_ github repo
|
||||
has example ister configuration files under ``static/ister``.
|
||||
#. Save the cloud-init document to the :file:`static/roles` directory within
|
||||
the web hosting directory for **ICIS** with the name of a role you would
|
||||
like to create. For example, a role may be "database", "web", or "ciao".
|
||||
|
||||
Here is an example ister-template.json file.
|
||||
#. After creating the roles, also known as cloud-init files, assign roles to
|
||||
MAC addresses of PXE clients. To do so, modify the :file:`config.txt` file
|
||||
in the ``static`` directory within the web hosting directory of **ICIS**.
|
||||
The following example shows one such assignment:
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"DestinationType" : "phyiscal",
|
||||
"PartitionLayout" : [ { "disk" : "/dev/sda", "partition" : 1,
|
||||
"size" : "64M", "type" : "EFI" },
|
||||
{ "disk" : "/dev/sda", "partition" : 2,
|
||||
"size" : "2G", "type" : "linux" } ],
|
||||
"FilesystemTypes" : [ { "disk" : "/dev/sda", "partition" : 1, "type" : "vfat" },
|
||||
{ "disk" : "/dev/sda", "partition" : 2, "type" : "ext4" } ],
|
||||
"PartitionMountPoints" : [ { "disk" : "/dev/sda", "partition" : 1,
|
||||
"mount" : "/boot" },
|
||||
{ "disk" : "/dev/sda", "partition" : 2,
|
||||
"mount" : "/" } ],
|
||||
"Version": 6580,
|
||||
"Bundles": ["kernel-native", "os-core-update", "os-core",
|
||||
"bootloader", "sysadmin-hostmgmt", "openssh-server"],
|
||||
"PostNonChroot": ["./installation-image-post-update-version.py"],
|
||||
"IsterCloudInitSvc": ["http://192.168.1.1/icis/"]
|
||||
}
|
||||
# MAC address,role
|
||||
00:01:02:03:04:05,ciao
|
||||
|
||||
* Configure ICIS to map MAC addresses to role files appropriately. Then create the role files, which
|
||||
are ``cloud-init`` configuration files. Note, it is possible to simply specify a "default" role for
|
||||
any unmatched MAC address; this may be handy when all install targets are to be configured identically.
|
||||
If MAC addresses of PXE clients are not listed within the
|
||||
:file:`config.txt` file, a default role for those MAC address may be
|
||||
defined as follows:
|
||||
|
||||
* Final pre-flight check. Assuming your iPXE server is at 192.168.1.1, all of the following urls need to be working.
|
||||
.. code-block:: json
|
||||
|
||||
- http://192.168.1.1/icis/static/ister/ister.conf
|
||||
- http://192.168.1.1/icis/static/ister/ister_config.json
|
||||
- http://192.168.1.1/icis/get_config/<MAC ADDR>
|
||||
- http://192.168.1.1/icis/get_role/<role returned from previous url>
|
||||
- http://192.168.1.1/ipxe_boot_script.txt
|
||||
# MAC address,role
|
||||
default,ciao
|
||||
|
||||
* Boot an iPXE client and watch Clear Linux install.
|
||||
#. Verify the following URLs are accessible:
|
||||
|
||||
* http://192.168.1.1:60000/icis/static/ister/ister.conf
|
||||
* http://192.168.1.1:60000/icis/static/ister/ister.json
|
||||
* http://192.168.1.1:60000/icis/get_config/<MAC address>
|
||||
* http://192.168.1.1:60000/icis/get_role/<role>
|
||||
* http://192.168.1.1:60000/ipxe/ipxe_boot_script.txt
|
||||
|
||||
#. Power on the PXE client and watch it boot and install |CL|.
|
||||
|
||||
#. Power-cycle the PXE client and watch it customize the |CL| installation.
|
||||
|
||||
**Congratulations!** You have successfully performed a bulk provision of |CL|.
|
||||
|
||||
|
||||
.. _ICIS GitHub repository:
|
||||
https://github.com/clearlinux/ister-cloud-init-svc
|
||||
|
||||
.. _cloud-init Read the Docs:
|
||||
https://cloudinit.readthedocs.io
|
||||
|
||||
Reference in New Issue
Block a user