mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-08-24 08:48:15 +00:00
Remove ciao content from clearlinux.org/documentation
Ciao has created their own content in their repos. Therefore, this content is redundant and causes confusion. Signed-off-by: Rodrigo Caballero <rodrigo.caballero.abraham@intel.com>
This commit is contained in:
Rodrigo Caballero
@@ -1,12 +0,0 @@
|
||||
.. _ciao:
|
||||
|
||||
ciao
|
||||
####
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
cluster-setup-quick
|
||||
cluster-setup
|
||||
deployment/deployment
|
||||
deployment/ubuntu
|
||||
@@ -1,11 +0,0 @@
|
||||
.. _cluster-setup-quick:
|
||||
|
||||
Easy ciao development cluster setup
|
||||
###################################
|
||||
|
||||
For quick ciao development cluster setup, see:
|
||||
|
||||
https://github.com/01org/ciao/wiki/Single-Machine-Development-Environment
|
||||
|
||||
For more detailed and configurable manual setup of ciao, see
|
||||
:ref:`cluster-setup`.
|
||||
@@ -1,541 +0,0 @@
|
||||
.. contents::
|
||||
|
||||
.. _cluster-setup:
|
||||
|
||||
Detailed manual ciao cluster setup
|
||||
##################################
|
||||
|
||||
This topic explains how to set up a cluster of machines running Clear Linux\*
|
||||
OS for Intel® Architecture with the
|
||||
:abbr:`Cloud Integrated Advanced Orchestrator (CIAO)`.
|
||||
|
||||
While the table of contents provides links to specific points of information,
|
||||
this topic is intended as an ordered workflow. Be sure to start your cluster
|
||||
components in the correct order as explained below.
|
||||
|
||||
Infrastructure prerequisites
|
||||
============================
|
||||
|
||||
Hardware needs
|
||||
--------------
|
||||
|
||||
You'll need at least four machines and a switch connecting them to form your
|
||||
beginning ciao cluster. The switch is assumed to be plugged directly into an
|
||||
"upstream" network running a DHCP server. See the illustration below as an
|
||||
example:
|
||||
|
||||
.. image:: ./figures/image-blob-ciao-networking.png
|
||||
|
||||
The following examples assume you have four nodes on a ``192.168.0.0/16``
|
||||
network:
|
||||
|
||||
Controller node
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
* IP ``192.168.0.101``
|
||||
* Runs Controller, Scheduler, SSL Keystone.
|
||||
|
||||
|
||||
Network node ("nn")
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* IP ``192.168.0.102``
|
||||
* Runs Launcher with ``--network=nn`` option
|
||||
* Has CNCI image in ``/var/lib/ciao/images``. See below for more on CNCI
|
||||
image preparation.
|
||||
|
||||
Compute node 1 ("cn1")
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* IP ``192.168.0.103``
|
||||
* Runs Launcher with ``--network=cn`` option
|
||||
* Has workload images in ``/var/lib/ciao/images``
|
||||
|
||||
Compute node 2 ("cn2")
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* ``IP 192.168.0.104``
|
||||
* Runs Launcher with ``--network=cn option``
|
||||
* Has workload images in ``/var/lib/ciao/images``
|
||||
|
||||
|
||||
Network needs
|
||||
-------------
|
||||
|
||||
Our system assumes cluster nodes have full connectivity at a routed
|
||||
IP level. Additionally, the network node must have access to a DHCP
|
||||
server offering addresses that are routable across the cluster.
|
||||
|
||||
|
||||
Node setup
|
||||
==========
|
||||
|
||||
Install Clear Linux OS for Intel Architecture as host on all nodes
|
||||
------------------------------------------------------------------
|
||||
|
||||
Install Clear Linux OS for Intel Architecture as the host
|
||||
OS on all nodes by following the instructions in the topic
|
||||
:ref:`clear-host`. The current `downloadable installer images`_
|
||||
are compatible with ciao.
|
||||
|
||||
After the base installation on each node, add the following additional
|
||||
bundle, which adds components needed by CIAO::
|
||||
|
||||
$ swupd bundle-add cloud-control
|
||||
|
||||
|
||||
|
||||
Build the CIAO software
|
||||
-----------------------
|
||||
|
||||
Ciao is written in the Go programming language. It requires Go 1.7 to
|
||||
build. Most Linux distributions have out-of-date versions of Go in their
|
||||
repositories, so you will probably need to download and install a recent
|
||||
version of Go. This can be easily done by following the
|
||||
`Go installation instructions <https://golang.org/doc/install>`__.
|
||||
|
||||
On a Linux development machine with Go language development tooling
|
||||
present, use the ``go get`` tool to fetch and build ciao and its go
|
||||
dependencies::
|
||||
|
||||
$ cd $GOPATH/src
|
||||
$ go get -v -u github.com/01org/ciao
|
||||
|
||||
The binaries will install to ``$GOPATH/bin``. You should have
|
||||
``ciao-cli``, ``ciao-cert``, ``cnci_agent``, ``ciao-launcher``,
|
||||
``ciao-controller``, and ``ciao-scheduler``.
|
||||
|
||||
Build certificates
|
||||
------------------
|
||||
|
||||
Create the ssntp-internal communications certificates
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
On your development machine, generate the certificates for each of your
|
||||
roles; general instructions can be found under the `ciao-cert`_ documentation.
|
||||
|
||||
When generating the certificates, pass in the IP and host name for
|
||||
the host on which you will be running the scheduler in the ``-ip`` and
|
||||
``-host`` arguments, respectively. The scheduler acts as the cluster
|
||||
SSNTP server, and connecting clients will validate credentials matched by
|
||||
those embedded in the certificates.
|
||||
|
||||
Create unique certificates for each of your scheduler, compute node, network
|
||||
node launchers, cnciagent, controller, and the CNCI launcher; save each with a
|
||||
unique name. The names, locations, and contents (signer and role) of the
|
||||
certificates are very important. The rest of this topic will consistently use
|
||||
the following example filenames:
|
||||
|
||||
* ``CAcert-[scheduler-node-hostname].pem``: copy to all nodes' ``/etc/pki/ciao`` and the CNCI image's ``/var/lib/ciao``. See below for more on CNCI image preparation.
|
||||
* ``cert-CNAgent-localhost.pem``: copy to all compute nodes' ``/etc/pki/ciao``.
|
||||
* ``cert-CNCIAgent-localhost.pem``: copy into your network node's ``/var/lib/ciao``. A script later will copy it into the CNCI appliance image. See below for more on CNCI image preparation.
|
||||
* ``cert-Controller-localhost.pem``: copy into your controller node's ``/etc/pki/ciao``.
|
||||
* ``cert-NetworkingAgent-localhost.pem``: copy into your network node's ``/etc/pki/ciao``.
|
||||
* ``cert-Scheduler-[scheduler-node-hostname].pem``: copy into your controller node's ``/etc/pki/ciao``.
|
||||
|
||||
Create the controller web certificates
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
On your development box, generate ssl certificates for the controller's https service::
|
||||
|
||||
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout controller_key.pem -out controller_cert.pem
|
||||
|
||||
Copy the ``controller_cert.pem`` and ``controller_key.pem`` files to your
|
||||
controller node. You can use the same location where you will be storing
|
||||
your controller binary (``ciao-controller``).
|
||||
|
||||
You'll also need to pull that certificate into your browser as noted below in
|
||||
the `Starting a workload` section.
|
||||
|
||||
Keystone node
|
||||
-------------
|
||||
|
||||
You need to run a Keystone service. General documentation on setting
|
||||
up Keystone services can be found at the `OpenStack developer`_ website.
|
||||
We need a few configuration points. For example:::
|
||||
|
||||
$ openstack service create --name ciao compute
|
||||
$ openstack user create --password hello csr
|
||||
$ openstack role add --project service --user csr admin
|
||||
$ openstack user create --password giveciaoatry demo
|
||||
$ openstack role add --project demo --user demo user
|
||||
|
||||
This adds a ciao compute service, a keystone user and project for the
|
||||
controller (a.k.a. ``csr``) node, and a demo user with the password
|
||||
``giveciaoatry``.
|
||||
|
||||
Controller node setup
|
||||
---------------------
|
||||
|
||||
The controller node will host your controller and scheduler. Certificates are assumed
|
||||
to be in ``/etc/pki/ciao``, generated with the correct roles and names
|
||||
as previously described.
|
||||
|
||||
Scheduler
|
||||
~~~~~~~~~
|
||||
|
||||
Copy in the scheduler binary from your build/development machine to any
|
||||
location, then launch it first (does not require root)::
|
||||
|
||||
$ ./ciao-scheduler --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Scheduler-[scheduler-node-hostname].pem --heartbeat
|
||||
|
||||
With the optional ``--heartbeat`` option, the scheduler console will
|
||||
output once per-second a heartbeat message showing connected Controller
|
||||
and Compute Node client statistics. It also displays a line of
|
||||
information for each command or event traversing the SSNTP server.
|
||||
As the sole SSNTP server in the ciao cluster, it is a key debugging point
|
||||
to understand failed flows of actions/reactions across your cluster.
|
||||
Launching it first means this console output helps confirm your subsequent
|
||||
cluster configurations actions are indeed succeeding.
|
||||
|
||||
Compute node setup
|
||||
------------------
|
||||
|
||||
Each compute node needs one launcher daemon connected to the scheduler.
|
||||
Certificates are assumed to be in ``/etc/pki/ciao``, generated with the
|
||||
correct roles and names as previously described.
|
||||
|
||||
Copy in the launcher binary from your build/development machine to any
|
||||
location.
|
||||
|
||||
Prepopulate the OS image cache
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Ciao has not yet integrated with an existing image server; so for
|
||||
simplicity, presume a prepopulated image cache for each compute
|
||||
node in ``/var/lib/ciao/images``.
|
||||
|
||||
We have tested the `Fedora 23 Cloud`_, Clear Linux OS for Intel
|
||||
Architecture cloud `downloadable cloud images`_, and Ubuntu* images. Each image
|
||||
will be referenced very specifically by a UUID in our configuration
|
||||
files, so follow the instructions here exactly. You may wish to create
|
||||
the needed UUID named image files as symlinks to a more human readable
|
||||
and descriptively named image files as is done in the following example::
|
||||
|
||||
$ mkdir -p /var/lib/ciao/images
|
||||
$ cd /var/lib/ciao/images
|
||||
|
||||
Fedora Cloud::
|
||||
|
||||
$ curl -O https://dl.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2
|
||||
$ ln -s Fedora-Cloud-Base-23-20151030.x86_64.qcow2 73a86d7e-93c0-480e-9c41-ab42f69b7799
|
||||
|
||||
Clear Linux OS for Intel Architecture Cloud::
|
||||
|
||||
$ LATEST=$(curl https://download.clearlinux.org/latest)
|
||||
$ curl -O https://download.clearlinux.org/image/clear-${LATEST}-cloud.img.xz
|
||||
$ xz -T0 --decompress clear-${LATEST}-cloud.img.xz
|
||||
$ ln -s clear-${LATEST}-cloud.img df3768da-31f5-4ba6-82f0-127a1a705169
|
||||
|
||||
Docker* images will be pulled down automatically at the time of first usage.
|
||||
|
||||
Each compute node needs its ``/var/lib/ciao/images`` directory populated with
|
||||
images with which you wish to test.
|
||||
|
||||
Start the compute node launcher
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The launcher is run with options declaring certificates, maximum VMs
|
||||
(controls when FULL is returned by a node, scaling to the resources
|
||||
available on your node), server location, and compute node ("cn")
|
||||
launching type. For example::
|
||||
|
||||
$ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem --server=<your-server-address> --network=cn --compute-net <node compute subnet> --mgmt-net <node management subnet>
|
||||
|
||||
Optionally, add ``-logtostderr`` (more verbose with also ``-v=2``) to get
|
||||
console logging output.
|
||||
|
||||
The launcher runs as root because launching QEMU/KVM virtual machines
|
||||
requires ``/dev/kvm`` and other restricted resource access.
|
||||
|
||||
Network node setup
|
||||
------------------
|
||||
|
||||
The network node hosts VMs running the :abbr:`Compute Network Concentrators
|
||||
Instance (CNCI)` or the **CNCI Agent**, one for each tenant. These VMs
|
||||
are automatically launched by the controller.
|
||||
|
||||
Certificates are assumed to be in ``/etc/pki/ciao``, generated with the
|
||||
correct roles and names as previously described.
|
||||
|
||||
Pre-populate the CNCI image cache
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This section describes how to generate a CNCI image from a vanilla
|
||||
Clear Cloud qcow2 image::
|
||||
|
||||
$ cd /var/lib/ciao/images
|
||||
$ curl -O https://download.clearlinux.org/demos/ciao/clear-7470-ciao-networking.img.xz
|
||||
$ xz -T0 --decompress clear-7470-ciao-networking.img.xz
|
||||
$ ln -s clear-7470-ciao-networking.img 4e16e743-265a-4bf2-9fd1-57ada0b28904
|
||||
$ $GOPATH/src/github.com/01org/ciao/networking/cnci_agent/scripts/update_cnci_cloud_image.sh /var/lib/ciao/images/clear-7470-ciao-networking.img /etc/pki/ciao/
|
||||
|
||||
Start the network node launcher
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The network node's launcher is run similarly to the compute node's launcher.
|
||||
The primary difference is that it uses the network node ("nn") launching
|
||||
type::
|
||||
|
||||
$ sudo ./ciao-launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem --server=<your-server-address> --network=nn --compute-net <network node compute subnet> --mgmt-net <network node management subnet>
|
||||
|
||||
Start the controller
|
||||
--------------------
|
||||
|
||||
Starting the Controller on the controller node is what truly activates your
|
||||
cluster for use.
|
||||
|
||||
#. Copy in the ciao-controller binary from your build/development machine to any
|
||||
location. Certificates are assumed to be in ``/etc/pki/ciao``, generated with
|
||||
the correct roles and names as previously described.
|
||||
|
||||
#. Copy in the initial database table data from the ciao-controller source
|
||||
(``$GOPATH/src/github.com/01org/ciao/ciao-controller`` on your
|
||||
build/development) to the same directory as the ciao-controller binary.
|
||||
Copying in ``*.csv`` will work if you are testing a Clear Cloud image,
|
||||
Fedora image and Docker. Other images will require edits to the csv
|
||||
config files.
|
||||
|
||||
#. Copy in the test.yaml file from
|
||||
``$GOPATH/src/github.com/01org/ciao/ciao-controller/test.yaml``.
|
||||
|
||||
The `ciao-controller workload_resources.csv`_ and the
|
||||
`ciao-controller workload_template.csv`_ have four stanzas, so yours
|
||||
should as well, in order to successfully run each of the four images
|
||||
currently described earlier on this page (Fedora, Clear, Docker Ubuntu,
|
||||
CNCI). To run other images of your choosing, follow a process similar to
|
||||
the above: pre-populate OS images and edit each of these two files on
|
||||
your controller node.
|
||||
|
||||
If the controller is on the same physical machine as the scheduler, the
|
||||
``--url`` option is optional; otherwise it refers to your scheduler
|
||||
SSNTP server IP.
|
||||
|
||||
In order for the ciao-controller's go code to correctly use the CA
|
||||
certificate(s) generated earlier when you built your keystone server,
|
||||
this certificate needs to be installed in the control node and be
|
||||
part of the control node CA root. On Clear Linux OS for Intel
|
||||
Architecture, this is accomplished with::
|
||||
|
||||
$ sudo mkdir /etc/ca-certs
|
||||
$ sudo cp cacert.pem /etc/ca-certs
|
||||
$ sudo c_hash /etc/ca-certs/cacert.pem
|
||||
|
||||
Note the generated hash from the prior command and use it in the next commands::
|
||||
|
||||
$ sudo ln -s /etc/ca-certs/cacert.pem /etc/ca-certs/<hashvalue>
|
||||
$ sudo mkdir /etc/ssl
|
||||
$ sudo ln -s /etc/ca-certs/ /etc/ssl/certs
|
||||
$ sudo ln -s /etc/ca-certs/cacert.pem /usr/share/ca-certs/<hashvalue>
|
||||
|
||||
You will need to tell the controller where the keystone service is located and
|
||||
pass the ciao service username and password to it. DO NOT USE
|
||||
localhost for your server name; **it must be the fully qualified DNS
|
||||
name of the system that is hosting the keystone service**.
|
||||
An SSL-enabled Keystone is required, with additional parameters
|
||||
for ciao-controller pointing at its certificates::
|
||||
|
||||
$ sudo ./ciao-controller --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Controller-localhost.pem -identity=https://[keystone-FQDN]:35357 --username=<Ciao keystone service username> --password=<Ciao keystone service password> --url <scheduler-FQDN> --httpskey=./key.pem --httpscert=./cert.pem
|
||||
|
||||
Optionally add ``-logtostderr`` (more verbose with also ``-v=2``) to get
|
||||
console logging output.
|
||||
|
||||
Use the `ciao-cli`_ command line tool to verify that your cluster is
|
||||
now up and running (NOTE: see section below for required environment
|
||||
variables for `ciao-cli`_)::
|
||||
|
||||
$ ciao-cli node status
|
||||
$ ciao-cli node list -compute
|
||||
$ ciao-cli node list -cnci
|
||||
|
||||
``node status`` shows the number of nodes in your cluster, and the
|
||||
status of each.
|
||||
|
||||
``node list -compute`` displays a more detailed view (number of instances per node,
|
||||
available resources per node, etc.).
|
||||
|
||||
``node list -cnci`` provides information about the current CNCI VMs, and their statuses.
|
||||
|
||||
Interacting with your cluster
|
||||
=============================
|
||||
|
||||
Ciao Web UI setup
|
||||
-----------------
|
||||
|
||||
In addition to `ciao-cli`_, a node.js-based web UI offers a means of
|
||||
interacting with your cluster visually. Documentation for this is in
|
||||
the `ciao-webui`_ github repository. A simple JSON configuration file
|
||||
allows you to specify the webui configuration and point its back end to
|
||||
your keystone and ciao-controller systems.
|
||||
|
||||
Ciao CLI setup
|
||||
--------------
|
||||
|
||||
The `ciao-cli`_ command-line tool can be set up by exporting a set of ciao-
|
||||
specific environment variables:
|
||||
|
||||
* ``CIAO_CONTROLLER`` exports the ciao controller FQDN
|
||||
* ``CIAO_IDENTITY`` exports the ciao keystone instance FQDN
|
||||
* ``CIAO_COMPUTEPORT`` exports the ciao compute alternative port
|
||||
* ``CIAO_USERNAME`` exports the ciao username
|
||||
* ``CIAO_PASSWORD`` export the ciao password for ``CIAO_USERNAME``
|
||||
|
||||
For example::
|
||||
|
||||
$ cat ciao-cli-example.sh
|
||||
|
||||
export CIAO_CONTROLLER=ciao-ctl.intel.com
|
||||
export CIAO_IDENTITY=https://ciao-identity.intel.com:35357
|
||||
export CIAO_USERNAME=user
|
||||
export CIAO_PASSWORD=ciaouser
|
||||
|
||||
$ source ciao-cli-example.sh
|
||||
|
||||
Defining those variables is optional. The same pieces of information
|
||||
can be passed to `ciao-cli`_ through the various command line options.
|
||||
The command line options will take precedence over the ciao environment
|
||||
variables and override them:
|
||||
|
||||
* ``CIAO_CONTROLLER`` can be defined by the ``--controller`` option
|
||||
* ``CIAO_IDENTITY`` can be defined by the ``--identity`` option
|
||||
* ``CIAO_COMPUTEPORT`` can be defined by the ``--computeport`` option
|
||||
* ``CIAO_USERNAME`` can be defined by the ``--username`` option
|
||||
* ``CIAO_PASSWORD`` can be defined by the ``--password`` option
|
||||
|
||||
|
||||
Start a workload
|
||||
================
|
||||
|
||||
As a valid user, the `ciao-cli` tool allows you to start a workload.
|
||||
|
||||
First, you may want to know which workloads are available::
|
||||
|
||||
$ ciao-cli workload list
|
||||
|
||||
Then you can launch one or more workloads::
|
||||
|
||||
$ ciao-cli instance add -workload <workload UUID> -instances <number of instances to launch>
|
||||
|
||||
And you can monitor all your instances statuses (``pending`` or ``running``)::
|
||||
|
||||
$ ciao-cli instance list
|
||||
|
||||
Performance data can be obtained (optionally) by adding a specific label
|
||||
to all your instances::
|
||||
|
||||
$ ciao-cli instance add -label <instance-label> -workload <workload UUID> -instances <number of instances to launch>
|
||||
|
||||
And eventually fetch the performance data::
|
||||
|
||||
$ ciao-cli trace show -label <instance-label>
|
||||
|
||||
You will also see activity related to this launch across your cluster
|
||||
components if you have consoles open and logging to standard output as
|
||||
described above.
|
||||
|
||||
|
||||
Access your workload
|
||||
====================
|
||||
|
||||
Once your workload is up, you need to know its IP address and assigned
|
||||
port, you can find it via ciao-cli::
|
||||
|
||||
$ ciao-cli instance list
|
||||
|
||||
Then look for the column "SSH IP", there is the IP assigned to your
|
||||
workload, next to it, you will see the column SSH PORT; that's the
|
||||
port that you will use to access a specific workload using your
|
||||
private key defined in the cloud-init configuration and the demo
|
||||
user name ``demouser``::
|
||||
|
||||
$ ssh -p <workload_port> -i </path/to/your/private-key> demouser@<workload-ip>
|
||||
|
||||
|
||||
Reset your cluster
|
||||
==================
|
||||
|
||||
First you should delete all instances with the `ciao-cli`_ command line
|
||||
tool::
|
||||
|
||||
$ ciao-cli instance delete -all
|
||||
|
||||
On your scheduler node, run the following command::
|
||||
|
||||
$ sudo killall -w -9 qemu-system-x86_64
|
||||
|
||||
On your controller node, go to the directory where you ran the
|
||||
ciao-controller binary and run the following commands::
|
||||
|
||||
$ sudo killall -w -9 ciao-controller
|
||||
$ sudo rm $HOME/bin/ciao-controller.db /tmp/ciao-controller-stats.db
|
||||
|
||||
On the node running your keystone VM, run the following command::
|
||||
|
||||
$ sudo killall -w -9 qemu-system-x86_64
|
||||
|
||||
On the network node, run the following commands::
|
||||
|
||||
$ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem --server=<your-server-address> --network=nn --compute-net <node compute subnet> --mgmt-net <node management subnet> --hard-reset
|
||||
$ sudo killall -9 qemu-system-x86_64
|
||||
$ sudo rm -rf /var/lib/ciao/instances/
|
||||
$ sudo reboot
|
||||
|
||||
If you were unable to successfully delete all workload VM instances
|
||||
through the UI, then on each compute node run these commands::
|
||||
|
||||
$ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem --server=<your-server-address> --network=cn --compute-net <node compute subnet> --mgmt-net <node management subnet> --hard-reset
|
||||
$ sudo killall -9 qemu-system-x86_64
|
||||
$ sudo docker rm $(sudo docker ps -qa)
|
||||
$ sudo docker network rm $(sudo docker network ls -q -f "type=custom")
|
||||
$ sudo rm -rf /var/lib/ciao/instances/
|
||||
$ sudo reboot
|
||||
|
||||
Restart your scheduler, network node launcher, compute node launcher,
|
||||
and controller.
|
||||
|
||||
Debug tips
|
||||
==========
|
||||
|
||||
For general debugging, you can:
|
||||
|
||||
* Reset you cluster.
|
||||
* Pull in updated go binaries.
|
||||
* Enable verbose console logging with ``-logtostderr -v=2`` on the go
|
||||
binaries' command lines.
|
||||
* Reduce your tenants to one (specifically the one with no limits).
|
||||
* Launch fewer VMs in a herd. A small Intel NUC with 16GB of RAM can handle as many as
|
||||
50-100 2vcpu 218MB RAM VMs starting at once per compute node. Larger dual-socket
|
||||
many threaded CPUs with hundreds of GB RAM Haswell-EP servers can handle as many as
|
||||
500 such VMs starting at once per compute node.
|
||||
* Tweak the launcher to enable remote access: go get with ``--tags=debug`` to enable
|
||||
a netcat based console redirection for each VM. The launcher console verbose output
|
||||
will indicate per VM how to connect to the console. For example::
|
||||
|
||||
$ netcat 192.168.0.102 6309
|
||||
|
||||
* Ssh into the compute node(s) by IP, looking at top, df, ps, ip a, ip r, netstat -a, etc.
|
||||
* Ssh into the CNCI(s) by IP, looking at top, df, ps, ip a, ip r, netstat -a, etc.
|
||||
* Ssh into the workload instance VMs via CNCI IP and port redirection. Each VM will be
|
||||
at a port composed from the VM's IP address added to 33000. For example::
|
||||
|
||||
33000+ip[2]<<8+ip[3]
|
||||
|
||||
The VM IP is available in the `ciao-cli`_.
|
||||
* Instance credentials for netcat or ssh connectivity depend on the contents of
|
||||
the cloud-init configuration used by ciao-controller for the workload.
|
||||
|
||||
Please contact our `mailing list`_ for more help with initial bringup and
|
||||
testing.
|
||||
|
||||
.. _ciao-controller workload_resources.csv: https://github.com/01org/ciao/blob/master/ciao-controller/workload_resources.csv
|
||||
.. _ciao-controller workload_template.csv: https://github.com/01org/ciao/blob/master/ciao-controller/workload_template.csv
|
||||
.. _downloadable installer images: https://download.clearlinux.org/image
|
||||
.. _downloadable cloud images: https://download.clearlinux.org/image
|
||||
.. _Fedora 23 Cloud: https://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2
|
||||
.. _Openstack developer: http://docs.openstack.org/developer/keystone/setup.html
|
||||
.. _go: https://golang.org/doc/articles/go_command.html
|
||||
.. _ciao-cert: https://github.com/01org/ciao/blob/master/ssntp/ciao-cert/README.md
|
||||
.. _CNCI Agent: https://github.com/01org/ciao/tree/master/networking/cnci_agent
|
||||
.. _mailing list: https://lists.clearlinux.org/mailman/listinfo/ciao-devel
|
||||
.. _ciao-cli: https://github.com/01org/ciao/tree/master/ciao-cli
|
||||
.. _ciao-webui: https://github.com/01org/ciao-webui
|
||||
@@ -1,165 +0,0 @@
|
||||
.. contents::
|
||||
|
||||
.. _deployment:
|
||||
|
||||
Deploying ciao via automation
|
||||
#############################
|
||||
|
||||
Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload
|
||||
scheduler designed to address limitations of current cloud OS projects.
|
||||
Ciao provides a lightweight, fully TLS-based minimal config, is
|
||||
workload-agnostic, easy to update, offers an optimized-for-speed
|
||||
scheduler, and is currently optimized for OpenStack*.
|
||||
|
||||
For more information, see https://clearlinux.org/ciao.
|
||||
|
||||
Environment
|
||||
===========
|
||||
|
||||
For this example, we'll use a total of five nodes:
|
||||
- A deployment machine which will be used to run the playbooks.
|
||||
- A `controller`_ node which will be used to communicate with Keystone.
|
||||
- Two `compute nodes`_, which will spawn the VMs and containers.
|
||||
- A `network node`_ which will handle the networking for the workloads.
|
||||
|
||||
.. note::
|
||||
|
||||
The deployment machine is not a necessary part of the cluster; it could be
|
||||
the sysadmin's computer or a CI/CD server.
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
Ansible* uses :command:`ssh` to run commands on the remote nodes. In order to do
|
||||
that, configure a user for passwordless SSH connections from the deployment
|
||||
container to the cluster nodes. This user must also have passwordless sudo
|
||||
privileges on the cluster nodes.
|
||||
|
||||
|
||||
Configure your cluster setup
|
||||
============================
|
||||
Ensure the following packages are installed before running the deployment
|
||||
node:
|
||||
|
||||
* ansible version 2.2.1.0 or higher
|
||||
* openstackclient
|
||||
* python-netaddr
|
||||
* gcc
|
||||
* go version 1.7 or higher
|
||||
* git
|
||||
|
||||
Install all of the needed packages with the following commands:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# swupd bundle-add sysadmin-hostmgmt c-basic go-basic
|
||||
# pip2 install python-keystoneclient
|
||||
|
||||
|
||||
You will need to download the ciao example deployment as follows:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ git clone https://github.com/01org/ciao.git
|
||||
|
||||
Once you're cloned the repo, continue working in the
|
||||
`ciao/_DeploymentAndDistroPackaging/ansible/` directory
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cd $(pwd)/ciao/_DeploymentAndDistroPackaging/ansible/
|
||||
|
||||
Next, edit the configuration files for the cluster:
|
||||
|
||||
* The `hosts`_ file is the hosts inventory file which contains the IP
|
||||
addresses/FQDN of your nodes, grouped under the roles they will serve.
|
||||
|
||||
* The `groups_vars/all`_ file contains variables that will be applied
|
||||
to your ciao setup. The mandatory variables are already there; be
|
||||
sure to change the values accordingly to fit your environment.
|
||||
|
||||
* The ``ciao_guest_key`` value in :file:`groups_var/all` is the key to be
|
||||
used to connect to the VMs created by ciao; you can use the
|
||||
``ssh-keygen`` command to create one.
|
||||
|
||||
A full list of available variables can be found in the
|
||||
:file:`defaults/main.yml` file of each role at
|
||||
https://github.com/01org/ciao/tree/master/_DeploymentAndDistroPackaging/ansible/roles.
|
||||
|
||||
|
||||
Run the playbook
|
||||
================
|
||||
|
||||
After the variables and the :file:`hosts` file are configured, start the
|
||||
deployment with the following commands:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cd ciao/_DeploymentAndDistroPackaging/ansible/
|
||||
|
||||
# ansible-playbook -i hosts ciao.yml \
|
||||
--private-key=~/.ssh/id_rsa \
|
||||
--user=<REMOTE_USER>
|
||||
|
||||
.. note::
|
||||
|
||||
Note: The playbook will create the following files in the current folder of
|
||||
the machine running the playbooks.
|
||||
|
||||
* ``./certificates``: This directory contains the certificates
|
||||
that where created and copied to the cluster nodes.
|
||||
|
||||
* ``./images``: This directory contains the images used by the
|
||||
ciao cluster (fedora, clearlinux, cnci, ovmf.fd).
|
||||
|
||||
* ``./ciaorc``: This file contains environment variables needed
|
||||
by ciao cli to authenticate to the ciao cluster.
|
||||
|
||||
* ``./openrc``: This file contains environment variables needed by
|
||||
openstack cli to authenticate with the ciao cluster.
|
||||
|
||||
Verify
|
||||
======
|
||||
After Ansible is done with the setup, you can verify the cluster is ready
|
||||
by running the following command on the controller node. Change the **username**,
|
||||
**password**, **controller**, and **identity** values to match your setup, as
|
||||
was specified in the ``groups_var/all`` file:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# ciao-cli -identity=https://ciao-controller.example.com:35357 -username ciao -password ciaoUserPassword -controller=ciao-controller.example.com node status
|
||||
Total Nodes 3
|
||||
Ready 0
|
||||
Full 3
|
||||
Offline 0
|
||||
Maintenance 0
|
||||
|
||||
You could also take a look at the ``./ciaorc`` file created on your
|
||||
deployment node, which contains the following environment variables:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cat ciaorc
|
||||
export CIAO_CONTROLLER=ciao-controller.example.com
|
||||
export CIAO_IDENTITY=https://ciao-controller.example.com:35357
|
||||
export CIAO_USERNAME=ciao
|
||||
export CIAO_PASSWORD=ciaoUserPassword
|
||||
|
||||
Then you could verify with the following command:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# source ciaorc
|
||||
# ciao-cli node status
|
||||
Total Nodes 3
|
||||
Ready 0
|
||||
Full 3
|
||||
Offline 0
|
||||
Maintenance 0
|
||||
|
||||
.. _controller: https://github.com/01org/ciao/tree/master/ciao-controller
|
||||
.. _compute nodes: https://github.com/01org/ciao/tree/master/ciao-launcher
|
||||
.. _network node: https://github.com/01org/ciao/tree/master/ciao-launcher
|
||||
.. _hosts: https://github.com/01org/ciao/blob/master/_DeploymentAndDistroPackaging/ansible/hosts
|
||||
.. _groups_vars/all: https://github.com/01org/ciao/blob/master/_DeploymentAndDistroPackaging/ansible/group_vars/all
|
||||
.. _CNCI image: https://github.com/01org/ciao/tree/master/networking/ciao-cnci-agent#cnci-agent
|
||||
.. _Docker* documentation: https://docs.docker.com/engine/reference/commandline/run/
|
||||
@@ -1,218 +0,0 @@
|
||||
.. contents::
|
||||
|
||||
.. _ubuntu:
|
||||
|
||||
Deploying ciao via automation -- Ubuntu 16.04
|
||||
#############################################
|
||||
|
||||
Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload
|
||||
scheduler designed to address limitations of current cloud OS projects.
|
||||
Ciao provides a lightweight, fully TLS-based minimal config, is
|
||||
workload-agnostic, easy to update, offers an optimized-for-speed
|
||||
scheduler, and is currently optimized for OpenStack\*.
|
||||
|
||||
For more information, see https://clearlinux.org/ciao.
|
||||
|
||||
Environment
|
||||
===========
|
||||
|
||||
For this example, we'll use a total of five nodes:
|
||||
- A deployment machine which will be used to run the playbooks.
|
||||
- A `controller`_ node which will be used to communicate with Keystone.
|
||||
- Two `compute nodes`_, which will spawn the VMs and containers.
|
||||
- A `network node`_ which will handle the networking for the workloads.
|
||||
|
||||
Note: The deployment machine is not a necessary part of the cluster; it
|
||||
could be the sysadmin's computer or a CI/CD server.
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
Ansible* uses :command:`ssh` to run commands on the remote nodes. In
|
||||
order to do that, the nodes must be configured to allow passwordless SSH
|
||||
connections from the deployment machine to the cluster nodes. The user
|
||||
should also have sudo privileges on the cluster nodes.
|
||||
|
||||
The deployment machine needs::
|
||||
|
||||
Python 2.7
|
||||
Ansible >= 2.1
|
||||
Golang >= 1.7
|
||||
qemu-utils
|
||||
python-docker
|
||||
python-openstackclient
|
||||
Ansible ciao roles from ansible-galaxy
|
||||
|
||||
The managed nodes need::
|
||||
|
||||
python 2.6 >=
|
||||
|
||||
|
||||
Proxy
|
||||
=====
|
||||
|
||||
If you are running behind a proxy, the following additional steps are needed:
|
||||
|
||||
In the deployment node
|
||||
----------------------
|
||||
|
||||
Make sure apt can run. Edit ``/etc/apt/apt.conf`` (to install ansible and
|
||||
dependencies), appending it with::
|
||||
|
||||
Acquire::http::Proxy "http://yourproxyaddress:proxyport";
|
||||
Install the CIAO roles from ansible-galaxy:
|
||||
$ sudo -E ansible-galaxy -r requirements --ignore-certs
|
||||
|
||||
In the managed nodes
|
||||
--------------------
|
||||
|
||||
Make sure apt can run. Edit ``/etc/apt/apt.conf``, appending it with::
|
||||
|
||||
Acquire::http::Proxy "http://yourproxyaddress:proxyport";
|
||||
|
||||
Install Docker and make sure you can pull images. Edit
|
||||
``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it
|
||||
with::
|
||||
|
||||
[Service]
|
||||
Environment="HTTP_PROXY=http://hostname:port/"
|
||||
Environment="HTTPS_PROXY=http://hostname:port/"
|
||||
Environment=”NO_PROXY=localhost,127.0.0.1,.example.com”
|
||||
|
||||
Note: replace the hostname and port of your proxy server and append your local domain name.
|
||||
|
||||
Reload and restart the docker daemon::
|
||||
|
||||
$ sudo systemctl daemon-reload && sudo systemctl restart docker
|
||||
|
||||
(OPTIONAL): You can download the docker images used in the CIAO deployment.
|
||||
$ sudo docker pull clearlinux/keystone
|
||||
$ sudo docker pull clearlinux/ciao-webui
|
||||
|
||||
|
||||
Install the software
|
||||
====================
|
||||
|
||||
Install ansible from ubuntu packages if the package version is >= 2.1.0;
|
||||
otherwise, install it from pip (``pip install ansible``).
|
||||
|
||||
Install Dependencies in the deployment machine
|
||||
----------------------------------------------
|
||||
|
||||
::
|
||||
|
||||
python-docker (apt-get) or docker-py (pip)
|
||||
python-openstackclient (apt-get)
|
||||
qemu-utils (apt-get)
|
||||
|
||||
Download the ``clear-config-management`` project from github::
|
||||
|
||||
$ git clone https://github.com/clearlinux/clear-config-management.git
|
||||
|
||||
|
||||
Create the playbook
|
||||
===================
|
||||
|
||||
The ``clear-config-management`` project includes some sample playbooks that
|
||||
you may use and customize for your own needs. Start by making a copy of the
|
||||
sample playbook into your home folder::
|
||||
|
||||
# cp -r clear-config-management/examples/ciao ~/
|
||||
|
||||
|
||||
Note: These files are also hosted in github
|
||||
|
||||
The relevant files in the playbook are the following:
|
||||
|
||||
* The `ciao.yml`_ file is the master playbook file and includes a playbook
|
||||
for each component of the cluster.
|
||||
|
||||
* The `hosts`_ file is the hosts inventory file and contains the IP
|
||||
addresses/FQDN of your nodes, grouped under the roles they will serve
|
||||
|
||||
* The `groups_vars/all`_ file contains variables that will be applied
|
||||
to your ciao setup. The mandatory variables are already there; be
|
||||
sure to change the values accordingly to fit your environment
|
||||
|
||||
* The ``ciao_guest_key`` value in :file:`groups_var/all` is the key
|
||||
to be used to connect to the VMs created by ciao; you can use the
|
||||
``ssh-keygen`` command to create one.
|
||||
|
||||
A full list of available variables can be found in the :file:`defaults/main.yml`
|
||||
file of each role at https://github.com/clearlinux/clear-config-management/tree/master/roles
|
||||
|
||||
|
||||
Install the required ansible-roles
|
||||
==================================
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# ansible-galaxy install -r requirements.yml
|
||||
|
||||
|
||||
Run the playbook
|
||||
================
|
||||
|
||||
Once you have your variables and hosts file configured, the deployment can
|
||||
be started with the following command:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ ansible-playbook -i hosts ciao.yml --private-key=<ssh_key>
|
||||
|
||||
Note: If you want the latest CIAO changes, change the ``ciao_dev``
|
||||
variable to ``True`` in the :file:`group_vars/all` file In the
|
||||
``clear-config-management`` project.
|
||||
|
||||
|
||||
Verify
|
||||
======
|
||||
|
||||
After ansible is done with the setup, you can verify the cluster is ready
|
||||
by running the following command on the controller node. Change the **username**,
|
||||
**password**, **controller**, and **identity** values to match your setup, as
|
||||
was specified in the ``groups_var/all`` file:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# ciao-cli -identity=https://ciao-controller.example.com:35357 \
|
||||
-username admin \
|
||||
-password secret \
|
||||
-controller=ciao-controller.example.com
|
||||
# node status
|
||||
Total Nodes 3
|
||||
Ready 0
|
||||
Full 3
|
||||
Offline 0
|
||||
Maintenance 0
|
||||
|
||||
You could also take a look at the :file:`./ciaorc` file created on your
|
||||
deployment node, which contains the following environment variables:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cat ciaorc
|
||||
export CIAO_CONTROLLER=ciao-controller.example.com
|
||||
export CIAO_IDENTITY=https://ciao-controller.example.com:35357
|
||||
export CIAO_USERNAME=csr
|
||||
export CIAO_PASSWORD=secret
|
||||
|
||||
then you could verify with the following command:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# source ciaorc
|
||||
# ciao-cli node status
|
||||
Total Nodes 3
|
||||
Ready 0
|
||||
Full 3
|
||||
Offline 0
|
||||
Maintenance 0
|
||||
|
||||
.. _controller: https://github.com/01org/ciao/tree/master/ciao-controller
|
||||
.. _compute nodes: https://github.com/01org/ciao/tree/master/ciao-launcher
|
||||
.. _network node: https://github.com/01org/ciao/tree/master/ciao-launcher
|
||||
.. _ciao.yml: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/ciao.yml
|
||||
.. _hosts: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/hosts
|
||||
.. _groups_vars/all: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/group_vars/all
|
||||
.. _github: https://github.com/clearlinux/clear-config-management/tree/master/examples/ciao
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 128 KiB |
@@ -8,7 +8,6 @@ Clear technologies - Documentation
|
||||
|
||||
clear-linux/clear-linux
|
||||
clear-containers/clear-containers
|
||||
ciao/ciao
|
||||
|
||||
License and disclaimers
|
||||
=======================
|
||||
|
||||
Reference in New Issue
Block a user