diff --git a/README.rst b/README.rst index 19bb6991..4b75bfe8 100644 --- a/README.rst +++ b/README.rst @@ -1,28 +1,27 @@ Documentation Build Instructions -================================ +################################ -The `website documentation`_ for Clear Linux* OS for Intel Architecture -should be written in :abbr:`ReStructuredText (ReST)` AKA ``.rst``, which -makes it easy to build parsable, command-line readable, indexed, and +The `website documentation`_ for Clear Linux\* OS for Intel Architecture +should be written in :abbr:`ReStructuredText (ReST)` AKA ``.rst``, which +makes it easy to build parsable, command-line readable, indexed, and search-friendly documentation and APIs with `Sphinx`_. - .. _requirements: Requirements ------------- +============ -To build documentation with Sphinx, ensure your system has these prerequisites: +To build documentation with Sphinx, ensure your system has these +prerequisites: * `GNU make`_ * `Python`_ * `PIP`_ +* `Sphinx`_ -The instructions for installing these varies according to OS. On a basic -out-of-the-box Ubuntu-like OS (which usually has Python installed by default), -check your python version - -you might need something like: +The instructions for installing these varies according to OS. On a basic out- +of-the-box Ubuntu-like OS (which usually has Python installed by default), +check your python version you might need something like: .. code-block:: console @@ -36,153 +35,23 @@ you might need something like: $ python -c 'print __import__("sphinx").__version__' 1.3.1 +Cloning the documentation repository +==================================== We have confirmed Sphinx installed. The next step is to clone Gitlab -repo to our local machine. +repository to our local machine. .. code-block:: console - $ git clone git@clrgitlab.intel.com:clr-documentation/project-docs.git - Cloning into 'project-docs'... - - - -**Note**: If the first time you've cloned ``project-docs`` is following -along with these instructions, you may skip this section; go straight -ahead to the :ref:`Run make` section. However, if you cloned an earlier -version and ran into trouble generating HTML documentation locally, try -the steps documented here. - -Before running Sphinx, we may need to correct some of the problems in the -legacy Gitlab repo. Running :command:`make` straightaway from the root of -our clone won't work. We need to delete the existing :file:`conf.py` file -and also rename the existing index file so it can generate a new one with -the correct parameters. Some files in the Gitlab repo are remnant of a build -on a Windows box, and they don't quite work on Linux. - -.. code-block:: console - - $ cd project-docs/ - $ ls - make.bat Makefile source/ - $ rm -rf Makefile make.bat - $ rm -rf source/conf.py - $ mv source/index.rst source/oldindex.rst - -In the cloned source directory, we have all the ``.rst`` files needed to -build the docs. We run a native instance of :command:`sphinx-quickstart`. -The program will run you through a series of questions. The main things to be -conscious of here are: - -* Tell it to use the existing :file:`source/` directory as the Root path for - the documentation; this is where it looks to find what it needs to generate - the HTML. -* It's better to tell it to **not** separate the source and build directories; - if you answer "y" here, Sphinx will generate *another* :file:`source/` directory, - which can be confusing. -* Running quickstart also creates a :file:`_static` directory where you - should put all images, screenshots, and other content that is linked as static - content. The builder has been known to complain about this directory if it exists - already, but it's easy to fix. -* It is only necessary to run the builder **once**. - -What follows here is a log from a successful :command:`sphinx-quickstart` build -started from within an older clone of the :file:`project-docs/` directory. Blank -answers indicate default. - -.. code-block:: console - - $ sphinx-quickstart - Welcome to the Sphinx 1.3.1 quickstart utility. - - Please enter values for the following settings (just press Enter to - accept a default value, if one is given in brackets). - - Enter the root path for documentation. - > Root path for the documentation [.]: source/ - - You have two options for placing the build directory for Sphinx output. - Either, you use a directory "_build" within the root path, or you separate - "source" and "build" directories within the root path. - > Separate source and build directories (y/n) [n]: n - - Inside the root directory, two more directories will be created; "_templates" - for custom HTML templates and "_static" for custom stylesheets and other static - files. You can enter another prefix (such as ".") to replace the underscore. - > Name prefix for templates and static dir [_]: - - The project name will occur in several places in the built documentation. - > Project name: ClearLinux Docs - > Author name(s): Intel OTC - - Sphinx has the notion of a "version" and a "release" for the - software. Each version can have multiple releases. For example, for - Python the version is something like 2.5 or 3.0, while the release is - something like 2.5.1 or 3.0a1. If you don't need this dual structure, - just set both to the same value. - > Project version: 1.0.0 - > Project release [1.0.0]: 1.0.0 - - If the documents are to be written in a language other than English, - you can select a language here by its language code. Sphinx will then - translate text that it generates into that language. - - For a list of supported codes, see - http://sphinx-doc.org/config.html#confval-language. - > Project language [en]: en - - The file name suffix for source files. Commonly, this is either ".txt" - or ".rst". Only files with this suffix are considered documents. - > Source file suffix [.rst]: .rst - - One document is special in that it is considered the top node of the - "contents tree", that is, it is the root of the hierarchical structure - of the documents. Normally, this is "index", but if your "index" - document is a custom template, you can also set this to another filename. - > Name of your master document (without suffix) [index]: - - Sphinx can also add configuration for epub output: - > Do you want to use the epub builder (y/n) [n]: n - - Please indicate if you want to use one of the following Sphinx extensions: - > autodoc: automatically insert docstrings from modules (y/n) [n]: n - > doctest: automatically test code snippets in doctest blocks (y/n) [n]: n - > intersphinx: link between Sphinx documentation of different projects (y/n) [n]: n - > todo: write "todo" entries that can be shown or hidden on build (y/n) [n]: n - > coverage: checks for documentation coverage (y/n) [n]: n - > pngmath: include math, rendered as PNG images (y/n) [n]: n - > mathjax: include math, rendered in the browser by MathJax (y/n) [n]: y - > ifconfig: conditional inclusion of content based on config values (y/n) [n]: y - > viewcode: include links to the source code of documented Python objects (y/n) [n]: y - - A Makefile and a Windows command file can be generated for you so that you - only have to run e.g. "make html" instead of invoking sphinx-build - directly. - > Create Makefile? (y/n) [y]: y - > Create Windows command file? (y/n) [n]: n - - Creating file source/conf.py. - Creating file source/index.rst. - Creating file source/Makefile. - - Finished: An initial directory structure has been created. - - You should now populate your master file source/index.rst and create other - documentation source files. Use the Makefile to build the docs, like so: - make builder - where "builder" is one of the supported builders, e.g. html, latex or - linkcheck. - - -.. _run_make: + $ git clone git@clrgitlab.intel.com:clr-documentation/project-docs.git Run make --------- +======== Finally are we ready to run :command:`make`. Be sure to :command:`cd` to the -:file:`source/` directory where your ``.rst`` files are (or will be), before -running :command:`make` ``html``, or the doc format of your choice. +:file:`source/` directory where your ``.rst`` files are, before +running :command:`make html`, or the doc format of your choice. .. code-block:: console @@ -198,10 +67,8 @@ running :command:`make` ``html``, or the doc format of your choice. Build finished. The HTML pages are in _build/html. -Open one of the .html pages in a web browser to view the rendered -documentation. If needed, you can copy the contents of the oldindex.rst -into the generated index file, re-run :command:`make`, to generate the -new HTML, and your local Table of Contents should update accordingly. +Open one of the .html pages in a web browser to view the rendered +documentation. For tips on how to contribute documentation formatted in the .rst style needed to integrate beautifully on the clearlinux.org website, please see diff --git a/source/_static/images/clearcontainersV1.svg b/source/_static/images/clearcontainersV1.svg deleted file mode 100644 index 23d68ea4..00000000 --- a/source/_static/images/clearcontainersV1.svg +++ /dev/null @@ -1,125 +0,0 @@ - - - - - - - -GitLab - - - - - - - - - - - - - - -
-
- -
-
-
-
-
-
-
-You need to sign in before continuing. -
-
- -
-
- - -
-
- - -
-
-
-
-
-
- -
- - diff --git a/source/_static/images/clearcontainersV2.svg b/source/_static/images/clearcontainersV2.svg deleted file mode 100644 index 7658c0e4..00000000 --- a/source/_static/images/clearcontainersV2.svg +++ /dev/null @@ -1,125 +0,0 @@ - - - - - - - -GitLab - - - - - - - - - - - - - - -
-
- -
-
-
-
-
-
-
-You need to sign in before continuing. -
-
- -
-
- - -
-
- - -
-
-
-
-
-
- -
- - diff --git a/source/_static/images/dax-V1.svg b/source/_static/images/dax-V1.svg deleted file mode 100644 index 9c0d54ad..00000000 --- a/source/_static/images/dax-V1.svg +++ /dev/null @@ -1,125 +0,0 @@ - - - - - - - -GitLab - - - - - - - - - - - - - - -
-
- -
-
-
-
-
-
-
-You need to sign in before continuing. -
-
- -
-
- - -
-
- - -
-
-
-
-
-
- -
- - diff --git a/source/_static/images/dax-V2.svg b/source/_static/images/dax-V2.svg deleted file mode 100644 index 24d07802..00000000 --- a/source/_static/images/dax-V2.svg +++ /dev/null @@ -1,125 +0,0 @@ - - - - - - - -GitLab - - - - - - - - - - - - - - -
-
- -
-
-
-
-
-
-
-You need to sign in before continuing. -
-
- -
-
- - -
-
- - -
-
-
-
-
-
- -
- - diff --git a/source/_static/images/image-blob-ciao-networking.png b/source/_static/images/image-blob-ciao-networking.png deleted file mode 100644 index a572ef4d..00000000 Binary files a/source/_static/images/image-blob-ciao-networking.png and /dev/null differ diff --git a/source/_static/images/image-blob-ciao-networking.svg b/source/_static/images/image-blob-ciao-networking.svg deleted file mode 100644 index a76c4ae2..00000000 --- a/source/_static/images/image-blob-ciao-networking.svg +++ /dev/null @@ -1,766 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - image-blob-ciao-networking - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/source/_static/images/openstack_example_architecture.jpg b/source/_static/images/openstack_example_architecture.jpg deleted file mode 100644 index 5323d796..00000000 Binary files a/source/_static/images/openstack_example_architecture.jpg and /dev/null differ diff --git a/source/ceph-deploy.rst b/source/ceph-deploy.rst deleted file mode 100644 index 0c8e1cc5..00000000 --- a/source/ceph-deploy.rst +++ /dev/null @@ -1,210 +0,0 @@ -.. _ceph-deploy: - -Ceph* -##### - -Ceph is a distributed storage system designed to scale well. It can be -integrated with OpenStack* as the storage backend for the OS images and -volumes. - -Deploying a Ceph storage cluster is simple using Clear Linux* OS for -Intel® Architecture and Ansible*. - - -Environment -=========== - -For this example, we'll use a total of six nodes: a **deployment node**, -to run the playbooks, a **monitor node**, a **metadata node** and three -**storage nodes**. - -Install each component on its own server for best results; however, -for testing purposes you can install the monitor, metadata, and storage -nodes all on the same host. - - -Prerequisites -============= - -Ansible uses ``ssh`` to run commands on the remote servers. In order to -do that, the servers must be configured to allow passwordless ssh -connections from the root user. Follow these steps to configure -your nodes. - -On your cluster nodes ---------------------- - -#. Enable root login:: - - # echo "PermitRootLogin yes" >> /etc/ssh/sshd_config - -#. Enable sshd service:: - - # systemctl enable sshd - # systemctl start sshd - -On your deployment node ------------------------ - -#. Generate an ssh key if you have not already:: - - # ssh-keygen - -#. Copy your public key (:file:`~/.ssh/id_rsa.pub`) to :file:`/root/.ssh/authorized_keys` on each node:: - - # ssh-copy-id root@node - - -Install the software -==================== - -Install the `sysadmin-hostmgmt`_ bundle on the deployment node. This -bundle contains the Ansible software required to run the playbooks, as -well as some Ansible roles and sample playbooks that you can use to -build your own:: - - # swupd bundle-add sysadmin-hostmgmt - - -Create the playbook -=================== - -The ``sysadmin-hostmgmt`` bundle includes some sample playbooks that -can be customized for your own needs. Start by making a copy of the -sample playbook into your home folder.:: - - # cp -r /usr/share/ansible/examples/ceph ~/ - -The playbook consist of four files that you should modify to fit -your needs:: - - ceph - |-- group_vars/ - | |-- all - | +-- mons - |-- hosts - +-- ceph_deploy.yml - -The ``hosts`` file contains the IP addresses of your servers grouped -under the roles they will serve:: - - [mons] - 172.28.128.7 - - [osds] - 172.28.128.8 - 172.28.128.9 - 172.28.128.10 - - [mdss] - 172.28.128.11 - -This :file:`groups_var/all` file contains variables that will be applied -to all your nodes. The mandatory variables are already there; be sure -to change the values accordingly to fit your environment. It should -look something like this:: - - --- - journal_size: 1024 - monitor_interface: enp0s8 - public_network: 172.28.128.0/24 - cluster_network: "{{ public_network }}" - -A full list of available variables can be found under each role -in ``defaults/main.yml``:: - - * `ceph-common/defaults/main.yml`_ - * `ceph-mon/defaults/main.yml`_ - * `ceph-mds/defaults/main.yml`_ - * `ceph-osd/defaults/main.yml`_ - -This :file:`groups_var/osd` file contains variables that apply only -to the hosts under the ``[mons]`` section in your hosts file. You can -choose one of the three available scenarios for this playbook. - -#. **Journal and osd_data on the same device**: This will co-locate both - journal and data on the same disk, creating a partition at the - beginning of the device:: - - journal_collocation: true - devices: - - /dev/sdb - - /dev/sdc - - /dev/sdd - -#. **N journal devices for N OSDs**: In this example, the ``sdb`` - partition will be used for journaling of ``sdc``. The ``sdd sdf`` - will be used for journaling of ``sde``:: - - raw_multi_journal: true - devices: - - /dev/sdc - - /dev/sdd - - /dev/sde - raw_journal_devices: - - /dev/sdb - - /dev/sdb - - /dev/sdf - -#. **Specify a directory instead of disk for OSDs**:: - - osd_directory: true - osd_directories: - - /var/lib/ceph/osd/mydir1 - - /var/lib/ceph/osd/mydir2 - - /var/lib/ceph/osd/mydir3 - - Note: The directories should reside on an XFS filesystem. EXT4 is not supported. - - -Run the playbook -================ - -Once the variables and hosts file is configured, -deployment is as simple as issuing the command:: - - # ansible-playbook -i hosts ceph_deploy.yml - - -Verify -====== - -After Ansible has finished deployment, you may -like to verify and watch the health of the cluster -with Ceph utilites such as ``ceph status`` and -``ceph osd tree``:: - - # ceph status - cluster ee1fae3b-b95b-494c-abd7-f0629d113446 - health HEALTH_OK - monmap e1: 1 mons at {node2=172.28.128.5:6789/0} - election epoch 2, quorum 0 node2 - osdmap e8: 3 osds: 3 up, 3 in - flags sortbitwise - pgmap v14: 64 pgs, 1 pools, 0 bytes data, 0 objects - 7566 MB used, 49647 MB / 59896 MB avail - 64 active+clean - -:: - - # ceph osd tree - ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY - -1 0.05699 root default - -2 0.01900 host node3 - 0 0.01900 osd.0 up 1.00000 1.00000 - -3 0.01900 host node4 - 1 0.01900 osd.1 up 1.00000 1.00000 - -4 0.01900 host node5 - 2 0.01900 osd.2 up 1.00000 1.00000 - -:: - - # ceph fs ls - name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] - -.. _ceph-common/defaults/main.yml: https://github.com/clearlinux/clear-config-management/blob/master/roles/ceph-common/defaults/main.yml -.. _ceph-mon/defaults/main.yml: https://github.com/clearlinux/clear-config-management/blob/master/roles/ceph-mon/defaults/main.yml -.. _ceph-mds/defaults/main.yml: https://github.com/clearlinux/clear-config-management/blob/master/roles/ceph-mds/defaults/main.yml -.. _ceph-osd/defaults/main.yml: https://github.com/clearlinux/clear-config-management/blob/master/roles/ceph-osd/defaults/main.yml -.. _sysadmin-hostmgmt: https://github.com/clearlinux/clr-bundles/blob/master/bundles/sysadmin-hostmgmt - diff --git a/source/ciao/ciao.rst b/source/ciao/ciao.rst new file mode 100644 index 00000000..2b28d3c7 --- /dev/null +++ b/source/ciao/ciao.rst @@ -0,0 +1,12 @@ +.. _ciao: + +ciao +#### + +.. toctree:: + :maxdepth: 2 + + cluster-setup-quick + cluster-setup + deployment/deployment + deployment/ubuntu \ No newline at end of file diff --git a/source/ciao-cluster-setup-quick.rst b/source/ciao/cluster-setup-quick.rst similarity index 55% rename from source/ciao-cluster-setup-quick.rst rename to source/ciao/cluster-setup-quick.rst index bd997435..6033d7b6 100644 --- a/source/ciao-cluster-setup-quick.rst +++ b/source/ciao/cluster-setup-quick.rst @@ -1,4 +1,4 @@ -.. _ciao-cluster-setup-quick: +.. _cluster-setup-quick: Easy ciao development cluster setup ################################### @@ -7,6 +7,5 @@ For quick ciao development cluster setup, see: https://github.com/01org/ciao/wiki/Single-Machine-Development-Environment -For more detailed and configurable manual setup of ciao, see: - - https://clearlinux.org/documentation/ciao-cluster-setup.html +For more detailed and configurable manual setup of ciao, see +:ref:`cluster-setup`. diff --git a/source/ciao-cluster-setup.rst b/source/ciao/cluster-setup.rst similarity index 95% rename from source/ciao-cluster-setup.rst rename to source/ciao/cluster-setup.rst index fda12aad..84eda71c 100644 --- a/source/ciao-cluster-setup.rst +++ b/source/ciao/cluster-setup.rst @@ -1,16 +1,17 @@ -.. _ciao-cluster-setup: +.. contents:: -.. contents:: +.. _cluster-setup: Detailed manual ciao cluster setup ################################## -This topic explains how to set up a cluster of machines running Clear Linux* OS -for Intel® Architecture with :abbr:`Cloud Integrated Advanced Orchestrator (CIAO)`, or ciao. +This topic explains how to set up a cluster of machines running Clear Linux\* +OS for Intel® Architecture with the +:abbr:`Cloud Integrated Advanced Orchestrator (CIAO)`. -While the table of contents provides links to specific points of information, this -topic is intended as an ordered workflow. Be sure to start your cluster components -in the correct order as explained below. +While the table of contents provides links to specific points of information, +this topic is intended as an ordered workflow. Be sure to start your cluster +components in the correct order as explained below. Infrastructure prerequisites ============================ @@ -18,13 +19,15 @@ Infrastructure prerequisites Hardware needs -------------- -You'll need at least four machines and a switch connecting them to form -your beginning ciao cluster. The switch is assumed to be plugged directly -into an "upstream" network running a DHCP server. See the illustration below as an example: +You'll need at least four machines and a switch connecting them to form your +beginning ciao cluster. The switch is assumed to be plugged directly into an +"upstream" network running a DHCP server. See the illustration below as an +example: -.. image:: image-blob-ciao-networking.png +.. image:: ./figures/image-blob-ciao-networking.png -The following examples assume you have four nodes on a ``192.168.0.0/16`` network: +The following examples assume you have four nodes on a ``192.168.0.0/16`` +network: Controller node ~~~~~~~~~~~~~~~ @@ -38,7 +41,8 @@ Network node ("nn") * IP ``192.168.0.102`` * Runs Launcher with ``--network=nn`` option -* Has CNCI image in ``/var/lib/ciao/images``. See below for more on CNCI image preparation. +* Has CNCI image in ``/var/lib/ciao/images``. See below for more on CNCI + image preparation. Compute node 1 ("cn1") ~~~~~~~~~~~~~~~~~~~~~~ @@ -71,7 +75,7 @@ Install Clear Linux OS for Intel Architecture as host on all nodes Install Clear Linux OS for Intel Architecture as the host OS on all nodes by following the instructions in the topic -:ref:`gs_installing_clr_as_host`. The current `downloadable installer images`_ +:ref:`clear-host`. The current `downloadable installer images`_ are compatible with ciao. After the base installation on each node, add the following additional @@ -402,15 +406,15 @@ variables and override them: Start a workload ================ -As a valid user, the `ciao-cli`tool allows you to start a workload. +As a valid user, the `ciao-cli` tool allows you to start a workload. First, you may want to know which workloads are available:: - $ ciao-cli workload list + $ ciao-cli workload list Then you can launch one or more workloads:: - $ ciao-cli instance add -workload -instances + $ ciao-cli instance add -workload -instances And you can monitor all your instances statuses (``pending`` or ``running``):: @@ -433,14 +437,14 @@ described above. Access your workload ==================== -Once your workload is up, you need to know its IP address and assigned +Once your workload is up, you need to know its IP address and assigned port, you can find it via ciao-cli:: $ ciao-cli instance list -Then look for the column "SSH IP", there is the IP assigned to your -workload, next to it, you will see the column SSH PORT; that's the -port that you will use to access a specific workload using your +Then look for the column "SSH IP", there is the IP assigned to your +workload, next to it, you will see the column SSH PORT; that's the +port that you will use to access a specific workload using your private key defined in the cloud-init configuration and the demo user name ``demouser``:: diff --git a/source/ciao-deploy.rst b/source/ciao/deployment/deployment.rst similarity index 99% rename from source/ciao-deploy.rst rename to source/ciao/deployment/deployment.rst index 90e3e4ca..3c4f6915 100644 --- a/source/ciao-deploy.rst +++ b/source/ciao/deployment/deployment.rst @@ -1,7 +1,7 @@ -.. _ciao-deploy: - .. contents:: +.. _deployment: + Deploying ciao via automation ############################# @@ -27,8 +27,6 @@ For this example, we'll use a total of five nodes: The deployment machine is not a necessary part of the cluster; it could be the sysadmin's computer or a CI/CD server. -.. _prerequisites: - Prerequisites ============= Ansible* uses :command:`ssh` to run commands on the remote nodes. In order to do diff --git a/source/ciao-deploy-ubuntu.rst b/source/ciao/deployment/ubuntu.rst similarity index 92% rename from source/ciao-deploy-ubuntu.rst rename to source/ciao/deployment/ubuntu.rst index 0c28811f..f9e29b1d 100644 --- a/source/ciao-deploy-ubuntu.rst +++ b/source/ciao/deployment/ubuntu.rst @@ -1,7 +1,7 @@ -.. _ciao-deploy: - .. contents:: +.. _ubuntu: + Deploying ciao via automation -- Ubuntu 16.04 ############################################# @@ -9,7 +9,7 @@ Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload scheduler designed to address limitations of current cloud OS projects. Ciao provides a lightweight, fully TLS-based minimal config, is workload-agnostic, easy to update, offers an optimized-for-speed -scheduler, and is currently optimized for OpenStack*. +scheduler, and is currently optimized for OpenStack\*. For more information, see https://clearlinux.org/ciao. @@ -22,17 +22,15 @@ For this example, we'll use a total of five nodes: - Two `compute nodes`_, which will spawn the VMs and containers. - A `network node`_ which will handle the networking for the workloads. - Note: The deployment machine is not a necessary part of the cluster; it could be - the sysadmin's computer or a CI/CD server. - -.. _prerequisites: + Note: The deployment machine is not a necessary part of the cluster; it + could be the sysadmin's computer or a CI/CD server. Prerequisites ============= -Ansible* uses :command:`ssh` to run commands on the remote nodes. In -order to do that, the nodes must be configured to allow passwordless SSH -connections from the deployment machine to the cluster nodes. The user +Ansible* uses :command:`ssh` to run commands on the remote nodes. In +order to do that, the nodes must be configured to allow passwordless SSH +connections from the deployment machine to the cluster nodes. The user should also have sudo privileges on the cluster nodes. The deployment machine needs:: @@ -46,7 +44,7 @@ The deployment machine needs:: Ansible ciao roles from ansible-galaxy The managed nodes need:: - + python 2.6 >= @@ -58,7 +56,7 @@ If you are running behind a proxy, the following additional steps are needed: In the deployment node ---------------------- -Make sure apt can run. Edit ``/etc/apt/apt.conf`` (to install ansible and +Make sure apt can run. Edit ``/etc/apt/apt.conf`` (to install ansible and dependencies), appending it with:: Acquire::http::Proxy "http://yourproxyaddress:proxyport"; @@ -71,9 +69,9 @@ In the managed nodes Make sure apt can run. Edit ``/etc/apt/apt.conf``, appending it with:: Acquire::http::Proxy "http://yourproxyaddress:proxyport"; - -Install Docker and make sure you can pull images. Edit -``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it + +Install Docker and make sure you can pull images. Edit +``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it with:: [Service] @@ -95,7 +93,7 @@ Reload and restart the docker daemon:: Install the software ==================== -Install ansible from ubuntu packages if the package version is >= 2.1.0; +Install ansible from ubuntu packages if the package version is >= 2.1.0; otherwise, install it from pip (``pip install ansible``). Install Dependencies in the deployment machine @@ -115,8 +113,8 @@ Download the ``clear-config-management`` project from github:: Create the playbook =================== -The ``clear-config-management`` project includes some sample playbooks that -you may use and customize for your own needs. Start by making a copy of the +The ``clear-config-management`` project includes some sample playbooks that +you may use and customize for your own needs. Start by making a copy of the sample playbook into your home folder:: # cp -r clear-config-management/examples/ciao ~/ @@ -136,11 +134,11 @@ The relevant files in the playbook are the following: to your ciao setup. The mandatory variables are already there; be sure to change the values accordingly to fit your environment - * The ``ciao_guest_key`` value in :file:`groups_var/all` is the key - to be used to connect to the VMs created by ciao; you can use the + * The ``ciao_guest_key`` value in :file:`groups_var/all` is the key + to be used to connect to the VMs created by ciao; you can use the ``ssh-keygen`` command to create one. -A full list of available variables can be found in the :file:`defaults/main.yml` +A full list of available variables can be found in the :file:`defaults/main.yml` file of each role at https://github.com/clearlinux/clear-config-management/tree/master/roles @@ -162,8 +160,8 @@ be started with the following command: $ ansible-playbook -i hosts ciao.yml --private-key= -Note: If you want the latest CIAO changes, change the ``ciao_dev`` -variable to ``True`` in the :file:`group_vars/all` file In the +Note: If you want the latest CIAO changes, change the ``ciao_dev`` +variable to ``True`` in the :file:`group_vars/all` file In the ``clear-config-management`` project. @@ -178,9 +176,9 @@ was specified in the ``groups_var/all`` file: .. code-block:: console # ciao-cli -identity=https://ciao-controller.example.com:35357 \ - -username admin \ - -password secret \ - -controller=ciao-controller.example.com + -username admin \ + -password secret \ + -controller=ciao-controller.example.com # node status Total Nodes 3 Ready 0 diff --git a/source/image-blob-ciao-networking.png b/source/ciao/figures/image-blob-ciao-networking.png similarity index 100% rename from source/image-blob-ciao-networking.png rename to source/ciao/figures/image-blob-ciao-networking.png diff --git a/source/clear-containers.rst b/source/clear-containers/architecture-overview.rst similarity index 86% rename from source/clear-containers.rst rename to source/clear-containers/architecture-overview.rst index f3d5f6cd..d887b412 100644 --- a/source/clear-containers.rst +++ b/source/clear-containers/architecture-overview.rst @@ -1,36 +1,9 @@ -.. _clear_containers.rst: - -Intel® Clear Containers -####################### - -Introduction -============ - -Intel® Clear Containers is a collection of tools, configurations, -and techniques anchored on an implementation that leverages Intel® -Architecture to optimize container launching and execution workflow. -These optimizations improve speed, size, and efficiency while offering -a number of benefits that can be derived only from hardware-backed -virtual machines (hardware-enforced isolation and security, for -example) on Intel® VT technology. - -These methods are applied across all levels of the host/virtual machine -hierarchy: from the host-side userland software stack down through the host -Linux* kernel, and into the client-side kernel and userland. - -Although it is available as a standalone offering, the Clear Containers -technology works best when it is able to leverage optimizations designed -into the Clear Linux Project. - -Customers can integrate all or parts of Intel Clear Containers into a -container infrastructure. - -.. _architecture_overview.rst: +.. _architecture-overview: Architecture Overview -===================== +##################### -Intel Clear Containers are architected around the Linux +Intel Clear Containers are architected around the Linux :abbr:`Kernel Virtual Machine (KVM)` virtualization infrastructure to make best use of Intel Architecture VT features. Operational speed gets improved and overhead gets reduced by optimizing existing code, @@ -38,25 +11,83 @@ removing redundant components, and implementing new techniques for containers with :abbr:`KVM (Kernel Virtual Machine)`. Version 1.0 of Clear Containers was designed as a lightweight container -system based around `kvmtool`_'s ``lkvm``, +system based around `kvmtool`_'s ``lkvm``, :abbr:`KVM (Kernel Virtual Machine)` and Intel VT-x features; the -initial version was aimed primarily at Docker* integration. Version -2.0 replaces ``lkvm`` with a lightweight version of +initial version was aimed primarily at Docker\* integration. Version +2.0 replaces ``lkvm`` with a lightweight version of :abbr:`QEMU (Quick EMUlator)` `(link) `_. Version 2.0 also expands the feature set to include key technologies, such as `SR-IOV`_, and the :abbr:`Open Container Initiative (OCI)` runtime API. - - -V1.0 +V2.0 ==== -V1.0 (also known as **Intel® Clear Containers for Docker -Engine**) is based around `kvmtool`_, with example host integrations for -Docker and `rkt`_. +Intel Clear Containers V2.0 adopts an optimized version of the established +`QEMU`_ host virtualization engine, in order to support extra features not +found in Clear Containers V1.0. Clear Containers. V2.0 is also compatible with +the :abbr:`OCI (Open Container Initiative)` runtime-specification standard, +introducing a host-side abstraction tool to ease host-side integration and to +isolate integration instances from future changes to the underlying Clear +Containers architecture. -.. figure:: _static/images/clear-containers-v1.png +.. figure:: ./figures/clear-containers-v2.png + :align: center + :alt: Clear Containers V2.0 + +Host kernel optimizations +------------------------- + +V2.0 host kernel optimizations are currently the same as +the V1.0 optimizations. + +Host user space +--------------- + +Host user space is based around an optimized version of `QEMU`_ called +``qemu-lite``, with an :abbr:`OCI (Open Container Initiative)` +runtime-compliant wrapper called ``cor``. + +Our version of ``qemu-lite`` has the following modifications: + +* :abbr:`DAX (Direct Access)` support, **enabling fast and space efficient** + file access through zero-copy mapping and multi-container sharing of raw + client filesystem images from the host filesystem. +* **Reduced "slimline" PC model** to reduce startup costs in both `QEMU`_ + and the client kernel. +* **Removed need for BIOS**, saving boot time. +* **No bootloader requirement**, to speed up boot. +* **Reduced memory footprint** by disabling memory-hungry features that + are not required by the client system. +* **Direct kernel boot**, allowing fast booting by loading the kernel as + an uncompressed ELF binary. Although the kernel image is slightly larger + than a compressed one, it is faster to read and boot the larger + file than it is to uncompress and boot the slightly smaller file. +* **Added an** :abbr:`OCI (Open Container Initiative)` **runtime-compliant + wrapper**, AKA ``cor``, for easier integration with + :abbr:`OCI (Open Container Initiative)`-compliant host orchestration systems. + +Client mini-OS +-------------- + +The Client mini-OS is based on the same Clear Linux OS-based system as +used in Intel Clear Containers V1.0; however, it may be built from more +recent versions and with more current components, such as the kernel version. + +Client customer images +---------------------- + +Client customer images are supported in the same manner as they are +in V1.0. + + +Legacy V1.0 +=========== + +V1.0 (also known as **Intel® Clear Containers for Docker Engine**) is based +around `kvmtool`_, with example host integrations for Docker and `rkt`_. + +.. figure:: ./figures/clear-containers-v1.png :align: center :alt: Intel Clear Containers V1.0 @@ -71,7 +102,7 @@ optimizations are applied: is recommended for efficient page sharing of VM pages. Kernel documentation can be found in Documentation/vm/ksm.txt Config symbol: ``CONFIG_KSM`` * Using a kernel version >= v4.0 (or backporting appropriate - patches if your kernel version is less than v4.0), to get the best + patches if your kernel version is less than v4.0), to get the best :abbr:`KVM (Kernel Virtual Machine)` VM startup times .. note:: @@ -86,8 +117,8 @@ optimizations are applied: Host user space --------------- -Intel Clear Containers V1.0 host user space is based around `kvmtool`_ as a fast -and lightweight hypervisor. Optimizations to `kvmtool`_ include: +Intel Clear Containers V1.0 host user space is based around `kvmtool`_ as a +fast and lightweight hypervisor. Optimizations to `kvmtool`_ include: * **File access**, enabling efficient *shmem* / *pci-bar* / :abbr:`Direct Access (DAX)` file access to client. @@ -108,10 +139,10 @@ and lightweight hypervisor. Optimizations to `kvmtool`_ include: Client mini-OS -------------- -Intel Clear Containers V1.0 uses an optimized client user space (mini-OS) as its -primary launch vehicle to execute workload commands. The mini-OS is built -with a Clear Linux distribution that has an optimized configuration for -time and space efficiency. The mini-OS includes: +Intel Clear Containers V1.0 uses an optimized client user space (mini-OS) as +its primary launch vehicle to execute workload commands. The mini-OS is built +with a Clear Linux distribution that has an optimized configuration for time +and space efficiency. The mini-OS includes: * Minimized ``systemd`` configuration * Optimized ``libc`` @@ -142,69 +173,6 @@ unless equivalent optimizations exist in the customer-created image, or have been migrated into the image they create. - -V2.0 -==== - -Intel Clear Containers V2.0 adopts an optimized version of the established `QEMU`_ -host virtualization engine, in order to support extra features not found in -Clear Containers V1.0. Clear Containers. V2.0 is also compatible with the -:abbr:`OCI (Open Container Initiative)` runtime-specification standard, -introducing a host-side abstraction tool to ease host-side integration and to -isolate integration instances from future changes to the underlying Clear -Containers architecture. - -.. figure:: _static/images/clear-containers-v2.png - :align: center - :alt: Clear Containers V2.0 - -Host kernel optimizations -------------------------- - -V2.0 host kernel optimizations are currently the same as -the V1.0 optimizations. - -Host user space ---------------- - -Host user space is based around an optimized version of `QEMU`_ called -``qemu-lite``, with an :abbr:`OCI (Open Container Initiative)` -runtime-compliant wrapper called ``cor``. - -Our version of ``qemu-lite`` has the following modifications: - -* :abbr:`DAX (Direct Access)` support, **enabling fast and space efficient** - file access through zero-copy mapping and multi-container sharing of raw - client filesystem images from the host filesystem. -* **Reduced "slimline" PC model** to reduce startup costs in both `QEMU`_ - and the client kernel. -* **Removed need for BIOS**, saving boot time. -* **No bootloader requirement**, to speed up boot. -* **Reduced memory footprint** by disabling memory-hungry features that - are not required by the client system. -* **Direct kernel boot**, allowing fast booting by loading the kernel as - an uncompressed ELF binary. Although the kernel image is slightly larger - than a compressed one, it is faster to read and boot the larger - file than it is to uncompress and boot the slightly smaller file. -* **Added an** :abbr:`OCI (Open Container Initiative)` **runtime-compliant - wrapper**, AKA ``cor``, for easier integration with - :abbr:`OCI (Open Container Initiative)`-compliant host orchestration systems. - -Client mini-OS --------------- - -The Client mini-OS is based on the same Clear Linux OS-based system as -used in Intel Clear Containers V1.0; however, it may be built from more -recent versions and with more current components, such as the kernel version. - -Client customer images ----------------------- - -Client customer images are supported in the same manner as they are -in V1.0. - - - Architectural component details =============================== @@ -220,7 +188,7 @@ Linux Kernel Documentation: Documentation/vm/ksm.txt and merge (share) identical memory pages within the system, even when they are not sourced from the same binary. When sourced from the same binary, the kernel will naturally share through the -:abbr:`copy-on-write (COW)` method. +:abbr:`copy-on-write (COW)` method. :abbr:`KSM (Kernel Samepage Merging)` also allows the kernel to localize and to coalesce pages from within virtual machine memory @@ -352,7 +320,7 @@ mini-OS consists of a Linux kernel image and root filesystem image. re-built as desired, for specific requirements. * **DAX** -- The :abbr:`Direct Access (DAX)` filesystem. - (Linux Kernel Documentation: ``Documentation/filesystems/dax.txt``). + (Linux Kernel Documentation: ``Documentation/filesystems/dax.txt``). Mapping host-side files into the memory map of the client allows the use of :abbr:`DAX (Direct Access)` to directly mount those files, bypassing the client side page cache and the virtual device mechanisms between host and @@ -362,14 +330,14 @@ mini-OS consists of a Linux kernel image and root filesystem image. (Direct Access)` is enabled in Intel Clear Containers V1.0 using a shmem PCI-BAR mechanism configured by `kvmtool`_. - .. figure:: _static/images/dax-v1.png - :align: center + .. figure:: ./figures/dax-v1.png + :align: center - :abbr:`DAX (Direct Access)` is enabled in Intel Clear Containers + :abbr:`DAX (Direct Access)` is enabled in Intel Clear Containers V2.0 using an NVDIMM `QEMU`_ memdev mechanism: - .. figure:: _static/images/dax-v2.png - :align: center + .. figure:: ./figures/dax-v2.png + :align: center :abbr:`DAX (Direct Access)` can only be used to mount single flat files from the host side (such as uncompressed filesystems), and not trees of @@ -396,7 +364,7 @@ the mini-OS to execute them using the mini-OS workload. Please refer to the `Intel Clear Containers integration guide`_ for further detail. -.. removed this section since it is in the GSG +.. removed this section since it is in the GSG FAQ === @@ -426,14 +394,10 @@ FAQ those that are based around a container style workflow. - - - - .. _SR-IOV: http://www.intel.com/content/www/us/en/pci-express/pci-sig-sr-iov-primer-sr-iov-technology-paper.html .. _QEMU: http://www.qemu.org .. _mmu.txt: Documentation/virtual/kvm/mmu.txt .. _Intel ARK website: http://ark.intel.com .. _kvmtool: https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/ .. _rkt: https://coreos.com/rkt/ -.. _Intel Clear Containers integration guide: https://clearlinux.org/documentation/gs-clear-containers-getting-started.html \ No newline at end of file +.. _Intel Clear Containers integration guide: https://clearlinux.org/documentation/cc-getting-started.html diff --git a/source/clear-containers/clear-containers.rst b/source/clear-containers/clear-containers.rst new file mode 100644 index 00000000..9b74c090 --- /dev/null +++ b/source/clear-containers/clear-containers.rst @@ -0,0 +1,29 @@ +.. _clear-containers.rst: + +Intel® Clear Containers +####################### + +Intel® Clear Containers is a collection of tools, configurations, +and techniques anchored on an implementation that leverages Intel® +Architecture to optimize container launching and execution workflow. +These optimizations improve speed, size, and efficiency while offering +a number of benefits that can be derived only from hardware-backed +virtual machines (hardware-enforced isolation and security, for +example) on Intel® VT technology. + +These methods are applied across all levels of the host/virtual machine +hierarchy: from the host-side userland software stack down through the host +Linux\* kernel, and into the client-side kernel and userland. + +Although it is available as a standalone offering, the Clear Containers +technology works best when it is able to leverage optimizations designed +into the Clear Linux Project. + +Customers can integrate all or parts of Intel Clear Containers into a +container infrastructure. + +.. toctree:: + :maxdepth: 2 + + getting-started + architecture-overview diff --git a/source/_static/images/clear-containers-v1.png b/source/clear-containers/figures/clear-containers-v1.png similarity index 100% rename from source/_static/images/clear-containers-v1.png rename to source/clear-containers/figures/clear-containers-v1.png diff --git a/source/_static/images/clear-containers-v2.png b/source/clear-containers/figures/clear-containers-v2.png similarity index 100% rename from source/_static/images/clear-containers-v2.png rename to source/clear-containers/figures/clear-containers-v2.png diff --git a/source/_static/images/dax-v1.png b/source/clear-containers/figures/dax-v1.png similarity index 100% rename from source/_static/images/dax-v1.png rename to source/clear-containers/figures/dax-v1.png diff --git a/source/_static/images/dax-v2.png b/source/clear-containers/figures/dax-v2.png similarity index 100% rename from source/_static/images/dax-v2.png rename to source/clear-containers/figures/dax-v2.png diff --git a/source/gs-clear-containers-getting-started.rst b/source/clear-containers/getting-started.rst similarity index 67% rename from source/gs-clear-containers-getting-started.rst rename to source/clear-containers/getting-started.rst index b6f038e1..d8b7f552 100644 --- a/source/gs-clear-containers-getting-started.rst +++ b/source/clear-containers/getting-started.rst @@ -1,4 +1,4 @@ -.. _gs-clear-containers-getting-started: +.. _cc-getting-started: Clear Containers getting started guide ###################################### @@ -12,12 +12,15 @@ This release enables executing existing Docker applications in the secure and fast Intel Clear Containers environment under Docker\* 1.12 and beyond via an :abbr:`Open Container Initiative (OCI)` compatible `runtime`. +Visit our :ref:`architecture-overview` for detailed architectural +information. + Installation instructions ========================= -The primary host platform is Clear Linux* Project for Intel® -Architecture. For instructions on installing Docker and Clear Containers under -Clear Linux, please refer to instructions from the runtime source tree: +The primary host platform is Clear Linux\* Project for Intel® Architecture. +For instructions on installing Docker and Clear Containers under Clear Linux, +please refer to instructions from the runtime source tree: • https://github.com/01org/cc-oci-runtime/wiki/Installation @@ -36,12 +39,3 @@ qemu-lite are publicly hosted on github: - https://github.com/01org/cc-oci-runtime - https://github.com/01org/qemu-lite -Architecture Overview -===================== - -Please see the the `Architecture Overview`_ section. - - - -.. _Architecture Overview: https://clearlinux.org/documentation/clear-containers.html#architecture-overview - diff --git a/source/clear-linux/bundles/bundles.rst b/source/clear-linux/bundles/bundles.rst new file mode 100644 index 00000000..98b0416a --- /dev/null +++ b/source/clear-linux/bundles/bundles.rst @@ -0,0 +1,10 @@ +Bundles +####### + +.. toctree:: + :maxdepth: 2 + + overview + containers-basic + openssh-server + os-core diff --git a/source/bdl-containers-basic.rst b/source/clear-linux/bundles/containers-basic.rst similarity index 97% rename from source/bdl-containers-basic.rst rename to source/clear-linux/bundles/containers-basic.rst index 8c07bff4..37f40fcf 100644 --- a/source/bdl-containers-basic.rst +++ b/source/clear-linux/bundles/containers-basic.rst @@ -1,4 +1,4 @@ -.. _bdl-containers-basic: +.. _containers-basic: containers-basic ################ diff --git a/source/bdl-openssh-server.rst b/source/clear-linux/bundles/openssh-server.rst similarity index 100% rename from source/bdl-openssh-server.rst rename to source/clear-linux/bundles/openssh-server.rst diff --git a/source/bdl-os-core.rst b/source/clear-linux/bundles/os-core.rst similarity index 96% rename from source/bdl-os-core.rst rename to source/clear-linux/bundles/os-core.rst index 2b477906..c1959554 100644 --- a/source/bdl-os-core.rst +++ b/source/clear-linux/bundles/os-core.rst @@ -1,4 +1,4 @@ -.. _bdl-os-core: +.. _os-core: os-core ####### @@ -14,7 +14,7 @@ To configure a static IP you should follow the next steps: does not exist, please create it. #. The minimum lines the :file:`50-static.network` file should contain are:: - + [Match] Name= @@ -27,21 +27,21 @@ To configure a static IP you should follow the next steps: #. If you want to add more options you can consult the `systemd network configuration`_ manual. -#. Restart *systemd-networkd* service using:: +#. Restart *systemd-networkd* service using:: # systemctl restart systemd-networkd or restart Clear Linux. #. Check new IP with:: - + # ip addr Setting Time ============ -Clear Linux utilizes **systemd-timesyncd.service** to sync time. +Clear Linux utilizes **systemd-timesyncd.service** to sync time. Default :abbr:`NTP (Network Time Protocol)` servers are configured as *time1.google.com, time2.google.com, time3.google.com, and time4.google.com*. It is not possible to set the time manually, via @@ -76,8 +76,4 @@ time is incorrect on your system, try these steps: .. note:: Check to make sure your time is correctly set: date - - - - .. _systemd network configuration: https://www.freedesktop.org/software/systemd/man/systemd.network.html diff --git a/source/bundles_overview.rst b/source/clear-linux/bundles/overview.rst similarity index 99% rename from source/bundles_overview.rst rename to source/clear-linux/bundles/overview.rst index 67a86d04..ddedc38b 100644 --- a/source/bundles_overview.rst +++ b/source/clear-linux/bundles/overview.rst @@ -1,4 +1,4 @@ -.. _bundles_overview: +.. _overview: Bundles overview ################ diff --git a/source/clear-linux/clear-linux.rst b/source/clear-linux/clear-linux.rst new file mode 100644 index 00000000..82ade7b6 --- /dev/null +++ b/source/clear-linux/clear-linux.rst @@ -0,0 +1,14 @@ +.. _clear-linux: + +Clear Linux\* Project for Intel® Architecture +############################################# + +.. toctree:: + :maxdepth: 2 + + getting-started/getting-started + bundles/bundles + how-to/how-to + network/network + virtual-environments/virtual-environments + tutorials/tutorials \ No newline at end of file diff --git a/source/gs_creating_bootable_usb.rst b/source/clear-linux/getting-started/bootable-usb.rst similarity index 86% rename from source/gs_creating_bootable_usb.rst rename to source/clear-linux/getting-started/bootable-usb.rst index 1936096e..3d870f87 100644 --- a/source/gs_creating_bootable_usb.rst +++ b/source/clear-linux/getting-started/bootable-usb.rst @@ -1,17 +1,17 @@ -.. _gs_creating_bootable_usb: +.. _bootable-usb: Creating a bootable USB to install the OS ########################################## Here's how to create a USB drive that initiates the process for -:ref:`gs_installing_clr_as_host`. Alternatively, you can test the -OS by :ref:`vm-kvm`. +:ref:`clear-host`. Alternatively, you can test the +OS by :ref:`kvm`. What you need ============= -* A USB stick, formatted as ``ext4``. Remember that the process of +* A USB stick, formatted as ``ext4``. Remember that the process of flashing data to a USB completely deletes the contents of the drive; as always, run ``dd`` with caution. * A Clear Linux* OS for Intel® Architecture image; the most current @@ -32,9 +32,9 @@ Download and checksum Confirm the mount point on the USB drive ======================================== -Using ``$ lsblk`` is helpful to show the block-level devices; +Using ``$ lsblk`` is helpful to show the block-level devices; a USB drive usually shows up under ``/sdb`` or ``/sdc`` (almost never -under ``/sda``), and should indicate disk space approximately the +under ``/sda``), and should indicate disk space approximately the size of the USB drive:: $ lsblk /dev/sdb @@ -42,7 +42,7 @@ size of the USB drive:: ?? sdb 8:16 1 14.9G 0 ?? sdb1 8:17 1 14.9G 0 part -Make sure the drive isn't already mounted. The easiest way is with +Make sure the drive isn't already mounted. The easiest way is with ``umount`` or :: @@ -53,8 +53,8 @@ Make sure the drive isn't already mounted. The easiest way is with Flash the image to the USB ========================== -Flash the image with the following command, adding the ``-v`` option -for verbose mode (recommended), as the image file may be large, and the +Flash the image with the following command, adding the ``-v`` option +for verbose mode (recommended), as the image file may be large, and the write can take a while. This may need to be done as root or with sudo:: $ xzcat -v clear-[release_number]-installer.img.xz | dd of=/dev/sdb @@ -63,15 +63,15 @@ write can take a while. This may need to be done as root or with sudo:: Wait for the final confirmation =============================== -This example shows ``clear-9130-installer.img.xz`` flashed to a 16GB +This example shows ``clear-9130-installer.img.xz`` flashed to a 16GB USB drive mounted on ``/sdb``. -.. image:: _static/images/gs_confirmation_screen.png +.. image:: ./figures/confirmation-screen.png :align: center :alt: confirmation Success! Your USB stick is now ready to boot and initiate the process for -:ref:`gs_installing_clr_as_host`. +:ref:`clear-host`. diff --git a/source/gs_installing_clr_as_host.rst b/source/clear-linux/getting-started/clear-host.rst similarity index 52% rename from source/gs_installing_clr_as_host.rst rename to source/clear-linux/getting-started/clear-host.rst index 8bda2983..86c8ca76 100644 --- a/source/gs_installing_clr_as_host.rst +++ b/source/clear-linux/getting-started/clear-host.rst @@ -1,32 +1,32 @@ -.. _gs_installing_clr_as_host: +.. _clear-host: Installing the OS as host ######################### -Running Clear Linux* OS for Intel® Architecture natively on a system is +Running Clear Linux\* OS for Intel® Architecture natively on a system is easy with the installer image. To get started you'll need the following: -* A USB stick flashed with the `installer image`_. For instructions on how to - do this, see :ref:`gs_creating_bootable_usb`. -* A host machine running :ref:`gs_supported_hardware`. +* A USB stick flashed with the `installer image`_. For instructions on how + to do this, see :ref:`bootable-usb`. +* A host machine running :ref:`supported-hardware`. * Network access via DHCP. Installing Clear Linux OS for Intel Architecture to a target system =================================================================== -#. Configure the BIOS; this may involve changing the priority of the boot - device to boot from USB port first. -#. Insert the USB device that is flashed with the - `installer image`_ into the target system, then reboot. -#. The installer user interface will appear. Once all appropriate configuration - options have been chosen, the installer will install the OS. +#. Configure the BIOS; this may involve changing the priority of the boot + device to boot from USB port first. +#. Insert the USB device that is flashed with the + `installer image`_ into the target system, then reboot. +#. The installer user interface will appear. Once all appropriate + configuration options have been chosen, the installer will install the OS. -The entire installation should take no more than a few minutes. +The entire installation should take no more than a few minutes. -For feedback on installation or other topics, please feel free to write in to our -`mailing list`_. +For feedback on installation or other topics, please feel free to write in to +our `mailing list`_. -Note: You can also try :ref:`vm-kvm`. +Note: You can also try :ref:`kvm`. .. _installer image: http://download.clearlinux.org/image .. _mailing list: https://lists.clearlinux.org/mailman/listinfo/dev diff --git a/source/_static/images/gs_confirmation_screen.png b/source/clear-linux/getting-started/figures/confirmation-screen.png similarity index 100% rename from source/_static/images/gs_confirmation_screen.png rename to source/clear-linux/getting-started/figures/confirmation-screen.png diff --git a/source/clear-linux/getting-started/getting-started.rst b/source/clear-linux/getting-started/getting-started.rst new file mode 100644 index 00000000..590ea51d --- /dev/null +++ b/source/clear-linux/getting-started/getting-started.rst @@ -0,0 +1,13 @@ +Getting started +############### + +.. toctree:: + :maxdepth: 2 + + installation-options + supported-hardware + bootable-usb + clear-host + kernels + + diff --git a/source/gs_getting_started.rst b/source/clear-linux/getting-started/installation-options.rst similarity index 54% rename from source/gs_getting_started.rst rename to source/clear-linux/getting-started/installation-options.rst index d600a646..c58adf9b 100644 --- a/source/gs_getting_started.rst +++ b/source/clear-linux/getting-started/installation-options.rst @@ -1,11 +1,12 @@ -.. _gs_getting_started: +.. _installation-options: -Getting started -############### +Installation options +#################### -There are a number of options available for downloading and running Clear -Linux* OS for Intel® Architecture. You can check your system hardware -capabilities against the list of `supported processors`_; alternatively, +There are a number of options available for downloading and running Clear +Linux\* OS for Intel® Architecture. You can check your system hardware +capabilities against the list of +:ref:`supported processors `; alternatively, download and run the `clear-linux-check-config`_ script to check the hardware compatibility of your system from the command line. @@ -20,10 +21,11 @@ a container, download the `clear-linux-check-config`_ script and run:: $ ./clear-linux-check-config.sh container -* **KVM Image** -- Download a recent KVM image, along with the UEFI firmware helper, - ``OVMF.fd`` and the KVM start helper script ``start_qemu.sh.`` Find the - ``clear-[version_number]-kvm.img.xz`` image in the `current`_ directory and see - :ref:`vm-kvm` for further detail. +* **KVM Image** -- Download a recent KVM image, along with the UEFI firmware + helper, ``OVMF.fd`` and the KVM start helper script ``start_qemu.sh.`` Find + the ``clear-[version_number]-kvm.img.xz`` image in the `current`_ directory + and see + :ref:`kvm` for further detail. * **Docker Image** -- A Clear Linux* Docker* image can be downloaded directly from the Docker hub and executed within the Docker environment. @@ -39,38 +41,41 @@ a container, download the `clear-linux-check-config`_ script and run:: Bare metal installer images =========================== -To test whether your system contains the necessary components to run the OS natively, -download the `clear-linux-check-config`_ script and run:: +To test whether your system contains the necessary components to run the OS +natively, download the `clear-linux-check-config`_ script and run:: $ ./clear-linux-check-config.sh host -* **Installer Image** -- The installer image allows you to manipulate and target - partitions, and to set other OS configuration features (hostname, administrative user) - commonly found in Linux installers. Find the ``clear-[version_number]-installer.img.xz`` - in our `current`_ directory and see :ref:`gs_installing_clr_as_host`. This `blog post`_ - contains more detail about the evolution of our installer. +* **Installer Image** -- The installer image allows you to manipulate and + target partitions, and to set other OS configuration features (hostname, + administrative user) commonly found in Linux installers. Find the + ``clear-[version_number]-installer.img.xz`` in our `current`_ directory and + see :ref:`clear-host`. This `blog post`_ contains more detail + about the evolution of our installer. -* **Provisioning Image** -- This image is intended to be used in a provioning environment. - Find the ``clear-[version_number]-provision.img.xz`` image in our `current`_ directory. - *Warning*: This installer repartitions ``/dev/sda`` and installs the OS to the new - partition. Don't use this image on a system where you care about existing data on - ``/dev/sda``. +* **Provisioning Image** -- This image is intended to be used in a provioning + environment. Find the ``clear-[version_number]-provision.img.xz`` image in + our `current`_ directory. *Warning*: This installer repartitions + ``/dev/sda`` and installs the OS to the new partition. Don't use this image + on a system where you care about existing data on ``/dev/sda``. Other installation options ========================== -* **Live Image** -- The live image can be used to boot the OS in a VM, or you can lay - the image down on a USB drive with a tool like ``dd`` and boot from USB. +* **Live Image** -- The live image can be used to boot the OS in a VM, or you + can lay the image down on a USB drive with a tool like ``dd`` and boot from + USB. - This option is a great way to kick the tires with a minimal amount of effort. Be - aware, however, that if you do **not** manually configure the install and instead use - the auto-install, it will repartition ``/dev/sda``. This image also enables telemetry - by default; see the `telemetry`_ feature page for more details. + This option is a great way to kick the tires with a minimal amount of + effort. Be aware, however, that if you do **not** manually configure the + install and instead use the auto-install, it will repartition ``/dev/sda``. + This image also enables telemetry by default; see the `telemetry`_ feature + page for more details. - To select this option, download the ``clear-[version_number]-live.img.xz`` image in - the `current`_ version's download directory. + To select this option, download the ``clear-[version_number]-live.img.xz`` + image in the `current`_ version's download directory. For older versions, see our `releases`_ page. @@ -79,5 +84,4 @@ For older versions, see our `releases`_ page. .. _blog post: https://clearlinux.org/blogs/clear-linux-installer-v20 .. _Clear Linux Highlights 3: https://clearlinux.org/blogs/clear-linux-highlights-3 .. _telemetry: https://clearlinux.org/features/telemetry -.. _supported processors: http://clearlinux.org/documentation/gs_supported_hardware.html .. _releases: https://download.clearlinux.org/releases diff --git a/source/gs_clear-linux-kernels.rst b/source/clear-linux/getting-started/kernels.rst similarity index 81% rename from source/gs_clear-linux-kernels.rst rename to source/clear-linux/getting-started/kernels.rst index 5204c8e8..806da024 100644 --- a/source/gs_clear-linux-kernels.rst +++ b/source/clear-linux/getting-started/kernels.rst @@ -1,4 +1,4 @@ -.. _gs_clear-linux-kernels: +.. _kernels: |CL| kernels ############ @@ -11,7 +11,7 @@ Kernel native The *kernel-native* bundle focuses on the bare metal platforms. It is optimized for fast booting and performs best on the Intel® architectures -described on the :ref:`supported hardware list`. The +described on the :ref:`supported hardware list`. The optimization patches are found in our `Linux`_ GitHub\* repo. Kernel LTS @@ -20,9 +20,9 @@ Kernel LTS The *kernel-lts* bundle focuses on the bare metal platforms but uses the latest :abbr:`LTS (Long Term Support)` Linux kernel. It is optimized for fast booting and performs best on the Intel® architectures described on the -:ref:`supported hardware list`. Additionally, this +:ref:`supported hardware list`. Additionally, this kernel includes the VirtualBox\* kernel modules, see our -:ref:`instructions on using Virtualbox` for more information. +:ref:`instructions on using Virtualbox` for more information. The optimization patches are found in our `Linux-LTS`_ GitHub repo. Kernel KVM @@ -31,12 +31,12 @@ Kernel KVM The *kernel-kvm* bundle focuses on the Linux :abbr:`KVM (Kernel-based Virtual Machine)`. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures -described on the :ref:`supported hardware list`. +described on the :ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS on top of *qemu/kvm*. Use this kernel with **cloud orchestrators** using *qemu/kvm* internally as their **hypervisor**. This kernel can be used as a stand alone Linux VM, see our -:ref:`instructions on using KVM` for more information. The +:ref:`instructions on using KVM` for more information. The optimization patches are found in our `Linux-KVM`_ GitHub repo. Kernel Hyper-V\* @@ -45,10 +45,10 @@ Kernel Hyper-V\* The *kernel-hyperv* bundle focuses on running Linux on Microsoft\* Hyper-V. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the -:ref:`supported hardware list`. +:ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in projects such as Microsoft `Azure`_\*. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +alone Linux VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV`_ GitHub repo. @@ -59,10 +59,10 @@ The *kernel-hyperv-lts* bundle focuses on running Linux on Microsoft Hyper-V but uses the latest :abbr:`LTS (Long Term Support)` Linux kernel. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the -:ref:`supported hardware list`. +:ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in projects such as Microsoft `Azure`_. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +alone Linux VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV-LTS`_ GitHub repo. @@ -72,7 +72,7 @@ Kernel Container The *kernel-container* bundle contains the kernel used by the `Intel® Clear Containers`_ project. This kernel is optimized for fast booting and performs best on |CC| running on the Intel® architectures -described on the :ref:`supported hardware list`. +described on the :ref:`supported hardware list`. The optimization patches are found in our `Linux-Container`_ GitHub repo. .. _Linux: https://github.com/clearlinux-pkgs/linux diff --git a/source/gs_supported_hardware.rst b/source/clear-linux/getting-started/supported-hardware.rst similarity index 97% rename from source/gs_supported_hardware.rst rename to source/clear-linux/getting-started/supported-hardware.rst index 600a1011..455e3fc6 100644 --- a/source/gs_supported_hardware.rst +++ b/source/clear-linux/getting-started/supported-hardware.rst @@ -1,4 +1,4 @@ -.. _gs_supported_hardware: +.. _supported-hardware: Supported hardware ################## diff --git a/source/clear_linux_os_on_azure.rst b/source/clear-linux/how-to/azure.rst similarity index 61% rename from source/clear_linux_os_on_azure.rst rename to source/clear-linux/how-to/azure.rst index a41576c4..6443f3af 100644 --- a/source/clear_linux_os_on_azure.rst +++ b/source/clear-linux/how-to/azure.rst @@ -1,7 +1,7 @@ -.. _clear_linux_os_on_azure +.. _azure: -Clear Linux* OS on Microsoft* Azure* -==================================== +Clear Linux\* OS on Microsoft\* Azure\* +####################################### Clear Linux OS is now an offering in the Azure Marketplace. @@ -10,46 +10,51 @@ and is tuned to maximize the performance and value of Intel® architecture. In Azure our boot times are lightning-quick, with all on-boot services launched in less than a second on nodes with warm caches [1]_. -There are three offerings of Clear Linux OS within the Azure Marketplace. They can -be created through the `Azure Portal `_ or by +There are three offerings of Clear Linux OS within the Azure Marketplace. They +can be created through the `Azure Portal `_ or by using the `Azure Command Line tools `_. Each offering can be further customized by using the swupd command to install -additional bundles. Learn more about Clear Linux OS and bundles -at https://clearlinux.org/documentation. +additional bundles. Learn more about Clear Linux OS and bundles in our +:ref:`documentation`. Offerings -~~~~~~~~~ +========= The three offerings, and the commands to launch VM instances from the command line for each, are: * **Basic** - This is a bare-bones generic offering, from which users may extend functionality by adding bundles of their choosing: -* **Containers** - This offering comes with the containers-basic bundle already installed. +* **Containers** - This offering comes with the containers-basic bundle already + installed. * **Machine Learning** - This offering comes pre-loaded with popular open source tools for developing machine learning applications. Azure Command Line Interface -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +============================ -The Azure Command Line Interface offers the ability to create and manage resources in Azure from the -command line. The examples here are based on Version 1.0 of the Azure CLI, which is implemented in -Javascript and distributed through the Node Package Manager. Version 2.0 of the Azure CLI is -implemented in Python. This example will be updated when Azure CLI 2.0 is fully released. +The Azure Command Line Interface offers the ability to create and manage +resources in Azure from the command line. The examples here are based on +Version 1.0 of the Azure CLI, which is implemented in Javascript and +distributed through the Node Package Manager. Version 2.0 of the Azure CLI is +implemented in Python. This example will be updated when Azure CLI 2.0 is fully +released. -The following command shows the version number of the most recent images that have been uploaded -into the Azure Marketplace. +The following command shows the version number of the most recent images that +have been uploaded into the Azure Marketplace. :: azure vm image list -p clear-linux-project -l westus -o clear-linux-os | grep -v "Getting" | cut -f5 -d: | sed -e 's/\s*//g'| sed -e 's/\..*//' | sort -u | tail -1 -The following scriplet can be used to create and start a virtual machine instance in Azure based on -a marketplace offering. The resource group needs to already exist. This script tries to use a -resource group of the form "-clr-vms". For example, for an Azure user named Rob -trying to create an instance of version ``12920`` of the ``basic`` offering for Clear Linux OS, -this script creates an instance with the public hostname of ``rob-12920-basic-1234``. +The following scriplet can be used to create and start a virtual machine +instance in Azure based on a marketplace offering. The resource group needs to +already exist. This script tries to use a resource group of the form +"-clr-vms". For example, for an Azure user named Rob trying to +create an instance of version ``12920`` of the ``basic`` offering for Clear +Linux OS, this script creates an instance with the public hostname of +``rob-12920-basic-1234``. :: @@ -86,26 +91,28 @@ this script creates an instance with the public hostname of ``rob-12920-basic-12 SSH Sessions -~~~~~~~~~~~~ +============ To keep SSH sessions to Clear Linux Guests in Azure from being dropped after periods of inactivity, you can give the following option to SSH via the command line:: - -o ServerAliveInterval=180 + -o ServerAliveInterval=180 Alternatively, you can add this setting to your SSH config file as shown below:: - Host *: - ServerAliveInterval 180 + Host *: + ServerAliveInterval 180 -.. [1] Software and workloads used in performance tests may have been optimized for -performance only on Intel microprocessors. Performance tests are measured using -specific computer systems, components, software, operations and functions. Any -change to any of those factors may cause the results to vary. You should consult -other information and performance tests to assist you in fully evaluating your -contemplated purchases, including the performance of that product when combined -with other products. For more complete information, visit -http://www.intel.com/performance/datacenter. Configuration: -Clear Linux OS release 11130 on SKU Standard_DS3_v2 in Microsoft* Azure*. +.. [1] + Software and workloads used in performance tests may have been optimized + for performance only on Intel microprocessors. Performance tests are + measured using specific computer systems, components, software, operations + and functions. Any change to any of those factors may cause the results to + vary. You should consult other information and performance tests to assist + you in fully evaluating your contemplated purchases, including the + performance of that product when combined with other products. For more + complete information, visit http://www.intel.com/performance/datacenter. + Configuration: Clear Linux OS release 11130 on SKU Standard_DS3_v2 in + Microsoft\* Azure\*. diff --git a/source/bulk_provisioning.rst b/source/clear-linux/how-to/bulk-provisioning.rst similarity index 96% rename from source/bulk_provisioning.rst rename to source/clear-linux/how-to/bulk-provisioning.rst index cfc76b3f..6f14fec1 100644 --- a/source/bulk_provisioning.rst +++ b/source/clear-linux/how-to/bulk-provisioning.rst @@ -1,4 +1,4 @@ -.. _bulk_provisioning: +.. _bulk-provisioning: Bulk Provisioning ################# @@ -17,7 +17,7 @@ files allow us to customize the instance of the installation. Figure 1 depicts the flow of information between a PXE server and a PXE client that needs to be set up to perform a bulk provision. -.. figure:: _static/images/bulk-provision-flow.png +.. figure:: ./figures/bulk-provision-flow.png :alt: Bulk provision information flow Figure 1: Bulk provision information flow @@ -27,7 +27,7 @@ Prerequisites Before performing a bulk provision, verify you have a PXE server capable of performing network boots of |CL|. Please refer to our -:ref:`guide on how to perform an iPXE boot` using +:ref:`guide on how to perform an iPXE boot` using :abbr:`NAT (network address translation)` for details. Because a bulk provision relies on a reboot, ensure the following @@ -49,7 +49,7 @@ Configuration with the steps it needs to perform an installation. The file outlines what partitions, file systems, and mount points **Ister** should set up. Lastly, the file outlines which bundles to install. See our - :ref:`bundles_overview` for the list of available bundles. The + :ref:`overview` for the list of available bundles. The following example shows the contents of an Ister installation file: .. code-block:: json diff --git a/source/_static/images/bulk-provision-flow.png b/source/clear-linux/how-to/figures/bulk-provision-flow.png similarity index 100% rename from source/_static/images/bulk-provision-flow.png rename to source/clear-linux/how-to/figures/bulk-provision-flow.png diff --git a/source/clear-linux/how-to/how-to.rst b/source/clear-linux/how-to/how-to.rst new file mode 100644 index 00000000..97601d62 --- /dev/null +++ b/source/clear-linux/how-to/how-to.rst @@ -0,0 +1,14 @@ +.. _how-to: + +Clear Linux\* OS How to guides +############################## + +.. toctree:: + :maxdepth: 2 + + azure + swupd/about + swupd/run + bulk-provisioning + mixer + validate-signatures diff --git a/source/clear-linux/how-to/mixer.rst b/source/clear-linux/how-to/mixer.rst new file mode 100644 index 00000000..cae0c0d7 --- /dev/null +++ b/source/clear-linux/how-to/mixer.rst @@ -0,0 +1,280 @@ +.. _mixer: + +Mixer tool +########## + +*Mixing* refers to composing an operating system for specific use cases. While +the default Clear Linux\* OS for Intel® Architecture provides options to +install bundles for various server capabilities, some developers may wish to 1) +augment the operating system itself with functionality from their own packages +or 2) modify the structure of current bundles to cater to their particular +needs. + +Prerequisites +============= + +To start working with the Mixer tools, you'll need a recent image of Clear +Linux OS for Intel Architecture with the following bundle installed. If you +don't have it already, you can add it with the :command:`swupd bundle-add` +command like so:: + + # swupd bundle-add mixer + +Current Workflow +================ + +Mixing +------ + +#. **Create a workspace**. Create an empty directory in your Clear image to + use as a "workspace" for mixing. For these steps, we assume your workspace + location is :file:`/home/clr/mix`. + +#. **Configure builder.conf**. Copy the template conf file + + .. code-block:: console + + # cp /usr/share/defaults/bundle-chroot-builder/builder.conf /home/clr/mix/ + + The file ``builder.conf`` will be read automatically from the current + workspace directory, but the ``-config`` option exists to specify where the + file is if you want to store it elsewhere. To use one in your current + workspace, copy the template to /home/clr/mix. The :file:`.yum-mix.conf` + file will be auto-generated for you, as will the :file:`Swupd_Root.pem`. A + yum configuration is needed for the chroot-builder to know where the RPMs + are hosted, and the certificate file is needed to sign the root Manifest to + provide security for content verification. + + Note there are different sections to the builder.conf. The ``[Builder]`` + section provides the mixer tools with required configuration options, + defining where generated bundles and update metadata should get published. + The ``[swupd]`` section is used by swupd-server to create an update with + specific update parameters. + + Edit the template configuration file according to your needs. For this + example, your ``builder.conf`` should look like this, with both URL + variables set to the domain or IP of the update server:: + + # vim /etc/bundle-chroot-builder/builder.conf: + + [Builder] + SERVER_STATE_DIR=/home/clr/mix/update + BUNDLE_DIR=/home/clr/mix/mix-bundles + YUM_CONF=/home/clr/mix/.yum-mix.conf + CERT=/home/clr/mix/Swupd_Root.pem + VERSIONS_PATH=/home/clr/mix/ + + [swupd] + BUNDLE=os-core-update + CONTENTURL= + VERSIONURL= + FORMAT=1 + + The SERVER_STATE_DIR is where the mix content will be outputted to, and it + is automatically created for you by the mixer. This can be set to any + location, but for this example let's use the workspace directory. The same + applies for BUNDLE_DIR; it will be generated for you in the location + specified in the builder.conf, in this case ``/home/clr/mix/mix-bundles``. + This is where the bundle definitions are stored for your mix, and it's where + the chroot-builder looks to know what bundles must be installed. + + You may change the ``CERT=/path/to/cert`` line, which tells the chroot + builder to insert the certificate specified for the mix in ``/os-core- + update/usr/share/clear/update-ca/``. This is the certificate used by the + software update client to verify the Manifest.MoM signature. For now, it is + `HIGHLY` recommended that you do not modify this line, as the certificate + swupd expects needs a very specific configuration to sign and verify + properly. The certificate will be automatically generated for you, and the + Manifest.MoM will be signed automatically as well, providing security for + the update content you create. + + The CONTENTURL and VERSIONURL may be an IP address, or a domain name, which + hosts the /home/clr/mix/update/www (SERVER_STATE_DIR) directory. Creating a + symlink to the directory in your server webdir is an easy way to host the + content. A client running the mix will look to that URL to figure out if + there is a new version available and the location from which to download the + update content. + + To learn more about the FORMAT option, please refer to the bottom of this + document "Format Version" and https://github.com/clearlinux/swupd-server/wiki/Format-Bumps. + For now leave the FORMAT value alone and do not increment it. + + The mix version and Clear version will come from two state files: + :file:`.mixversion` and :file:`.clearversion`, both of which will be created + for you when you set-up the workspace and added to the VERSIONS_PATH + defined. + +#. **Generate the starting point for your Mix**. In your workspace, run:: + + # sudo mixer init-mix -clearver 13180 -mixver 10 + + *If you wish to just build a mix that includes all Clear bundles with no modifications, run*:: + + # sudo mixer init-mix -all -clearver 13180 -mixver 10 + +#. **Create/locate RPMs for mix.**. (Steps 4-6 are necessary only if you + want to add your own RPMs to the Mix. If you are working only with Clear + bundles, then skip to Step 7.) + + If you are creating RPMs from scratch, you may use ``autospec``, + ``mock``, ``rpmbuild``, etc. to build them. If they are not + built on Clear, make sure your configuration and toolchain builds them correctly for Clear, or there is no guarantee + they will be compatible. + +#. **Import RPMs into workspace**. The way to do this is to create an + ``rpms`` directory in your workspace (for example ``/home/clr/mix/rpms``), + and to copy the RPMs you want into that directory. The mixer script will + look here for RPMs in order to build a local RPM repo for yum to use. + +#. **Create a local RPM repo**. Create an empty directory in your workspace + named ``local`` and add the paths in your builder.conf:: + + RPMDIR=/home/clr/mix/rpms + REPODIR=/home/clr/mix/local + + These variables are automatically read; you simply need to run:: + + # sudo mixer add-rpms + + After the script exits, you should see your RPMs and a repodata directory in + ``/home/clr/mix/local``. If the RPMs are not all in the local directory, check + to make sure that they are indeed valid RPM files and not corrupt. + +#. **Update/Add bundle definitions**. The mixer uses a local clone of the + ``clr-bundles`` repo to define bundles for the mix. + + To define your bundles: + + #. Navigate to the ``mix-bundles/`` directory. + #. Make any needed modifications to the bundle set. + #. Commit the result:: + + $ git add . + $ git commit -s -m 'Update bundles for mix #' + + You can easily copy bundles over from the + ``clr-bundles/clr-bundles-VER/bundles/`` directory in the case that you + want to simply use existing bundle sets. Note that ``mix-bundles`` should + not have any folders inside of it, only bundle definitions. + + Do *not* modify things in the clr-bundles dir, this is simply a mirror for + you to use or refer to the Clear Linux OS bundle definitions. + + Why do this? With Git history, mixes are easy to revert to or refer + to in the future if something were to go wrong with a new mix. If + you're just testing this out, or if you really do not want to mess with Git, + you can ignore committing for now. + + To add your own bundle, create a bundle definition file in ``mix-bundles/`` + and refer to :file:`mix-bundles/os-core-update` for formatting, but be sure + that the name does not conflict with another bundle. Add your package + name(s) in that bundle definition file to tell it what package(s) must be + installed as part of that bundle. + +#. **Build the bundle chroots** To build all of the ``chroots`` + that are based on the bundles you defined, in your workspace run:: + + # sudo mixer build-chroots + + If you have many bundles defined for your mix, this step may take some time. + +#. **Create update**. In the workspace, run:: + + # sudo mixer build-update + + When the build completes, you'll find your mix update content under + ``/home/clr/mix/update/www/VER``. In this example, it will be located in + ``/home/clr/mix/update/www/``, where is the mix + version you defined, or 10 by default. + + All content to make a fully usable mix will be created by this step, but + note that only zero packs are automatically generated. To create optional + delta packs, run the pack-maker as follows:: + + # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + + The pack-maker will generate all delta packs for changed bundles from + PAST_VERSION to MIX_VERSION. If your STATE_DIR is in a different location be + sure to specify where with the -S option. For the first build, no delta + packs can be created because the "update" is from version 0, which impicitly + has no content, thus no deltas can be generated. For subsequent builds, + mixer-pack-maker.sh can be run to generate delta content between them (i.e + 10 to 20). + +#. **Creating an image** To create a bootable image from your update content, + you will need the configuration file for ister to create images:: + + # curl -O https://raw.githubusercontent.com/clearlinux/ister/master/release-image-config.json + + Edit this to include all the bundles you want pre-installed into your + image. For a minimal, base image this would be:: + + "Bundles": ["os-core", "os-core-update", "kernel-native"] + + And lastly, set the "Version:" to say which mix version content the image + should be built from, i.e. 10 for your first build. To build the image, + run:: + + # sudo mixer build-image -format 1 + + The output from this should be an image that is bootable as a VM or + installable to baremetal. + + .. note:: + You need to pass in -format if the format you are + building is different than the format of Clear Linux OS you are currently + building on. Format version can be found via + + .. code-block:: console + + # cat /usr/share/defaults/swupd/format + +Creating your next Mix version +------------------------------ + +#. **Initialize next Mix version info**. To update the versions and prep for + your next mix: + + Update the .mixversion file to the next version number you want to build. + From this point you can iterate through, starting again at step 4 and doing + modifications as needed. For example: + + - Add/Remove/Modify Bundles + - sudo mixer build-chroots + - sudo mixer build-update + - (Optionally) sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + + +#. **Update Bundles (Optional)**. Update ``clr-bundles``. In the workspace, + run:: + + # sudo mixer get-bundles + + This step is optional because it is only needed when you want to update the + upstream clr-bundles in your workspace to a new version, which requires + updating the .clearversion file. + +Format Version +-------------- + +The "format" used in ``builder.conf`` might be more precisely referred to as an +OS "compatibility epoch". Versions of the OS within a given epoch are fully +compatible with themselves and can update to any version in that epoch. Across +the format boundary *something* has changed in the OS, such that updating from +build M in format X, to build N in format Y will not work. Generally this +occurs when the software updater or manifests changed in a way that is no +longer compatible with the previous update scheme. + +A format increment is the way we insure pre- and co-requisite changes flow out +with proper ordering. The update client will only ever update to the latest +release in its respective format version (unless overridden by command line +flags), thus we can guarantee all clients will update to the final version in +their given format, which *must* contain all the changes needed to understand +the content built in the following format. Only after reaching the final +release in the old format will a client be able to continue to update to +releases in the new format. + +For the creation of a custom mix, the format version should start at '1', or +some known number, and increment only when a compatibility breakage is +introduced. Normal updates (updating a software package for example) do not +require a format increment. diff --git a/source/swupdate_about_sw_update.rst b/source/clear-linux/how-to/swupd/about.rst similarity index 96% rename from source/swupdate_about_sw_update.rst rename to source/clear-linux/how-to/swupd/about.rst index ad592005..ee6a7260 100644 --- a/source/swupdate_about_sw_update.rst +++ b/source/clear-linux/how-to/swupd/about.rst @@ -1,3 +1,5 @@ +.. _swupd-about: + About software update ##################### @@ -19,7 +21,7 @@ each package individually or as a set, using tools such as ``yum`` and The Clear way ============= -Clear Linux OS for Intel® Architecture does things a little differently. +Clear Linux OS for Intel® Architecture does things a little differently. The following are a few notable features that distinguish it from standard OSes: bundles, versioning, updating, and update speed. @@ -121,13 +123,12 @@ and core update take merely seconds. While we realize our definition of bundles makes sense to us, data center operators may have special needs and ideas. Therefore, we provide a -`Mixer`_ tool. This tool allows users to customize and add bundles +:ref:`mixer`. This tool allows users to customize and add bundles or even add their own software, while keeping the operating system and its updates as the basis. Using this tool, system administrators can focus on the custom pieces their deployments require while staying on a controlled update stream. -.. [1] The software update technology for Clear Linux* OS for Intel Architecture - was first presented at the Linux Plumbers conference in 2012. -.. _Mixer: https://clearlinux.org/documentation/mixer.html +.. [1] The software update technology for Clear Linux* OS for Intel + Architecture was first presented at the Linux Plumbers conference in 2012. diff --git a/source/swupdate_how_to_run_the_updater.rst b/source/clear-linux/how-to/swupd/run.rst similarity index 88% rename from source/swupdate_how_to_run_the_updater.rst rename to source/clear-linux/how-to/swupd/run.rst index 52f991eb..8fa4bb20 100644 --- a/source/swupdate_how_to_run_the_updater.rst +++ b/source/clear-linux/how-to/swupd/run.rst @@ -1,10 +1,13 @@ +.. _swupd-run: + How to run the updater ###################### -This section describes how to manually trigger the software update code in Clear Linux* OS for Intel® Architecture. +This section describes how to manually trigger the software update code in +Clear Linux\* OS for Intel® Architecture. Prerequisites -~~~~~~~~~~~~~ +============= - System is up and running a Clear Linux OS for Intel Architecture official build, 7420 later. @@ -15,27 +18,32 @@ Prerequisites http://update.clearlinux.org/update/300/ Running the updater -~~~~~~~~~~~~~~~~~~~ +=================== #. Open a Terminal Emulator. -#. Become root and trigger an update with this command:: -.. code-block:: bash - $ swupd update +#. Become root and trigger an update with this command: + + .. code-block:: console + + $ swupd update #. If the updater console output indicates a kernel update occurred, then at your convenience reboot to begin using the new OS version. Updater options -~~~~~~~~~~~~~~~ +=============== Help options: .. code-block:: console + -h, --help Display help options. Application options: + .. code-block:: console + -d, --download Download all content, but do not actually install the update -u, --url=[URL] RFC-3986 encoded url for version string and content file downloads -P, --port=[port #] Port number to connect to at the url for version string and content file downloads diff --git a/source/validate_sigs.rst b/source/clear-linux/how-to/validate-signatures.rst similarity index 99% rename from source/validate_sigs.rst rename to source/clear-linux/how-to/validate-signatures.rst index 87bdc334..eeff86d9 100644 --- a/source/validate_sigs.rst +++ b/source/clear-linux/how-to/validate-signatures.rst @@ -1,4 +1,4 @@ -.. _validate_sigs: +.. _validate-signatures: Validating signatures ##################### @@ -113,4 +113,4 @@ these steps manually when performing a ``swupd update``. #. The output should contain ``Verification successful``. If the output contains ``bad_signature`` anywhere, then the MoM cannot be trusted. Because the MoM contains a list of hashes for bundle manifests, if the MoM - cannot be trusted, then the bundle content cannot be trusted. \ No newline at end of file + cannot be trusted, then the bundle content cannot be trusted. diff --git a/source/ac-dpdk.rst b/source/clear-linux/network/dpdk.rst similarity index 69% rename from source/ac-dpdk.rst rename to source/clear-linux/network/dpdk.rst index 6f300ed5..cdb86748 100644 --- a/source/ac-dpdk.rst +++ b/source/clear-linux/network/dpdk.rst @@ -1,31 +1,28 @@ -.. _ac-dpdk: +.. _dpdk: DPDK #### -Introduction -============ - -DPDK_ (Data Plane Development Kit) is a set of libraries and drivers for fast -packet processing. -This document describes how to run a basic use case for **l3fwd -DPDK example**. The objective is to *send packages between two platforms* using a -traffic generator called :ref:`pktgen `, where the l3fwd example -application will forward those packages. See (:ref:`f1`) +:abbr:`Data Plane Development Kit (DPDK)` is a set of libraries and drivers +for fast packet processing. This document describes how to run a basic use +case for **l3fwd DPDK example**. The objective is to *send packages between +two platforms* using a traffic generator called :ref:`pktgen `, +where the l3fwd example application will forward those packages, see +:ref:`figure 1 `. .. _f1: -.. figure:: _static/images/pktgen_lw3fd.png +.. figure:: ./figures/pktgen_lw3fd.png :align: center :alt: platform A and B - Figure 1: environment for l3fwd DPDK application. + Figure 1: Environment for l3fwd DPDK application. **Requirements:** -* Two platforms using Clear Linux* for Intel® Architecture (recommended release - `13330`_ or higher). +* Two platforms using Clear Linux* for Intel® Architecture (recommended + release `13330`_ or higher). * Both images have the **kernel-native bundle** added. * Install of **network-basic-dev** bundle: @@ -34,8 +31,8 @@ application will forward those packages. See (:ref:`f1`) # swupd bundle-add network-basic-dev * The platforms must have two NICs, at least one each. It's very important to - check network card compatibility with the DPDK project. You can do this on - the `dpdk.org NICS`_ site. + check network card compatibility with the `DPDK project`_. You can do this + on the `dpdk.org NICS`_ site. * Two network cables. Installing dpdk and build l3fwd example (Platform B) @@ -102,8 +99,8 @@ Intel Architecture, you must download it from upstream and build it: Binding NICs to DPDK kernel drivers (Platforms A and B) ======================================================= -The ``l3fwd`` application uses two NICs. DPDK has useful tools for binding NICs to -DPDK modules to run DPDK applications. +The ``l3fwd`` application uses two NICs. DPDK has useful tools for binding +NICs to DPDK modules to run DPDK applications. #. Load the dpdk I/O kernel module @@ -111,9 +108,9 @@ DPDK modules to run DPDK applications. # modprobe vfio-pci -#. Check the status of your NICs; this will show which network cards are not busy. When - another application is using them, the status shows ``Active``, and those NICs cannot be - bound. +#. Check the status of your NICs; this will show which network cards are not + busy. When another application is using them, the status shows ``Active``, + and those NICs cannot be bound. .. code-block:: bash @@ -127,15 +124,15 @@ DPDK modules to run DPDK applications. # dpdk-devbind --bind=vfio-pci 01:00.0 -#. Check that your NICs binded correctly by checking the status; ``drv`` should have ``igb_uio`` - value; at this point, the NICs are using the DPDK modules. +#. Check that your NICs binded correctly by checking the status; ``drv`` should + have ``igb_uio`` value; at this point, the NICs are using the DPDK modules. Setting hugepages (platforms A and B) ===================================== -Clear Linux OS for Intel Architecture supports ``hugepages`` for the large memory pool -allocation used for packet buffers. +Clear Linux OS for Intel Architecture supports ``hugepages`` for the large +memory pool allocation used for packet buffers. #. Set number of hugepages. @@ -151,7 +148,7 @@ allocation used for packet buffers. # echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages #. Make memory available for DPDK. - + .. code-block:: bash # mkdir -p /mnt/huge $ mount -t hugetlbfs nodev /mnt/huge @@ -162,13 +159,13 @@ allocation used for packet buffers. Setting a physical environment (Platforms A and B) ================================================== -To achieve the model proposed in the introduction of this topic, (:ref:`f1`), we need -to connect the first Grantley’s NICs to the second Grantley’s NICs using the network cables -(:ref:`f2`). +To achieve the model proposed in the introduction of this topic, (:ref:`f1`), +we need to connect the first Grantley’s NICs to the second Grantley’s NICs +using the network cables, see :ref:`figure 2`. .. _f2: -.. figure:: _static/images/pyshical_net.png +.. figure:: ./figures/pyshical_net.png Figure 2: Physical network environment. @@ -176,8 +173,9 @@ to connect the first Grantley’s NICs to the second Grantley’s NICs using the Running l3fwd application (Platform B) ====================================== -The ``l3fwd`` application is one of the DPDK examples available when you install the ``dpdk-dev`` -bundle; this application forwards packages from one NIC to another. +The ``l3fwd`` application is one of the DPDK examples available when you +install the ``dpdk-dev`` bundle; this application forwards packages from one +NIC to another. #. Open the l3fwd example directory. @@ -185,36 +183,40 @@ bundle; this application forwards packages from one NIC to another. # cd /usr/share/dpdk/examples/l3fwd -#. **This step is very important.** DPDK needs poll drivers for work; these poll drivers are - shared objects in :file:`/usr/lib64`. DPDK supports some NICs. The full list available at the - `dpdk.org NICS`_ docs. You should know which kernel module the NIC is using and choose a poll - driver according to your NICs. +#. **This step is very important.** DPDK needs poll drivers for work; these + poll drivers are shared objects in :file:`/usr/lib64`. DPDK supports some + NICs. The full list available at the `dpdk.org NICS`_ docs. You should know + which kernel module the NIC is using and choose a poll driver according to + your NICs. -#. At this point the system must have ``hugepages`` requirements. The NICs bound and the - configuration for running ``pktgen`` depends upon network use cases and available system - resources. Use the ``-d`` flag for setting the pull driver. For example, if the NICs are - using ``e1000`` network driver, they are going to use ``e1000`` poll driver - (``librte_pmd_e1000.so``); it should be in :file:`/usr/lib64` in Clear Linux OS for Intel Architecture, and it - should be enough to add the name. For example +#. At this point the system must have ``hugepages`` requirements. The NICs + bound and the configuration for running ``pktgen`` depends upon network use + cases and available system resources. Use the ``-d`` flag for setting the + pull driver. For example, if the NICs are using ``e1000`` network driver, + they are going to use ``e1000`` poll driver (``librte_pmd_e1000.so``); it + should be in :file:`/usr/lib64` in Clear Linux OS for Intel Architecture, + and it should be enough to add the name. For example .. code-block:: bash # ./build/l3fwd -c 0x3 -n 2 -d librte_pmd_e1000.so -- -p 0x3 --config="(0,0,0),(1,0,1)" -#. When the application starts to run, it will show information about the ``l3fwd`` running, so - pay attention when the application is Initializing ports. After port 0 initialization, you'll - see a MAC address and the same for port 1. Save this information for setting configuration - to `Pktgen` project. +#. When the application starts to run, it will show information about the + ``l3fwd`` running, so pay attention when the application is Initializing + ports. After port 0 initialization, you'll see a MAC address and the same + for port 1. Save this information for setting configuration to `Pktgen` + project. Running Pktgen application (Platform A) =========================================== -The `Pktgen` is network traffic generator. It measures the network packaging performance -in a forwarding use case. +The `Pktgen` is network traffic generator. It measures the network packaging +performance in a forwarding use case. -#. At this point the system must have ``hugepages`` requirements and the NICs bound. The - configuration for running ``pktgen`` depends upon the network use case and the available - system resources. The following is a basic configuration. +#. At this point the system must have ``hugepages`` requirements and the NICs + bound. The configuration for running ``pktgen`` depends upon the network use + case and the available system resources. The following is a basic + configuration. .. code-block:: bash @@ -226,8 +228,8 @@ in a forwarding use case. Pktgen> theme enable -#. The ``l3fwd`` application showed a MAC address per-port initialized; this MAC addresses - should have been set in the pktgen environment:: +#. The ``l3fwd`` application showed a MAC address per-port initialized; this + MAC addresses should have been set in the pktgen environment:: > set mac @@ -244,25 +246,28 @@ in a forwarding use case. Pktgen> start 0-1 -#. If you followed these steps correctly, you'll see that ``pktgen`` is sending and receiving - packages. For more information, see the `Pktgen documentation`_. +#. If you followed these steps correctly, you'll see that ``pktgen`` is sending + and receiving packages. For more information, see the `Pktgen + documentation`_. Annex A: Using pass-through for running on virtual machines =========================================================== -This section explains how to set up a virtual environment where virtual machines -control the host's NICs. +This section explains how to set up a virtual environment where virtual +machines control the host's NICs. #. Create a new directory and move to it. -#. Download or create a ``start_qemu.sh`` script for running a kvm virtual machine: +#. Download or create a ``start_qemu.sh`` script for running a kvm virtual + machine: .. code-block:: bash $ curl -O https://download.clearlinux.org/image/start_qemu.sh -#. Download a bare-metal image of Clear Linux OS for Intel Architecture and rename it as ``clear.img``. +#. Download a bare-metal image of Clear Linux OS for Intel Architecture and + rename it as ``clear.img``. #. Look for an entry for device and vendor & device ID: @@ -274,11 +279,13 @@ control the host's NICs. 03:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] - where ``8086:1521`` is ``vendor:device ID`` and ``03:00.0`` is the entry for device. Make - note of this information; it is necessary for unbinding a host's NICs. + where ``8086:1521`` is ``vendor:device ID`` and ``03:00.0`` is the entry for + device. Make note of this information; it is necessary for unbinding a + host's NICs. -#. Unbind NICs from host to do passthrough with virtual machines. Clear Linux OS for Intel Architecture - currently supports this action. You can use the following commands:: +#. Unbind NICs from host to do passthrough with virtual machines. Clear Linux + OS for Intel Architecture currently supports this action. You can use the + following commands:: echo "vendor device_ID" > /sys/bus/pci/drivers/pci-stub/new_id echo "entry for device" > /sys/bus/pci/drivers/igb/unbind @@ -292,9 +299,9 @@ control the host's NICs. $ echo "0000:03:00.0" > /sys/bus/pci/drivers/pci-stub/bind $ echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/remove_id -#. Assign to the KVM virtual machine (guest) the unbound NICs previously noted. Modify the - ``start_qemu.sh`` script in ``qemu-system-x86_64`` arguments, and add the lines with - the host's NICs information:: +#. Assign to the KVM virtual machine (guest) the unbound NICs previously noted. + Modify the ``start_qemu.sh`` script in ``qemu-system-x86_64`` arguments, and + add the lines with the host's NICs information:: -device pci-assign,host="",id=passnic0,addr=03.0 -device pci-assign,host="",id=passnic1,addr=04.0 @@ -306,8 +313,8 @@ control the host's NICs. -device pci-assign,host=03:00.0,id=passnic0,addr=03.0 \ -device pci-assign,host=03:00.3,id=passnic1,addr=04.0 \ -#. If you would like to add more NUMA machines to the virtual machine, you can add the next - line in the Makefile boot target:: +#. If you would like to add more NUMA machines to the virtual machine, you can + add the next line in the Makefile boot target:: -numa node,mem=,cpus= @@ -323,7 +330,7 @@ control the host's NICs. .. _13330: https://download.clearlinux.org/releases/13330/ -.. _DPDK: http://dpdk.org +.. _DPDK project: http://dpdk.org .. _dpdk.org NICS: http://dpdk.org/doc/nics .. _pktgen tar package: http://dpdk.org/browse/apps/pktgen-dpdk/refs .. _DPDK guide: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html diff --git a/source/_static/images/network-boot-flow.png b/source/clear-linux/network/figures/network-boot-flow.png similarity index 100% rename from source/_static/images/network-boot-flow.png rename to source/clear-linux/network/figures/network-boot-flow.png diff --git a/source/_static/images/network-boot-setup.png b/source/clear-linux/network/figures/network-boot-setup.png similarity index 100% rename from source/_static/images/network-boot-setup.png rename to source/clear-linux/network/figures/network-boot-setup.png diff --git a/source/_static/images/pktgen_lw3fd.png b/source/clear-linux/network/figures/pktgen_lw3fd.png similarity index 100% rename from source/_static/images/pktgen_lw3fd.png rename to source/clear-linux/network/figures/pktgen_lw3fd.png diff --git a/source/_static/images/pyshical_net.png b/source/clear-linux/network/figures/pyshical_net.png similarity index 100% rename from source/_static/images/pyshical_net.png rename to source/clear-linux/network/figures/pyshical_net.png diff --git a/source/_static/images/use_case.png b/source/clear-linux/network/figures/use-case.png similarity index 100% rename from source/_static/images/use_case.png rename to source/clear-linux/network/figures/use-case.png diff --git a/source/network_boot.rst b/source/clear-linux/network/network-boot.rst similarity index 98% rename from source/network_boot.rst rename to source/clear-linux/network/network-boot.rst index ec85a99e..03e78e22 100644 --- a/source/network_boot.rst +++ b/source/clear-linux/network/network-boot.rst @@ -1,4 +1,4 @@ -.. _network_boot: +.. _network-boot: Network Booting ############### @@ -16,7 +16,7 @@ network booting computers which lack built-in PXE support. Figure 1 depicts the flow of information between a PXE server and a PXE client that needs to be created for network booting |CL|. -.. figure:: _static/images/network-boot-flow.png +.. figure:: ./figures/network-boot-flow.png :alt: PXE information flow Figure 1: PXE information flow @@ -46,7 +46,7 @@ made: The required computer and network setup is depicted in figure 2. -.. figure:: _static/images/network-boot-setup.png +.. figure:: ./figures/network-boot-setup.png :alt: NAT network topology Figure 2: NAT network topology diff --git a/source/clear-linux/network/network.rst b/source/clear-linux/network/network.rst new file mode 100644 index 00000000..b9a3e8d6 --- /dev/null +++ b/source/clear-linux/network/network.rst @@ -0,0 +1,11 @@ +.. _network: + +Clear Linux\* OS Network guides +############################### + +.. toctree:: + :maxdepth: 2 + + network-boot + dpdk + ovs-dpdk \ No newline at end of file diff --git a/source/ac-ovs-dpdk.rst b/source/clear-linux/network/ovs-dpdk.rst similarity index 84% rename from source/ac-ovs-dpdk.rst rename to source/clear-linux/network/ovs-dpdk.rst index 77fb78e3..e61fc814 100644 --- a/source/ac-ovs-dpdk.rst +++ b/source/clear-linux/network/ovs-dpdk.rst @@ -1,16 +1,13 @@ -.. _ac-ovs-dpdk: +.. _ovs-dpdk: OpenvSwitch and DPDK #################### -Introduction -============ - Enabling `DPDK`_ support on the `OpenvSwitch`_ project can yield significant -network performance improvements. To illustrate one such example, we'll -cover a simple use case with :ref:`f1ovs`, where one virtual machine sends -1,000,000 HTTP requests to another virtual machine. -The connection between both VMs will be through a virtual bridge created by +network performance improvements. To illustrate one such example, we'll cover +the simple use case shown on :ref:`figure 1`, where one virtual +machine sends 1,000,000 HTTP requests to another virtual machine. The +connection between both VMs will be through a virtual bridge created by different tools: - Linux bridges. @@ -19,7 +16,7 @@ different tools: .. _f1ovs: -.. figure:: _static/images/use_case.png +.. figure:: ./figures/use-case.png Figure 1: Basic virtual network environment. @@ -47,7 +44,8 @@ Requirements Using Linux Bridges =================== -#. Create a script named :file:`qemu-ifup` for a Linux bridge in a virtual machine. +#. Create a script named :file:`qemu-ifup` for a Linux bridge in a virtual + machine. .. code-block:: bash @@ -68,23 +66,25 @@ Using Linux Bridges echo "Error: no interface specified" exit 1 fi - -#. Enable execute permission on the script, preserving its group permissions with - respect to other files. + +#. Enable execute permission on the script, preserving its group permissions + with respect to other files. .. code-block:: bash # chmod a+x qemu-ifup -#. Create a bridge using the :command:`brctl` tool; you can verify whether the bridge - was successfully created by using ip tool. +#. Create a bridge using the :command:`brctl` tool; you can verify whether + the bridge was successfully created by using ip tool. .. code-block:: bash # brctl addbr br0 - .. note:: At this point, it is possible to add a NIC with - ``brctl addif br0 ``, example: + .. note:: + + At this point, it is possible to add a NIC with + ``brctl addif br0 ``, example: .. code-block:: bash @@ -96,8 +96,8 @@ Using Linux Bridges # ip link set dev br0 up -#. Run guest virtual machine A using the next configuration as reference, where - the **$IMAGE** var is the Clear Linux image name. +#. Run guest virtual machine A using the next configuration as reference, + where the **$IMAGE** var is the Clear Linux image name. .. code-block:: bash @@ -110,13 +110,13 @@ Using Linux Bridges -net nic,macaddr=00:11:22:33:44:55,model=virtio -net tap,script=qemu-ifup \ -debugcon file:debug.log -global isa-debugcon.iobase=0x402 -#. Run guest virtual machine B using the configuration from the previous step; - take care to update the MAC address. +#. Run guest virtual machine B using the configuration from the previous + step; take care to update the MAC address. #. Follow the instructions from the `Setting IP Address`_ section. -#. Alternatively, clean the previous environment, turn off the virtual machines, - and delete the bridge. +#. Alternatively, clean the previous environment, turn off the virtual + machines, and delete the bridge. .. code-block:: bash @@ -132,15 +132,16 @@ Using OpenvSwitch # systemctl start openvswitch.service -#. Create a bridge using the OpenvSwitch tool; you can verify whether or not the - bridge was successfully created by running ip tool. +#. Create a bridge using the OpenvSwitch tool; you can verify whether or not + the bridge was successfully created by running ip tool. .. code-block:: bash # ovs-vsctl add-br br0 # ip a -#. Create an ``UP`` script named :file:`ovs-ifup` which can bring up the tap devices. +#. Create an ``UP`` script named :file:`ovs-ifup` which can bring up the tap + devices. .. code-block:: bash @@ -150,7 +151,8 @@ Using OpenvSwitch /usr/bin/ifconfig $1 0.0.0.0 up ovs-vsctl add-port ${switch} $1 -#. Create a ``DOWN`` script named :file:`ovs-ifdown` which can bring up the tap devices. +#. Create a ``DOWN`` script named :file:`ovs-ifdown` which can bring up the + tap devices. .. code-block:: bash @@ -161,17 +163,18 @@ Using OpenvSwitch ovs-vsctl del-port ${switch} $1 -#. Enable execute permission on the scripts, preserving their group permissions - with respect to other files. +#. Enable execute permission on the scripts, preserving their group + permissions with respect to other files. .. code-block:: bash # chmod a+x ovs-ifdown # chmod a+x ovs-ifup -#. Run guest virtual machine A using the next configuration as reference, where - **$IMAGE** var is the name of the Clear Linux* OS for Intel Architecture image. - Notice the network configuration uses the up-down scripts. +#. Run guest virtual machine A using the next configuration as reference, + where **$IMAGE** var is the name of the Clear Linux\* OS for Intel + Architecture image. Notice the network configuration uses the up-down + scripts. .. code-block:: bash @@ -189,8 +192,8 @@ Using OpenvSwitch #. Follow the instructions in the `Setting IP address`_ section. -#. Alternatively, clean the previous environment, turn off the virtual machines, - and delete the bridge. +#. Alternatively, clean the previous environment, turn off the virtual + machines, and delete the bridge. .. code-block:: bash @@ -202,10 +205,11 @@ Using Linux OpenvSwitch-DPDK #. Enable VT-d technology in the BIOS. -#. Enable VT-d in the host kernel command line, to enable VT-d in the host kernel - command line, the ``clear-linux-native-.conf`` - file must be edited. Add ``iommu=pt intel_iommu=on`` to the end of the line. - The file is found within the UEFI boot partition. +#. Enable VT-d in the host kernel command line, to enable VT-d in the host + kernel command line, the + :file:`clear-linux-native-{current-kernel-version}.conf` + file must be edited. Add ``iommu=pt intel_iommu=on`` to + the end of the line. The file is found within the UEFI boot partition. .. code-block:: bash @@ -213,13 +217,13 @@ Using Linux OpenvSwitch-DPDK # cd /boot/loader/entries/ #. Unmount the UEFI partition and reboot the machine. - + .. code-block:: bash # cd / # systemctl stop boot.mount # reboot - + #. Set number of hugepages. .. code-block:: bash @@ -261,27 +265,29 @@ Using Linux OpenvSwitch-DPDK # ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=2048 # ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true -#. Restart the OpenvSwitch service in order to update the new DPDK configuration. +#. Restart the OpenvSwitch service in order to update the new DPDK + configuration. .. code-block:: bash # systemctl restart openvswitch #. Create a virtual bridge using openvswitch. - + .. code-block:: bash # ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev #. Add the vhost-dpdk ports to the bridge. - + .. code-block:: bash # ovs-vsctl add-port br0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser # ovs-vsctl add-port br0 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser -#. Run guest virtual machine A using the next configuration as reference, where - **$IMAGE** var is the name of the Clear Linux* OS for Intel Architecture image. +#. Run guest virtual machine A using the next configuration as reference, + where **$IMAGE** var is the name of the Clear Linux\* OS for Intel + Architecture image. .. code-block:: bash @@ -298,9 +304,9 @@ Using Linux OpenvSwitch-DPDK -numa node,memdev=mem -mem-prealloc \ -debugcon file:debug.log -global isa-debugcon.iobase=0x402 -#. Run guest virtual machine B, use the configuration from the previous step; - simply change the MAC address and the port socket. You can use 00:00:00:00:00:02 as a - MAC address and vhost-user2 as a socket. +#. Run guest virtual machine B, use the configuration from the previous step; + simply change the MAC address and the port socket. You can use + 00:00:00:00:00:02 as a MAC address and vhost-user2 as a socket. #. Follow the instructions from the `Setting IP address`_ section. @@ -312,7 +318,7 @@ Setting IP address ================== #. Set an IP address to virtual machine for virtual machine A: - + .. code-block:: bash # ip addr add dev enp0s2 10.0.0.5/24 @@ -323,14 +329,15 @@ Setting IP address # ip addr add dev enp0s2 10.0.0.6/24 -#. Check if there is communication between both virtual machines using ping tool. +#. Check if there is communication between both virtual machines using ping + tool. #. Verify that Apache service is running: .. code-block:: bash # systemctl status httpd.service - # systemctl start httpd.service + # systemctl start httpd.service Start httpd service only if it is inactive. Use apache benchmark to get information about the network performance between both virtual machines. diff --git a/source/tutorials/bare-metal-install/bare-metal-install.rst b/source/clear-linux/tutorials/bare-metal-install/bare-metal-install.rst similarity index 99% rename from source/tutorials/bare-metal-install/bare-metal-install.rst rename to source/clear-linux/tutorials/bare-metal-install/bare-metal-install.rst index fa003e6e..2252b55e 100644 --- a/source/tutorials/bare-metal-install/bare-metal-install.rst +++ b/source/clear-linux/tutorials/bare-metal-install/bare-metal-install.rst @@ -33,7 +33,7 @@ follow these instructions to find out. * Install and run a Linux distribution directly on your system. * Run a live image from a USB stick. -.. _bootable-usb: +.. _create-usb: Create a Clear Linux bootable USB drive ======================================= diff --git a/source/tutorials/bare-metal-install/compatibility-check.rst b/source/clear-linux/tutorials/bare-metal-install/compatibility-check.rst similarity index 100% rename from source/tutorials/bare-metal-install/compatibility-check.rst rename to source/clear-linux/tutorials/bare-metal-install/compatibility-check.rst diff --git a/source/tutorials/flatpak/figures/flatpak1.png b/source/clear-linux/tutorials/flatpak/figures/flatpak1.png similarity index 100% rename from source/tutorials/flatpak/figures/flatpak1.png rename to source/clear-linux/tutorials/flatpak/figures/flatpak1.png diff --git a/source/tutorials/flatpak/figures/flatpak2.png b/source/clear-linux/tutorials/flatpak/figures/flatpak2.png similarity index 100% rename from source/tutorials/flatpak/figures/flatpak2.png rename to source/clear-linux/tutorials/flatpak/figures/flatpak2.png diff --git a/source/tutorials/flatpak/figures/flatpak3.png b/source/clear-linux/tutorials/flatpak/figures/flatpak3.png similarity index 100% rename from source/tutorials/flatpak/figures/flatpak3.png rename to source/clear-linux/tutorials/flatpak/figures/flatpak3.png diff --git a/source/tutorials/flatpak/flatpak.rst b/source/clear-linux/tutorials/flatpak/flatpak.rst similarity index 100% rename from source/tutorials/flatpak/flatpak.rst rename to source/clear-linux/tutorials/flatpak/flatpak.rst diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst new file mode 100644 index 00000000..9ec67f41 --- /dev/null +++ b/source/clear-linux/tutorials/tutorials.rst @@ -0,0 +1,10 @@ +Tutorials +######### + +.. toctree:: + :maxdepth: 2 + + wordpress.rst + flatpak/flatpak.rst + + diff --git a/source/tutorials/web-server-install/figures/web-server-install-1.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-1.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-1.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-1.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-2.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-2.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-2.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-2.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-3.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-3.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-3.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-3.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-4.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-4.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-4.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-4.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-5.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-5.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-5.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-5.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-6.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-6.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-6.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-6.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-7.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-7.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-7.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-7.png diff --git a/source/tutorials/web-server-install/figures/web-server-install-8.png b/source/clear-linux/tutorials/web-server-install/figures/web-server-install-8.png similarity index 100% rename from source/tutorials/web-server-install/figures/web-server-install-8.png rename to source/clear-linux/tutorials/web-server-install/figures/web-server-install-8.png diff --git a/source/tutorials/web-server-install/web-server-install.rst b/source/clear-linux/tutorials/web-server-install/web-server-install.rst similarity index 100% rename from source/tutorials/web-server-install/web-server-install.rst rename to source/clear-linux/tutorials/web-server-install/web-server-install.rst diff --git a/source/tutorials/wordpress.rst b/source/clear-linux/tutorials/wordpress.rst similarity index 100% rename from source/tutorials/wordpress.rst rename to source/clear-linux/tutorials/wordpress.rst diff --git a/source/tutorials/wp-install/figures/wp-install-1.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-1.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-1.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-1.png diff --git a/source/tutorials/wp-install/figures/wp-install-2.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-2.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-2.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-2.png diff --git a/source/tutorials/wp-install/figures/wp-install-3.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-3.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-3.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-3.png diff --git a/source/tutorials/wp-install/figures/wp-install-4.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-4.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-4.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-4.png diff --git a/source/tutorials/wp-install/figures/wp-install-5.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-5.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-5.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-5.png diff --git a/source/tutorials/wp-install/figures/wp-install-6.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-6.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-6.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-6.png diff --git a/source/tutorials/wp-install/figures/wp-install-7.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-7.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-7.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-7.png diff --git a/source/tutorials/wp-install/figures/wp-install-8.png b/source/clear-linux/tutorials/wp-install/figures/wp-install-8.png similarity index 100% rename from source/tutorials/wp-install/figures/wp-install-8.png rename to source/clear-linux/tutorials/wp-install/figures/wp-install-8.png diff --git a/source/tutorials/wp-install/wp-install.rst b/source/clear-linux/tutorials/wp-install/wp-install.rst similarity index 100% rename from source/tutorials/wp-install/wp-install.rst rename to source/clear-linux/tutorials/wp-install/wp-install.rst diff --git a/source/_static/images/7zipwin.png b/source/clear-linux/virtual-environments/figures/7zipwin.png similarity index 100% rename from source/_static/images/7zipwin.png rename to source/clear-linux/virtual-environments/figures/7zipwin.png diff --git a/source/_static/images/vbox-cd.png b/source/clear-linux/virtual-environments/figures/vbox-cd.png similarity index 100% rename from source/_static/images/vbox-cd.png rename to source/clear-linux/virtual-environments/figures/vbox-cd.png diff --git a/source/_static/images/vbox-convert-image.png b/source/clear-linux/virtual-environments/figures/vbox-convert-image.png similarity index 100% rename from source/_static/images/vbox-convert-image.png rename to source/clear-linux/virtual-environments/figures/vbox-convert-image.png diff --git a/source/_static/images/vbox-create-vm.png b/source/clear-linux/virtual-environments/figures/vbox-create-vm.png similarity index 100% rename from source/_static/images/vbox-create-vm.png rename to source/clear-linux/virtual-environments/figures/vbox-create-vm.png diff --git a/source/_static/images/vbox-efi.png b/source/clear-linux/virtual-environments/figures/vbox-efi.png similarity index 100% rename from source/_static/images/vbox-efi.png rename to source/clear-linux/virtual-environments/figures/vbox-efi.png diff --git a/source/_static/images/vbox-hdisk.png b/source/clear-linux/virtual-environments/figures/vbox-hdisk.png similarity index 100% rename from source/_static/images/vbox-hdisk.png rename to source/clear-linux/virtual-environments/figures/vbox-hdisk.png diff --git a/source/_static/images/vbox-memory-size.png b/source/clear-linux/virtual-environments/figures/vbox-memory-size.png similarity index 100% rename from source/_static/images/vbox-memory-size.png rename to source/clear-linux/virtual-environments/figures/vbox-memory-size.png diff --git a/source/_static/images/vbox-no-vtx.png b/source/clear-linux/virtual-environments/figures/vbox-no-vtx.png similarity index 100% rename from source/_static/images/vbox-no-vtx.png rename to source/clear-linux/virtual-environments/figures/vbox-no-vtx.png diff --git a/source/_static/images/vbox-x.png b/source/clear-linux/virtual-environments/figures/vbox-x.png similarity index 100% rename from source/_static/images/vbox-x.png rename to source/clear-linux/virtual-environments/figures/vbox-x.png diff --git a/source/vm-hyper-v.rst b/source/clear-linux/virtual-environments/hyper-v.rst similarity index 93% rename from source/vm-hyper-v.rst rename to source/clear-linux/virtual-environments/hyper-v.rst index 6c1bf7bf..066b75ed 100644 --- a/source/vm-hyper-v.rst +++ b/source/clear-linux/virtual-environments/hyper-v.rst @@ -1,4 +1,4 @@ -.. _vm-hyper-v: +.. _hyper-v: Using Hyper-V\* ############### @@ -7,9 +7,9 @@ This section explains how to run |CLOSIA| inside a `Windows Server Virtualization`_\* or **Hyper-V** environment. Please ensure you have enabled `Intel® Virtualization Technology -`_ +`_ (Intel® VT) and `Intel® Virtualization Technology for Directed I/O -`_ +`_ (Intel® VT-d) in your BIOS/UEFI firmware configuration. Install Hyper-V diff --git a/source/vm-kvm.rst b/source/clear-linux/virtual-environments/kvm.rst similarity index 80% rename from source/vm-kvm.rst rename to source/clear-linux/virtual-environments/kvm.rst index 50929dae..2384609a 100644 --- a/source/vm-kvm.rst +++ b/source/clear-linux/virtual-environments/kvm.rst @@ -1,4 +1,4 @@ -.. _vm-kvm: +.. _kvm: Using KVM ######### @@ -8,9 +8,9 @@ a recent KVM image from the `images`_ directory. Here you'll find a kvm image file, the UEFI firmware helper, and the KVM start helper script. Please ensure you have enabled `Intel® Virtualization Technology -`_ +`_ (Intel® VT) and `Intel® Virtualization Technology for Directed I/O -`_ +`_ (Intel® VT-d) in your BIOS/UEFI firmware configuration. Starter script @@ -19,10 +19,10 @@ Starter script To start the image, run the `qemu shell script`_ available in the `images`_ directory. -SSH is needed for remote logins; however, SSH not enabled by default. To enable -it, log in through serial console with the username ``root``. After setting the -password, enable root login via SSH by configuring :file:`/etc/ssh/sshd_config` -with this line:: +SSH is needed for remote logins; however, SSH not enabled by default. To +enable it, log in through serial console with the username ``root``. After +setting the password, enable root login via SSH by configuring +:file:`/etc/ssh/sshd_config` with this line:: PermitRootLogin yes @@ -49,8 +49,8 @@ Alternatively, there are a few other ways to approach this. $ ./start_qemu.sh clr_image & -If you run into any trouble with qemu getting locked up, try editing the `qemu shell script`_ -and removing the ``aio=threads`` +If you run into any trouble with qemu getting locked up, try editing the +`qemu shell script`_ and removing the ``aio=threads`` .. _qemu shell script: http://download.clearlinux.org/image/start_qemu.sh diff --git a/source/clear-linux/virtual-environments/virtual-environments.rst b/source/clear-linux/virtual-environments/virtual-environments.rst new file mode 100644 index 00000000..216b5994 --- /dev/null +++ b/source/clear-linux/virtual-environments/virtual-environments.rst @@ -0,0 +1,13 @@ +.. _virtual-environments: + +Running in virtual environments +############################### + +.. toctree:: + :maxdepth: 2 + + kvm + virtualbox + vmware-esxi + vmware-player + hyper-v \ No newline at end of file diff --git a/source/vm-virtualbox.rst b/source/clear-linux/virtual-environments/virtualbox.rst similarity index 82% rename from source/vm-virtualbox.rst rename to source/clear-linux/virtual-environments/virtualbox.rst index 9de48a0e..abf6d9c8 100644 --- a/source/vm-virtualbox.rst +++ b/source/clear-linux/virtual-environments/virtualbox.rst @@ -1,23 +1,21 @@ -.. _vm-virtualbox: +.. _virtualbox: -Using VirtualBox* -################# +Using VirtualBox\* +################## -This section explains how to run |CLOSIA| -inside a `VirtualBox`_\* environment. +This section explains how to run |CLOSIA| inside a `VirtualBox environment`_. Please ensure you have enabled `Intel® Virtualization Technology -`_ +`_ (Intel® VT) and `Intel® Virtualization Technology for Directed I/O -`_ +`_ (Intel® VT-d) in your BIOS/UEFI firmware configuration. Download VirtualBox =================== -VirtualBox is a hypervisor supported by Oracle. You can -download it from the `official VirtualBox website`_ and select -the operating system you are using. +VirtualBox is a hypervisor supported by Oracle. You can download it from the +`official VirtualBox website`_ and select the operating system you are using. Download **version 5.0 or greater** to ensure support for the :abbr:`AVX (Advanced Vector Extensions)` needed to run @@ -42,7 +40,7 @@ Create a virtual machine in VirtualBox - Right-click the file to *extract in the same directory*. - .. image:: _static/images/7zipwin.png + .. image:: ./figures/7zipwin.png :alt: 7zip extract here command #. To convert a raw image to :abbr:`VDI (VirtualBox Disk Image)` @@ -58,12 +56,13 @@ Create a virtual machine in VirtualBox .. note:: Be sure you have VirtualBox directory in your PATH (i.e., on Windows :file:`C:\\Program Files\\Oracle\\VirtualBox`). - + On windows: launch a **Command Prompt** program and type :: + + On windows: launch a **Command Prompt** program and type + .. code-block:: console - set PATH=%PATH%;"C:\Program Files\Oracle\VirtualBox" + set PATH=%PATH%;"C:\Program Files\Oracle\VirtualBox" - .. image:: _static/images/vbox-convert-image.png + .. image:: ./figures/vbox-convert-image.png :alt: Convert image in Windows command propt #. Create a virtual machine using the VirtualBox assistant: @@ -71,26 +70,24 @@ Create a virtual machine in VirtualBox a. Type: **Linux** b. Version: **Linux 2.6 / 3.x / 4.x (64-bit)** - .. image:: _static/images/vbox-create-vm.png + .. image:: ./figures/vbox-create-vm.png :alt: Create a new image in VirtualBox - | c. Select default memory size. - .. image:: _static/images/vbox-memory-size.png + .. image:: ./figures/vbox-memory-size.png - | d. Attach the virtual disk created in step number 3 as a virtual hard disk file. Click the folder icon (lower right) to browse to find the VDI file. - .. image:: _static/images/vbox-hdisk.png + .. image:: ./figures/vbox-hdisk.png #. After it is created, go to settings to enable **EFI support** * System -> Enable EFI (special OSes only) - .. image:: _static/images/vbox-efi.png + .. image:: ./figures/vbox-efi.png :alt: Enable EFI on VirtualBox @@ -100,7 +97,7 @@ Run your new VM |CLOSIA| supports VirtualBox kernel modules used by the Linux kernel 4.9 :abbr:`LTS (Long Term Support)` (*kernel-lts bundle*). This kernel was selected because |CL| OS's main kernel -(``kernel-native``) bundle keeps up-to-date with the upstream Linux kernel, +(``kernel-native``) bundle keeps up-to-date with the upstream Linux kernel, and sometimes VirtualBox kernel modules aren't compatible with pre-kernel releases. @@ -140,12 +137,12 @@ Install Guest Additions ----------------------- The kernel modules are shipped with the ``kernel-lts`` bundle; however, -you'll need to install the *user* Linux Guest Additions. To install the +you'll need to install the *user* Linux Guest Additions. To install the VirtualBox Guest Additions, follow these steps: #. Insert Guest Additions CD image using *Devices* menu - .. image:: _static/images/vbox-cd.png + .. image:: ./figures/vbox-cd.png :alt: VirtualBox CD #. Install Linux users Guest Additions:: @@ -161,14 +158,14 @@ VirtualBox Guest Additions, follow these steps: # swupd bundle-add os-utils-gui - once the ``os-utils-gui`` bundle is installed, start your graphical + once the ``os-utils-gui`` bundle is installed, start your graphical user interface with:: # startx |CL| doesn't provide a graphical display manager. - .. image:: _static/images/vbox-x.png + .. image:: ./figures/vbox-x.png :alt: XFCE |CL| on Virtual Box @@ -178,10 +175,10 @@ Troubleshooting On Windows OS, *VirtualBox* cannot do a **Hardware Virtualization** when *Hyper-V* is enabled. -.. image:: _static/images/vbox-no-vtx.png +.. image:: ./figures/vbox-no-vtx.png :alt: VirtualBox hardware acceleration error -| + To disable *Hyper-V* you should execute:: bcdedit /set {current} hypervisorlaunchtype off @@ -194,6 +191,6 @@ To enable Hyper-V again, you should execute:: .. _official VirtualBox website: https://www.virtualbox.org/wiki/Downloads -.. _VirtualBox: https://www.virtualbox.org/ +.. _VirtualBox environment: https://www.virtualbox.org/ .. _latest: https://download.clearlinux.org/image/ .. _7zip: http://www.7-zip.org/ diff --git a/source/vm-vmware-esxi.rst b/source/clear-linux/virtual-environments/vmware-esxi.rst similarity index 97% rename from source/vm-vmware-esxi.rst rename to source/clear-linux/virtual-environments/vmware-esxi.rst index b7c5a6ca..df4681ef 100644 --- a/source/vm-vmware-esxi.rst +++ b/source/clear-linux/virtual-environments/vmware-esxi.rst @@ -1,4 +1,4 @@ -.. _vm-vmware-esxi: +.. _vmware-esxi: Using VMware\* ESXi guest ########################## @@ -30,7 +30,7 @@ Run Clear Linux OS for Intel Architecture - Right-click the file and select :menuselection:`Extract in the same directory`. - .. image:: _static/images/7zipwin.png + .. image:: ./figures/7zipwin.png :alt: 7zip extract here command #. Create a virtual machine with the following configuration: diff --git a/source/vm-vmware-player.rst b/source/clear-linux/virtual-environments/vmware-player.rst similarity index 76% rename from source/vm-vmware-player.rst rename to source/clear-linux/virtual-environments/vmware-player.rst index 7e9a866d..983dbe5c 100644 --- a/source/vm-vmware-player.rst +++ b/source/clear-linux/virtual-environments/vmware-player.rst @@ -1,33 +1,33 @@ -.. _vm-vmware-player: +.. _vmware-player: -Using VMware* Player -#################### +Using VMware\* Player +##################### -This section explains how to run Clear Linux OS for Intel® Architecture +This section explains how to run Clear Linux OS for Intel® Architecture within a `VMware Player`_ environment. Please ensure you have enabled `Intel® Virtualization Technology -`_ +`_ (Intel® VT) and `Intel® Virtualization Technology for Directed I/O -`_ +`_ (Intel® VT-d) in your BIOS/UEFI firmware configuration. Install VMware Player ===================== VMware Workstation Player, formerly VMware Player, is a virtualization -software package for x64 computers running Microsoft Windows or Linux. Download -VMware player from the `VMware website`_. +software package for x64 computers running Microsoft Windows or Linux. +Download VMware player from the `VMware website`_. Player on Linux --------------- -For the Linux option, you should have a ``VMware-Player-[version]_FILE.bundle`` file. To -install it, run: +For the Linux option, you should have a :file:`VMware-Player-{version}_FILE.bundle` +file. To install it, run: :: - $ sudo bash ./VMware-Player-[version].x86_64.bundle + $ sudo bash ./VMware-Player-{version}.x86_64.bundle Player on Windows ----------------- @@ -50,7 +50,7 @@ Prepare Image - Right-click the file to *extract in the same directory*. - .. image:: _static/images/7zipwin.png + .. image:: ./figures/7zipwin.png :alt: 7zip extract here command #. Convert the installer to :abbr:`VMDK (Virtual Machine Disk)` format. @@ -65,8 +65,8 @@ Prepare Image :ref:`how to create a VM on VirtualBox ` as example. -Run using VMware* Player -======================== +Run using VMware Player +======================= Create a new virtual machine by following the next steps. @@ -75,19 +75,22 @@ Create a new virtual machine by following the next steps. #. On main window click on “Create a new Virtual Machine”. - * Select “**I will install the operating system later**”, and click on “Next”. - * Select “**Linux**” as “Guest operating system” and version **Other Linux 3.x kernel 64-bit**. + * Select “**I will install the operating system later**”, and click on + “Next”. + * Select “**Linux**” as “Guest operating system” and version **Other Linux + 3.x kernel 64-bit**. * Type a name for the new virtual machine. * Perform the *remaining steps* using the default options. -#. Change boot type to EFI. You must change the VMware virtual machine *configuration* - to **Support EFI firmware**; you can do this by editing the configuration ``.vmx`` - file located in the virtual machine folder and adding the following line:: +#. Change boot type to EFI. You must change the VMware virtual machine + *configuration* to **Support EFI firmware**; you can do this by editing + the configuration ``.vmx`` file located in the virtual machine folder and + adding the following line:: firmware = "efi" -#. Attach the prepared image as SATA disk. And when you have a new virtual machine, - edit its configuration as follows: +#. Attach the prepared image as SATA disk. And when you have a new virtual + machine, edit its configuration as follows: * Click on “Edit virtual machine settings”. * Remove any default attached hard disk. @@ -96,8 +99,8 @@ Create a new virtual machine by following the next steps. * Choose **SATA** as the virtual disk type. * Use the existing Clear Linux OS for Intel Architecture virtual disk - The live disk must be set as ``SATA 0:1 Hard Disk (SATA)``; you can verify - this under the “Advanced" section of the disk settings. + The live disk must be set as ``SATA 0:1 Hard Disk (SATA)``; you can + verify this under the “Advanced" section of the disk settings. Start the virtual machine ========================= diff --git a/source/configure_openstack_environment.rst b/source/configure_openstack_environment.rst deleted file mode 100644 index b1859dac..00000000 --- a/source/configure_openstack_environment.rst +++ /dev/null @@ -1,43 +0,0 @@ -.. _configure_openstack_environment: - -Configure environment -##################### - -Before continuing with the installation process, it's important to understand: - -* The :file:`hosts` file; this file defines the roles your machines play in the - overall scheme of your network mapping. - **Note**: For more in-depth information, see :ref:`openstack_env_inventory_file` - -* The ``group_vars/all`` structuring can be used to override variable - values. **Note**: For more in-depth information, see :ref:`openstack_conf_vars_about` - -Once you understand these components and are comfortable modifying them to affect -the network configuration, you can proceed as follows: - -#. Copy ``/usr/share/ansible/examples/openstack`` to your working directory. - -#. Move into the copied directory. - -#. Edit the :file:`hosts` file to specify node roles. - -#. Edit the contents under ``group_vars/all`` to set passwords and other needed - variables. **Note**: Default variables exist for every role in the - :file:`roles//defaults/main.yml` file. To override a default, define - it with specifications under ``group_vars/all``. - - -Run the installer -================= - -Finally, you can run the installer. - -#. Run the installer as follows, replacing ```` with the key you've - previously set up:: - - $ ansible-playbook -i hosts openstack_deployment.yml --private-key= - -After running the previous command you should see the output of the tasks -that are running. - -At the end of the execution Ansible* will display a summary of the results. diff --git a/source/image-blob-ciao-networking.pptx b/source/image-blob-ciao-networking.pptx deleted file mode 100644 index 397da79f..00000000 Binary files a/source/image-blob-ciao-networking.pptx and /dev/null differ diff --git a/source/index-openstack.rst b/source/index-openstack.rst deleted file mode 100644 index 12856ae4..00000000 --- a/source/index-openstack.rst +++ /dev/null @@ -1,13 +0,0 @@ -OpenStack* implementation -######################### - -.. toctree:: - :maxdepth: 1 - - openstack_use_cases - openstack_bundle_and_service_summary - installing_openstack - configure_openstack_environment - openstack_env_inventory_file - openstack_conf_vars_about - openstack_conf_vars_list diff --git a/source/index-storage.rst b/source/index-storage.rst deleted file mode 100644 index f4379440..00000000 --- a/source/index-storage.rst +++ /dev/null @@ -1,7 +0,0 @@ -Storage solutions -################# - -.. toctree:: - :maxdepth: 2 - - ceph-deploy diff --git a/source/index-ts.rst b/source/index-ts.rst deleted file mode 100644 index cec9aa7d..00000000 --- a/source/index-ts.rst +++ /dev/null @@ -1,14 +0,0 @@ -Tutorials -######### - -.. toctree:: - :maxdepth: 2 - - gs-clear-containers-getting-started - ciao-deploy - ciao-cluster-setup-quick.rst - ciao-cluster-setup - tutorials/wordpress.rst - tutorials/flatpak/flatpak.rst - - diff --git a/source/index.rst b/source/index.rst index 2f50edba..d96ca398 100644 --- a/source/index.rst +++ b/source/index.rst @@ -1,84 +1,45 @@ .. ClearLinux Documentation documentation master file, created by - sphinx-quickstart on Fri Nov 13 12:23:35 2015. + sphinx-quickstart on Fri Nov 13 12:23:35 2015. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Clear Linux* Project for Intel® Architecture--Documentation -############################################################ +Clear technologies - Documentation +################################## -Getting started -================ +Clear Linux\* Project for Intel® Architecture +============================================= .. toctree:: :maxdepth: 2 - index_gs.rst + clear-linux/clear-linux -Running in a virtualized environment -==================================== +Intel® Clear Containers +======================= .. toctree:: :maxdepth: 2 - index_vm.rst + clear-containers/clear-containers - -Software update -=============== +Ciao +==== .. toctree:: :maxdepth: 2 - index_sw_update.rst - - -Bundles -======= - -.. toctree:: - :maxdepth: 2 - - index_bundles.rst - - -Advanced configuration -====================== - -.. toctree:: - :maxdepth: 2 - - index_advanced_configuration.rst - - -Storage solutions -================= - -.. toctree:: - :maxdepth: 2 - - index-storage.rst - - -Tutorials -========= - -.. toctree:: - :maxdepth: 2 - - index-ts.rst + ciao/ciao Indices and tables ================== * :ref:`genindex` -* :ref:`modindex` * :ref:`search` License and disclaimers ======================= -.. toctree:: .. include:: documentation_license.rst diff --git a/source/index_advanced_configuration.rst b/source/index_advanced_configuration.rst deleted file mode 100644 index a28012b6..00000000 --- a/source/index_advanced_configuration.rst +++ /dev/null @@ -1,15 +0,0 @@ -Advanced configuration -###################### - -.. toctree:: - :maxdepth: 2 - - clear-containers - network_boot - bulk_provisioning - mixer - ac-dpdk - ac-ovs-dpdk - validate_sigs - clear_linux_os_on_azure - diff --git a/source/index_bundles.rst b/source/index_bundles.rst deleted file mode 100644 index 768ba61f..00000000 --- a/source/index_bundles.rst +++ /dev/null @@ -1,10 +0,0 @@ -Bundles -####### - -.. toctree:: - :maxdepth: 2 - - bundles_overview - bdl-containers-basic - bdl-openssh-server - bdl-os-core diff --git a/source/index_gs.rst b/source/index_gs.rst deleted file mode 100644 index 178de866..00000000 --- a/source/index_gs.rst +++ /dev/null @@ -1,13 +0,0 @@ -Getting started -############### - -.. toctree:: - :maxdepth: 2 - - gs_getting_started - gs_supported_hardware - gs_creating_bootable_usb - gs_installing_clr_as_host - gs_clear-linux-kernels - - diff --git a/source/index_sw_update.rst b/source/index_sw_update.rst deleted file mode 100644 index 0d4ca479..00000000 --- a/source/index_sw_update.rst +++ /dev/null @@ -1,8 +0,0 @@ -Software update -############### - -.. toctree:: - :maxdepth: 2 - - swupdate_about_sw_update - swupdate_how_to_run_the_updater \ No newline at end of file diff --git a/source/index_vm.rst b/source/index_vm.rst deleted file mode 100644 index 8719be12..00000000 --- a/source/index_vm.rst +++ /dev/null @@ -1,12 +0,0 @@ -Running in a virtualized environment -#################################### - -.. toctree:: - :maxdepth: 2 - - vm-kvm - vm-virtualbox - vm-vmware-esxi - vm-vmware-player - vm-hyper-v - diff --git a/source/installing_openstack.rst b/source/installing_openstack.rst deleted file mode 100644 index b2611e10..00000000 --- a/source/installing_openstack.rst +++ /dev/null @@ -1,78 +0,0 @@ -.. _installing_openstack: - -Installing OpenStack* -##################### - -This section details an OpenStack installation that uses -bundles available for Clear Linux* OS for Intel® Architecture. - -The sample configuration files that are included will likely -require modification for your specific environment. - -This Clear Config Management uses Ansible* as configuration management tool. - - -Components supported by Clear Config Management -=============================================== - -At the moment, this installer can deploy any or all of the following -components: - - - MariaDB - - RabbitMQ - - Keystone - - Glance - - Nova - - Neutron - - Dashboard (In identity node) - - Heat - - Swift - -Note: ------ - -Below you will find the reference to componets supported on Clear Linux OS for Intel Architecture, but -its support in Clear Config Management is still pending development: - -.. csv-table:: "Supported Components on ClearLinux*, but unsupported by Clear Config Management" - :header: "Component", "Bundles", "OpenStack* official documentation" - :widths: 20, 70, 100 - - "Cinder", "openstack-block-storage and openstack-block-storage-controller", "http://docs.openstack.org/developer/cinder/" - "Ceilometer", "openstack-telemetry", "http://docs.openstack.org/developer/ceilometer/" - - -Prerequisites -============= - -Before the installer can set up your cloud environment, these prerequisites -should be completed (if they aren't already): - -#. Create a pair of SSH keys. - -#. Copy your public key to each node. - -#. Create the :file:`sshd_config` in ``/etc/ssh`` if it doesn't exist:: - - # mkdir -p /etc/ssh && touch /etc/ssh/sshd_config - -#. Allow ssh root access by adding "PermitRootLogin yes" to the ssh - configuration.:: - - # echo "PermitRootLogin yes" >> /etc/ssh/sshd_config - - -Using the Installer -=================== - -This step presumes a machine running Clear Linux OS for Intel Architecture as the ansible host. - - -Install the bundle ------------------- - -#. Install the `sysadmin-hostmgmt` bundle:: - - # swupd bundle-add sysadmin-hostmgmt - -**Next**: :ref:`configure_openstack_environment` diff --git a/source/mixer.rst b/source/mixer.rst deleted file mode 100644 index 572355c0..00000000 --- a/source/mixer.rst +++ /dev/null @@ -1,253 +0,0 @@ -.. _mixer_tool: - -Mixer Tool -########## - -*Mixing* refers to composing an operating system for specific use cases. -While the default Clear Linux* OS for Intel® Architecture provides options to install -bundles for various server capabilities, some developers may wish to 1) augment the -operating system itself with functionality from their own packages or 2) modify the -structure of current bundles to cater to their particular needs. - -Prerequisites -============= - -To start working with the Mixer tools, you'll need a recent image of Clear Linux OS for Intel Architecture -with the following bundle installed. If you don't have it already, -you can add it with the :command:`swupd bundle-add` command like so:: - - # swupd bundle-add mixer - -Current Workflow -================ - -Mixing ------- - -#. **Create a workspace**. Create an empty directory in your Clear image to - use as a "workspace" for mixing. For these steps, we assume your workspace - location is :file:`/home/clr/mix`. - -#. **Configure builder.conf**. Copy the template conf file:: - - # cp /usr/share/defaults/bundle-chroot-builder/builder.conf /home/clr/mix/ - - The file ``builder.conf`` will be read automatically from the current workspace directory, - but the ``-config`` option exists to specify where - the file is if you want to store it elsewhere. To use one in your current workspace, - copy the template to /home/clr/mix. - The :file:`.yum-mix.conf` file will be auto-generated for you, as will the :file:`Swupd_Root.pem`. - A yum configuration is needed for the chroot-builder to know where the RPMs are hosted, and the certificate - file is needed to sign the root Manifest to provide security for content verification. - - Note there are different sections to the builder.conf. The ``[Builder]`` section - provides the mixer tools with required configuration options, defining where - generated bundles and update metadata should get published. The ``[swupd]`` section - is used by swupd-server to create an update with specific update parameters. - - Edit the template configuration file according to your needs. For this example, - your ``builder.conf`` should look like this, with both *URL variables set to the domain or IP of the update server:: - - # vim /etc/bundle-chroot-builder/builder.conf: - - [Builder] - SERVER_STATE_DIR=/home/clr/mix/update - BUNDLE_DIR=/home/clr/mix/mix-bundles - YUM_CONF=/home/clr/mix/.yum-mix.conf - CERT=/home/clr/mix/Swupd_Root.pem - VERSIONS_PATH=/home/clr/mix/ - - [swupd] - BUNDLE=os-core-update - CONTENTURL= - VERSIONURL= - FORMAT=1 - - The SERVER_STATE_DIR is where the mix content will be outputted to, and it is automatically created for - you by the mixer. This can be set to any location, but for this example let's use the workspace directory. - The same applies for BUNDLE_DIR; it will be generated for you in the location specified in the builder.conf, - in this case ``/home/clr/mix/mix-bundles``. This is where the bundle definitions are stored for your mix, and - it's where the chroot-builder looks to know what bundles must be installed. - - You may change the ``CERT=/path/to/cert`` line, which tells the chroot builder to insert the certificate - specified for the mix in ``/os-core-update/usr/share/clear/update-ca/``. This is the certificate used by - the software update client to verify the Manifest.MoM signature. For now, it is `HIGHLY` recommended that - you do not modify this line, as the certificate swupd expects needs a very specific configuration to sign - and verify properly. The certificate will be automatically generated for you, and the Manifest.MoM will - be signed automatically as well, providing security for the update content you create. - - The CONTENTURL and VERSIONURL may be an IP address, or a domain name, which hosts the /home/clr/mix/update/www - (SERVER_STATE_DIR) directory. Creating a symlink to the directory in your server webdir is an easy way to host - the content. A client running the mix will look to that URL to figure out if there is a new version available - and the location from which to download the update content. - - To learn more about the FORMAT option, please refer to the bottom of this document "Format Version" and - https://github.com/clearlinux/swupd-server/wiki/Format-Bumps. For now leave the FORMAT - value alone and do not increment it. - - *The mix version and Clear version will come from two state files: :file:`.mixversion` and :file:`.clearversion`, - both of which will be created for you when you set-up the workspace and added to the VERSIONS_PATH defined. - -#. **Generate the starting point for your Mix**. In your workspace, run:: - - # sudo mixer init-mix -clearver 13180 -mixver 10 - - *If you wish to just build a mix that includes all Clear bundles with no modifications, run*:: - - # sudo mixer init-mix -all -clearver 13180 -mixver 10 - -#. **Create/locate RPMs for mix.**. (Steps 4-6 are necessary only if you - want to add your own RPMs to the Mix. If you are working only with Clear - bundles, then skip to Step 7.) - - If you are creating RPMs from scratch, you may use ``autospec``, - ``mock``, ``rpmbuild``, etc. to build them. If they are not - built on Clear, make sure your configuration and toolchain builds them correctly for Clear, or there is no guarantee - they will be compatible. - -#. **Import RPMs into workspace**. The way to do this is to create an - ``rpms`` directory in your workspace (for example ``/home/clr/mix/rpms``), - and to copy the RPMs you want into that directory. The mixer script will - look here for RPMs in order to build a local RPM repo for yum to use. - -#. **Create a local RPM repo**. Create an empty directory in your workspace - named ``local`` and add the paths in your builder.conf:: - - RPMDIR=/home/clr/mix/rpms - REPODIR=/home/clr/mix/local - - These variables are automatically read; you simply need to run:: - - # sudo mixer add-rpms - - After the script exits, you should see your RPMs and a repodata directory in - ``/home/clr/mix/local``. If the RPMs are not all in the local directory, check - to make sure that they are indeed valid RPM files and not corrupt. - -#. **Update/Add bundle definitions**. The mixer uses a local clone of the - ``clr-bundles`` repo to define bundles for the mix. - - To define your bundles: - #. Navigate to the ``mix-bundles/`` directory. - #. Make any needed modifications to the bundle set. - #. Commit the result:: - - $ git add . - $ git commit -s -m 'Update bundles for mix #' - - You can easily copy bundles over from the ``clr-bundles/clr-bundles-VER/bundles/`` directory in - the case that you want to simply use existing bundle sets. Note that - ``mix-bundles`` should not have any folders inside of it, only bundle definitions. - Do *not* modify things in the clr-bundles dir, this is simply a mirror for you to - use or refer to the Clear Linux OS bundle definitions. - - Why do this? With Git history, mixes are easy to revert to or refer - to in the future if something were to go wrong with a new mix. If - you're just testing this out, or if you really do not want to mess with Git, - you can ignore committing for now. - - To add your own bundle, create a bundle definition file in ``mix-bundles/`` - and refer to :file:`mix-bundles/os-core-update` for formatting, but be sure that - the name does not conflict with another bundle. Add your package - name(s) in that bundle definition file to tell it what package(s) - must be installed as part of that bundle. - -#. **Build the bundle chroots** To build all of the ``chroots`` - that are based on the bundles you defined, in your workspace run:: - - # sudo mixer build-chroots - - If you have many bundles defined for your mix, this step may take some time. - -#. **Create update**. In the workspace, run:: - - # sudo mixer build-update - - When the build completes, you'll find your mix update content under - ``/home/clr/mix/update/www/VER``. In this example, it will be located in - ``/home/clr/mix/update/www/``, where is the mix version you - defined, or 10 by default. - - All content to make a fully usable mix will be created by this step, but note that - only zero packs are automatically generated. To create optional delta packs, run - the pack-maker as follows:: - - # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update - - The pack-maker will generate all delta packs for changed bundles from PAST_VERSION - to MIX_VERSION. If your STATE_DIR is in a different location be sure to specify where - with the -S option. - For the first build, no delta packs can be created because the "update" is from version 0, which - impicitly has no content, thus no deltas can be generated. For subsequent builds, mixer-pack-maker.sh - can be run to generate delta content between them (i.e 10 to 20). - -#. **Creating an image** - To create a bootable image from your update content, you will need the configuration file for - ister to create images:: - - # curl -O https://raw.githubusercontent.com/clearlinux/ister/master/release-image-config.json - - Edit this to include all the bundles you want pre-installed into your image. For a minimal, base - image this would be:: - - "Bundles": ["os-core", "os-core-update", "kernel-native"] - - And lastly, set the "Version:" to say which mix version content the image should be built from, - i.e. 10 for your first build. To build the image, run:: - - # sudo mixer build-image -format 1 - - The output from this should be an image that is bootable as a VM or installable to baremetal. *Note* you - need to pass in -format if the format you are building is different than the - format of Clear Linux OS you are currently building on. Format version can be found via:: - - # cat /usr/share/defaults/swupd/format - -Creating your next Mix version ------------------------------- - -#. **Initialize next Mix version info**. To update the versions and prep for your - next mix:: - - Update the .mixversion file to the next version number you want to build. - From this point you can iterate through, starting again at step 4 and doing modifications - as needed. For example: - - - Add/Remove/Modify Bundles - - sudo mixer build-chroots - - sudo mixer build-update - - (Optionally) sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update - Next mix created. - -#. **Update Bundles (Optional)**. Update ``clr-bundles``. In the workspace, - run:: - - # sudo mixer get-bundles - - This step is optional because it is only needed when you want to update the upstream clr-bundles in your - workspace to a new version, which requires updating the .clearversion file. - -Format Version --------------- - -The "format" used in ``builder.conf`` might be more precisely referred to as an -OS "compatibility epoch". Versions of the OS within a given epoch are fully -compatible with themselves and can update to any version in that epoch. Across -the format boundary *something* has changed in the OS, such that updating from -build M in format X, to build N in format Y will not work. Generally this occurs -when the software updater or manifests changed in a way that is no longer -compatible with the previous update scheme. - -A format increment is the way we insure pre- and co-requisite -changes flow out with proper ordering. The update client will only ever update -to the latest release in its respective format version (unless overridden by -command line flags), thus we can guarantee all clients will update to the final -version in their given format, which *must* contain all the changes needed -to understand the content built in the following format. Only after reaching the -final release in the old format will a client be able to continue to update to -releases in the new format. - -For the creation of a custom mix, the format version should start at '1', -or some known number, and increment only when a compatibility breakage is -introduced. Normal updates (updating a software package for example) -do not require a format increment. diff --git a/source/openstack_bundle_and_service_summary.rst b/source/openstack_bundle_and_service_summary.rst deleted file mode 100644 index 362285ff..00000000 --- a/source/openstack_bundle_and_service_summary.rst +++ /dev/null @@ -1,65 +0,0 @@ -.. _openstack_bundle_and_service_summary: - -OpenStack* bundle and service summary -##################################### - -OpenStack provides an Infrastructure-as-a-Service (IaaS) solution -through a variety of complementary services. Each service offers an -application programming interface (API) that facilitates this -integration. The following table provides a list of OpenStack services: - -.. csv-table:: OpenStack services - :header: "Bundle Name (service)", "Project Name", "Description" - :widths: 90, 90, 300 - - "openstack-compute-controller", "Nova", "Manages the lifecycle of - compute instances in an OpenStack environment. Responsibilities include - spawning, scheduling and decommissioning of virtual machines on demand." - "openstack-identity", "Keystone", "Provides an authentication and - authorization service for other OpenStack services. Provides a catalog - of endpoints for all OpenStack services." - "openstack-image", "Glance", "Stores and retrieves virtual machine disk - images. OpenStack Compute makes use of this during instance provisioning." - "openstack-dashboard", "Horizon", "Provides a web-based self-service portal - to interact with underlying OpenStack services." - "openstack-network", "Neutron", "Enables Network-Connectivity-as-a-Service - for some OpenStack services. Besides, it provides an API for users to define - networks and the attachments into them." - "openstack-object-storage", "Swift", "Stores and retrieves arbitrary - unstructured data objects. It is highly fault tolerant with its data - replication and scale-out architecture." - "openstack-block-storage/openstack-block-storage-controller", "Cinder", "Provides - persistent block storage to running instances." - "openstack-telemetry", "Ceilometer", "Monitors and meters the - OpenStack cloud for billing, benchmarking, scalability, and statistical - purposes." - "openstack-orchestration", "Heat", "Orchestrates multiple composite cloud - applications by using either the native Heat Orchestration Template (HOT) - template format or the AWS CloudFormation template format." - -OpenStack is highly configurable to meet different needs. This guide -uses a two-node architecture. - -- The controller node runs the Identity Service, Image Service, the - management portion of Compute, and the Dashboard. It also includes - supporting services, such as an SQL database and message queue. - Optionally, the controller node runs management portions of Block - Storage services. -- The compute node runs the hypervisor portion of Compute that operates - project virtual machines or instances. By default, Compute uses KVM - as the hypervisor. Compute also provisions project networks and - provides firewalling (security groups) services. You can run more - than one compute node. -- (Optional) Block Storage node contains the disks that the Block - Storage service provisions for instances. You can deploy more than - one block storage node. -- (Optional) Object Storage node contain the disks that the Object - Storage service uses for storing accounts, containers, and objects. - This service requires two nodes. Each node requires a minimum of one - network interface. You can deploy more than two object storage nodes. -- The self-service networks option augments the provider networks option - with layer-3 (routing) services that enable self-service networks - using overlay segmentation methods. Additionally, this option provides - the foundation for advanced services such as LBaaS and FWaaS. - -Next topic: :ref:`installing_openstack`. diff --git a/source/openstack_conf_vars_about.rst b/source/openstack_conf_vars_about.rst deleted file mode 100644 index d5ad8ce4..00000000 --- a/source/openstack_conf_vars_about.rst +++ /dev/null @@ -1,14 +0,0 @@ -.. _openstack_conf_vars_about: - -About "group_vars/all" -###################### - -The ``group_vars/all`` section is where you will specify the -values for your passwords and other required or optional parameters. - -Variables in this section are built with yaml format. - -For a more detailed reference of the numerous variables and parameters -that can be set, see the :ref:`openstack_conf_vars_list`. - -**Back to Configuration Section** :ref:`configure_openstack_environment` diff --git a/source/openstack_conf_vars_list.rst b/source/openstack_conf_vars_list.rst deleted file mode 100644 index 682f02d1..00000000 --- a/source/openstack_conf_vars_list.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. _openstack_conf_vars_list: - -Variables for OpenStack* Deployment -################################### - -This installer provides a variety of configurations you can set through -variables; below you will find a reference of the components supported -with Clear Linux* OS for Intel® Architecture. - -Required Variables -================== - -.. csv-table:: - :header: "*(Component)* `Variable`", "Description" - :widths: 90, 150 - - "*(Heat, Neutron, Nova, RabbitMQ)* **rabbitmq_password**", "Password of RabbitMQ user " - "*(ALL)* **database_root_password**","Password for root database user" - "*(Keystone)* **keystone_database_password**", "Password for 'keystone' database" - "*(ALL)* **keystone_admin_password**", "Password for 'admin' user" - "*(Glance)* **glance_user_password**", "Password for 'glance' user" - "*(Glance)* **glance_database_password**", "Password for 'glance' database" - "*(Neutron, Nova)* **nova_user_password**", "Password for 'nova' user" - "*(Nova)* **nova_database_password**", "Password for 'nova' database" - "*(Neutron)* **neutron_database_password**", "Password for 'neutron' database" - "*(Neutron, Nova)* **neutron_user_password**", "Password for 'neutron' user" - "*(Neutron, Nova)* **metadata_proxy_shared_secret**", "Secret for the metadata proxy" - "*(Heat)* **heat_domain_admin_password**", "Password for heat domain admin user" - "*(Heat)* **heat_user_password**", "Password for heat user" - "*(Heat)* **heat_database_password**", "Password for heat database" - "*(Heat)* **heat_domain**", "The heat domain that contains projects and users for stacks" - "*(Swift)* **swift_user_password**", "Password for 'swift' user" - "*(Swift)* **swift_database_password**", "Password for 'swift' database" - "*(Swift)* **swift_replica_count**", "Replica number for each object. IMPORTANT: This number must be lower than the sum of all the storage devices among all storage nodes. It can be changed later." - "*(Swift)* **swift_hash_path_suffix**", "Suffix for the object path name" - "*(Swift)* **swift_hash_path_prefix**", "Prefix for the object path name" - "*(Swift)* **swift_storage_device_path**", "The path of the storage devices" - "*(Swift)* **swift_storage_devices**", "A list of the storage devices dedicated to swift deployment. For more information, see :ref:`openstack_swift_deployment_scenarios`" - -Optional Variables -================== - -.. csv-table:: - :header: "*(Component)* **Variable** : Default value", "Description" - :widths: 90, 150 - - "*(ALL)* **swupd_args**: unset", "Optional arguments for swupd" - "*(ALL)* **log_debug**: False", "Set to True to enable debug log level on all services" - "*(Heat, Neutron, Nova, RabbitMQ)* **rabbitmq_username**: openstack", "User ID for RabbitMQ" - "*(Neutron, Nova)* **neutron_public_interface_name**: unset", "Public interface of Neutron machines, if is not set, it will take the default interface reported by **ip route**" - "*(Nova)* **nova_public_interface_name**: unset", "Public interface of Neutron machines, if is not set, it will take the default interface reported by **ip route**" - "*(Nova)* **nova_virt_type**: qemu", "Virtualization type (qemu | kvm), if this is not set, then the playbook will try to guess it" - "*(Neutron)* **os_tuning_params**: net.ipv4.ip_forward: 1, net.ipv4.conf.default.rp_filter: 0, net.ipv4.conf.all.rp_filter: 0, net.bridge.bridge-nf-call-iptables: 1, net.bridge.bridge-nf-call-ip6tables: 1", "syctl values needed by neutron when using openvswitch deployment scenario" - "*(Swift)* **swift_public_interface_name**, "Public interface of storage nodes, also known as the storage network interface name; If is not set, it will take the default interface reported by **ip route**" - - -Note: ------ -If you would like to override a default, you can define it in the -:ref:`openstack_conf_vars_about`. - -**Back to Configuration Section** :ref:`configure_openstack_environment` diff --git a/source/openstack_env_inventory_file.rst b/source/openstack_env_inventory_file.rst deleted file mode 100644 index 8398bf5e..00000000 --- a/source/openstack_env_inventory_file.rst +++ /dev/null @@ -1,61 +0,0 @@ -.. _openstack_env_inventory_file: - -The "hosts" file -################ - -The :file:`hosts` file is an inventory file where you specify, via -``ini`` format, the roles your machine plays. - -The :file:`hosts` file can be structured with different groups, where each -group plays a specific role in your environment and network mapping. - -Consider the following example:: - - [openstack_identity] - 192.168.50.13 - -Here, the role ``[openstack_identity]`` syntax defines the group, and -``192.168.50.13`` specifies the machine(s) that possess that role. You -may define multiple machines in the same group; this is a standard -configuration for role ``[openstack_compute]`` nodes:: - - [openstack_compute] - 192.168.50.16 - 192.168.50.17 - - -Inventory file groups ---------------------- - -Below you will find the description of each group in the `hosts` file. - -.. csv-table:: "Inventory File Groups" - :header: "Group", "Components", "Comments" - :widths: 40, 40, 300 - - "[dbservers]", "MariaDB", "" - "[messaging_servers]", "RabbitMQ", "" - "[openstack_identity]", "Keystone, Horizon", "" - "[openstack_image]", "Glance", "" - "[openstack_compute_controller]", "Nova", "" - "[openstack_compute]", "Nova", "Accepts multiple entries to have multiple compute nodes. You can add more entries and re-run the installer to add them to your environment." - "[openstack_networking]", "Neutron", "" - "[openstack_orchestration]", "Heat", "" - "[openstack_object_storage_controller]", "Swift", "The Swift storage controller. It runs the swift proxy server and the memcache server" - "[openstack_object_storage]", "Swift", "The Swift Storage nodes. Accepts multiple entries to have multiple storage nodes" - - -Important notes ---------------- - -* To omit any role, do not add an entry under its group section. - -* To create an ``All In One`` scenario, specify the same machine name or IPv4 - Address under all of the groups. - -For further reference, check out the `Ansible`_ documentation. - - -**Back to Configuration Section** :ref:`configure_openstack_environment` - -.. _Ansible: http://docs.ansible.com/ansible/intro_inventory.html diff --git a/source/openstack_swift_deployment_scenarios.rst b/source/openstack_swift_deployment_scenarios.rst deleted file mode 100644 index 4c9de9b2..00000000 --- a/source/openstack_swift_deployment_scenarios.rst +++ /dev/null @@ -1,69 +0,0 @@ -.. _openstack_swift_deployment_scenarios: - -OpenStack* Swift* Deployment Scenarios -###################################### - -You can install OpenStack Swift using clear-config-management through two -possible scenarios. - -- **Scenario #1:** All the storage nodes are identical, with the same number of storage devices with the same names. - -- **Scenario #2:** The storage setup is heterogeneous. The list of storage devices must be provided for each storage node individually. - - -Scenario #1: Identical storage nodes scenario -============================================= - -To set up this scenario, the ``swift_storage_devices`` variable needs to be -defined in ``../group_vars/all`` as follows: - -.. code-block:: yaml - - ... - swift_storage_device_path: /dev/ - swift_storage_devices: - - sda - - sdb - ... - -Scenario #2: Different storage nodes scenario -============================================= - -With this setup, specific information about each storage node has to be provided. - -First, in the root of your Ansible* directory setup, create a new directory called ``host_vars``. -Inside ``host_vars``, for each storage node, create a new file with the name of the -storage node's hostname or IP. In each file, provide the list of the storage -devices. - -The Ansible directory setup should look similar to the following directory tree: - -.. code-block:: console - - openstack/ - ├── group_vars - │ └── all - ├── hosts - ├── host_vars - │ ├── storage-one - │ └── storage-two - ├── openstack_deployment.yml - └── README.md - -And the ``storage-one`` and ``storage-two`` storage node files should look similar to this: - -.. code-block:: yaml - - --- - devices: - - sdb - - sdc - - sdd - - -.. code-block:: yaml - - --- - devices: - - vdb - - vdc diff --git a/source/openstack_telemetry_compute.rst b/source/openstack_telemetry_compute.rst deleted file mode 100644 index 16db9093..00000000 --- a/source/openstack_telemetry_compute.rst +++ /dev/null @@ -1,82 +0,0 @@ -OpenStack* Telemetry - Enable compute service meters -############################################################ - -Telemetry uses a combination of notifications and an agent to -collect Compute meters. Perform these steps on each compute node. - -Configure components ------------------------------------------------ - -Edit the ``/etc/ceilometer/ceilometer.conf`` file and complete the following actions: - -#. In the ``[DEFAULT]`` and ``[oslo_messaging_rabbit]`` sections, configure RabbitMQ message queue access:: - - [DEFAULT] - ... - rpc_backend = rabbit - - [oslo_messaging_rabbit] - ... - rabbit_host = controller - rabbit_userid = openstack - rabbit_password = RABBIT_PASS - - Replace the ``RABBIT_PASS`` with the password you chose for the openstack account in RabbitMQ. - -#. In the ``[DEFAULT]`` and ``[keystone_authtoken]`` sections, configure Identity service access:: - - [DEFAULT] - ... - auth_strategy = keystone - - [keystone_authtoken] - ... - auth_uri = http://controller:5000 - auth_url = http://controller:35357 - auth_plugin = password - project_domain_id = default - user_domain_id = default - project_name = service - username = ceilometer - password = CEILOMETER_PASS - - Replace ``CEILOMETER_PASS`` with the password you chose for the Telemetry service database. - -#. In the ``[service_credentials]`` section, configure service credentials:: - - [service_credentials] - ... - os_auth_url = http://controller:5000/v2.0 - os_username = ceilometer - os_tenant_name = service - os_password = CEILOMETER_PASS - os_endpoint_type = internalURL - os_region_name = RegionOne - - Replace ``CEILOMETER_PASS`` with the password you chose for the ceilometer user in the Identity service. - - -Configure Compute to use Telemetry ------------------------------------------------ - -#. Edit the ``/etc/nova/nova.conf`` file and configure notifications in the ``[DEFAULT]`` section:: - - [DEFAULT] - ... - instance_usage_audit = True - instance_usage_audit_period = hour - notify_on_state_change = vm_and_task_state - notification_driver = messagingv2 - - -Finalize the installation ------------------------------------------------ - -#. Restart the agent:: - - # systemctl enable ceilometer-agent-compute.service - # systemctl restart ceilometer-agent-compute.service - -#. Restart the compute service:: - - # systemctl restart nova-compute.service diff --git a/source/openstack_telemetry_image.rst b/source/openstack_telemetry_image.rst deleted file mode 100644 index 86614c4b..00000000 --- a/source/openstack_telemetry_image.rst +++ /dev/null @@ -1,35 +0,0 @@ -OpenStack* Telemetry - Enable image service meters -############################################################ - -Telemetry uses notifications to collect Image service meters. -Perform these steps on the controller node. - - -Configure the Image service to use Telemetry ------------------------------------------------ - -Edit the ``/etc/glance/glance-api.conf`` and ``/etc/glance/glance-registry.conf`` -files and complete the following actions: - -#. In the ``[DEFAULT]`` section, configure notifications and RabbitMQ message broker access:: - - [DEFAULT] - ... - notification_driver = messagingv2 - rpc_backend = rabbit - - [oslo_messaging_rabbit] - ... - rabbit_host = controller - rabbit_userid = openstack - rabbit_password = RABBIT_PASS - - Replace the ``RABBIT_PASS`` with the password you chose for the openstack account in RabbitMQ. - -Finalize the installation ----------------------------- - -#. Restart the image service:: - - # systemctl restart glance-registry.service - # systemctl restart glance-api.service diff --git a/source/openstack_telemetry_verify.rst b/source/openstack_telemetry_verify.rst deleted file mode 100644 index a6a54714..00000000 --- a/source/openstack_telemetry_verify.rst +++ /dev/null @@ -1,59 +0,0 @@ -OpenStack* Telemetry - Verify Operation -############################################################ - -Verify Operation ------------------- - -Verify operation of the Telemetry service. These steps only include the Image service meters to reduce clutter. Environments with ceilometer integration for additional services contain more meters. - - -#. Source the ``admin`` credentials to gain access to admin-only CLI - commands:: - - $ source admin-openrc.sh - -#. List available meters:: - - $ ceilometer meter-list - +--------------+-------+-------+--------------------------------------+---------+------------+ - | Name | Type | Unit | Resource ID | User ID | Project ID | - +--------------+-------+-------+--------------------------------------+---------+------------+ - | image | gauge | image | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - | image.size | gauge | B | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - +--------------+-------+-------+--------------------------------------+---------+------------+ - -#. Download the CirrOS image from the Image service:: - - $ IMAGE_ID=$(glance image-list | grep 'cirros' | awk '{ print $2 }') - $ glance image-download $IMAGE_ID > /tmp/cirros.img - -#. List available meters again to validate detection of the image download:: - - $ ceilometer meter-list - +----------------+-------+-------+--------------------------------------+---------+------------+ - | Name | Type | Unit | Resource ID | User ID | Project ID | - +----------------+-------+-------+--------------------------------------+---------+------------+ - | image | gauge | image | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - | image.download | delta | B | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - | image.serve | delta | B | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - | image.size | gauge | B | acafc7c0-40aa-4026-9673-b879898e1fc2 | None | cf12a15... | - +----------------+-------+-------+--------------------------------------+---------+------------+ - -#. Retrieve usage statistics from the image.download meter:: - - $ ceilometer statistics -m image.download -p 60 - - +--------+---------------------+---------------------+------------+------------+------------+------------+-------+----------+----------------------------+----------------------------+ - | Period | Period Start | Period End | Max | Min | Avg | Sum | Count | Duration | Duration Start | Duration End | - +--------+---------------------+---------------------+------------+------------+------------+------------+-------+----------+----------------------------+----------------------------+ - | 60 | 2015-04-21T12:21:45 | 2015-04-21T12:22:45 | 13200896.0 | 13200896.0 | 13200896.0 | 13200896.0 | 1 | 0.0 | 2015-04-21T12:22:12.983000 | 2015-04-21T12:22:12.983000 | - +--------+---------------------+---------------------+------------+------------+------------+------------+-------+----------+----------------------------+----------------------------+ - -#. Remove the previously downloaded image file /tmp/cirros.img:: - - $ rm /tmp/cirros.img - - - - - diff --git a/source/openstack_use_cases.rst b/source/openstack_use_cases.rst deleted file mode 100644 index fe69751f..00000000 --- a/source/openstack_use_cases.rst +++ /dev/null @@ -1,27 +0,0 @@ -.. _openstack_use_cases: - -OpenStack* use cases with Clear Linux* OS for Intel® Architecture -################################################################### - -The installation and configuration of a Cloud Management Platform -depends on use cases and associated architectural choices. - -OpenStack supports the following use cases: - -- *General purpose* -- A cloud built with common components that should - address about 80 percent of common use cases. - -- *Compute-focused* -- A cloud designed to address compute-intensive - workloads such as high performance computing (HPC). - -- *Storage-focused* -- A cloud focused on storage intensive workloads - such as data analytics with parallel file systems. - -- *Network-focused* -- A cloud depending on high performance and - reliable networking, such as a Content Delivery Network (CDN). - -Clear Linux OS for Intel Architecture is designed with all of these use -cases in mind, but the initial target is to enable only those services -required for Cloud. - -Next topic: :ref:`openstack_bundle_and_service_summary`. \ No newline at end of file