Merge branch 'master' into bd-telemetry-e2e

updating stale branch
This commit is contained in:
Beth Dean
2018-12-11 08:53:59 -08:00
18 changed files with 1002 additions and 343 deletions
Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

+71 -12
View File
@@ -3,21 +3,80 @@
Mixer
#####
Mixing refers to composing a custom, versioned image of |CL-ATTR| for a
specific use case. While upstream |CL| provides options to install bundles
for various capabilities, some developers and OSVs may wish to either
augment the operating system with functionality from their own packages
or modify the structure of current bundles. Mixing offers a method to add
capabilities while retaining the ability to stay up to date with an upstream
version of |CL|.
|CL-ATTR| is a powerful, modular, and customizable OS. Upstream |CL| offers
many images to support different environments and use-cases. There are
hundreds of bundles that will meet most, if not all, of your OS and software
needs.
Mixing is a multi-step process that starts with installing the mixer bundle
using swupd. You will also need a working knowledge of
:abbr:`RPMs (RPM Package Manager files)` and how |CL| bundles work. For a
detailed guide on how to create a |CL| mix, visit :ref:`mixer`.
However, if you need additional customization or content, |CL| provides the
mixer tool. Depending on your needs, the mixer tool allows you to:
1. :ref:`create-mix` to create a distinct derivative of the |CL| that
contains your custom software.
2. :ref:`create-mixin` to add custom bundles but also keep updating the OS
from upstream.
.. _create-mix:
Create a mix
============
When creating a mix, you can
* Use any existing upstream bundles with no modification.
* Redefine what goes into existing bundles.
* Create completely new, custom bundles with your own custom packages.
With mixer you are not required to incorporate every upstream release into
your mix. You decide which upstream versions to update your derivative to, as
illustrated in Figure 1.
.. figure:: figures/mixer-about-1.png
:scale: 75%
:alt: Creating a custom mix.
Figure 1: With a custom mix, you add your custom bundle and decide which
upstream versions to update your mix to, on your own release cycle.
Creating your own mix forks away from the |CL| upstream and requires that you
act as your own OSV. There is a greater level of responsibility, requiring
more infrastructure and processes to adopt. However, with this approach, you
have a higher degree of control and customization of your custom |CL|.
.. _create-mixin:
Create a mixin
==============
The second option is to use the :command:`mixin` tool, a light wrapper for
mixer, to create custom bundles and sideload them into your upstream version
of |CL|. A mixin is useful when you need to add custom or 3rd
party content but want to keep on the upstream update cycle, as shown in
Figure 2. You can also create new bundles using upstream packages.
.. figure:: figures/mixer-about-2.png
:scale: 75%
:alt: Creating a custom mix.
Figure 2: With a mixin you can add custom bundles, but stay on upstream.
Mixin is primarily intended for end users. It is easier to adopt as it does
not require breaking from upstream or acting as an OSV. With mixin
* You are responsible for maintaining and testing your custom bundle(s).
* You retain access to all upstream bundles and updates.
* You can easily revert your system back to the upstream version.
Related topics
==============
* :ref:`swupd-about`
|CL| provides flexibility in how you customize your OS. Learn more about
mixer and related topics to decide which customization approach is best for
you.
* :ref:`mixer`
* :ref:`mixin`
* :ref:`bundles-about`
* :ref:`swupd-about`
* :ref:`deploy-at-scale`
@@ -3,10 +3,10 @@
Install |CL-ATTR| from the live desktop beta image
##################################################
The live desktop beta image allows you to boot |CL-ATTR| into a GNOME
desktop without modifying the host system. Using the live image, you can
explore the possibilities of developing with |CL|. You can also launch the
new installer and install |CL| on your target system.
The live desktop beta image allows you to boot |CL-ATTR| into a GNOME
desktop without modifying the host system. Using the live image, you can
explore the possibilities of developing with |CL|. You can also launch the
new installer and install |CL| on your target system.
.. contents:: :local:
:depth: 1
@@ -14,19 +14,19 @@ new installer and install |CL| on your target system.
System requirements
*******************
Assure that your target system supports the installation:
Assure that your target system supports the installation:
* :ref:`system-requirements`
* :ref:`compatibility-check`
Preliminary steps
Preliminary steps
*****************
#. `Navigate to the image directory`_.
#. `Navigate to the image directory`_.
#. Download the file :file:`clear-<release number>-live-desktop-beta.img.xz`
.. note::
.. note::
``<release-number>`` is the latest |CL| auto-numbered release.
@@ -42,15 +42,15 @@ Install from live image
***********************
After you've downloaded and burned the live desktop image on a
USB drive, follow these steps.
USB drive, follow these steps.
#. Insert the USB drive into an available USB slot.
#. Power on the system.
#. Open the system BIOS setup menu by pressing the F2 key.
#. Open the system BIOS setup menu by pressing the F2 key.
.. note::
.. note::
Your BIOS setup menu entry point may vary.
@@ -64,9 +64,9 @@ USB drive, follow these steps.
Launch the installer
====================
#. After the |CL| live desktop image boots, select the :guilabel:`Activities` menu in the upper left.
#. After the |CL| live desktop image boots, select the :guilabel:`Activities` menu in the upper left.
#. Select the icon, :guilabel:`Install Clear Linux OS`, shown in Figure 1.
#. Select the icon, :guilabel:`Install Clear Linux OS`, shown in Figure 1.
.. figure:: figures/bare-metal-install-beta-1.png
:scale: 50 %
@@ -82,37 +82,39 @@ Launch the installer
Figure 2: Clear Linux OS Installer
.. _minimum-installation-requirements:
.. _minimum-installation-requirements:
Minimum installation requirements
*********************************
To fulfill minimum installation requirements, complete
:guilabel:`Required options`. All other menu items are optional.
To fulfill minimum installation requirements, complete the
:guilabel:`Required options`. :guilabel:`Advanced options` are optional. Additional user help is described below.
.. note::
* In the Main menu, the :kbd:`Install` button is highlighted
**only after** you complete :ref:`required-options-installation`.
* The :kbd:`Install` button is only highlighted **after** you complete the
:ref:`required-options-installation`, and after you enter all required
values in submenus of :ref:`required-options-installation` and
:ref:`advanced-options-installation`.
* You must choose whether or not to participate in :ref:`telemetrics`
before you can :ref:`finish-cl-installation`.
before you can finish installation.
* You may wish to :ref:`test-network-settings` before you
:ref:`configure-network-interfaces`. Assure that a ``Success`` message is received.
* You may wish to :ref:`test-network-settings` before you
:ref:`configure-network-interfaces`. Assure that a ``Success`` message is received before installation.
Main menu
Main Menu
*********
In the |CL| Installer main menu, two tabs appear:
In the |CL| Installer Main Menu, two tabs appear:
:guilabel:`[R] Required options` and :guilabel:`[A] Advanced options`.
Navigate between tabs using the arrow keys or these shortcut keys:
* :kbd:`Shift+A` for :guilabel:`[A] Advanced options`
* :kbd:`Shift+R` for :guilabel:`[R] Required options`
To meet the minimum requirements, enter your choices in the
:guilabel:`Required options`. After confirmation, your selections appear
To meet the minimum requirements, enter your choices in the
:guilabel:`Required options`. After confirmation, your selections appear
below menu options in the Main Menu, beside the :guilabel:`>>` chevron.
Navigation
@@ -124,13 +126,13 @@ Navigation
* Select :kbd:`Cancel` or :kbd:`Esc` to cancel your choice.
The |CL| Installer main menu appears as shown in Figure 3.
The |CL| Installer Main Menu appears as shown in Figure 3.
.. figure:: figures/bare-metal-install-beta-3.png
:scale: 100 %
:alt: Clear Linux OS Installer main menu
:alt: Clear Linux OS Installer Main Menu
Figure 3: :guilabel:`Clear Linux OS Installer main menu`
Figure 3: :guilabel:`Clear Linux OS Installer Main Menu`
.. _required-options-installation:
@@ -140,7 +142,7 @@ Required options
Choose Timezone
===============
#. From the main menu, select :guilabel:`Choose Timezone`.
#. From the Main Menu, select :guilabel:`Choose Timezone`.
#. Select :kbd:`Enter` to continue.
@@ -154,7 +156,7 @@ Choose Timezone
.. note::
Select :guilabel:`Cancel` to restore the previously defined value and
return to the main menu.
return to the Main Menu.
.. figure:: figures/bare-metal-install-beta-4.png
:scale: 100 %
@@ -165,23 +167,23 @@ Choose Timezone
Choose Language
===============
#. From the main menu, select :guilabel:`Choose Language`.
#. From the Main Menu, select :guilabel:`Choose Language`.
#. Select :kbd:`Enter`.
#. In :guilabel:`Select System Language`, navigate to your desired language.
#. Select :kbd:`Tab` to highlight :kbd:`Confirm`.
#. Select :kbd:`Tab` to highlight :kbd:`Confirm`.
#. Select :kbd:`Enter` to :kbd:`Confirm`.
.. note::
Select :guilabel:`Cancel` to restore the previously defined value and return to the main menu.
.. note::
Select :guilabel:`Cancel` to restore the previously defined value and return to the Main Menu.
.. figure:: figures/bare-metal-install-beta-5.png
:scale: 100 %
:alt: Select System Language
:alt: Select System Language
Figure 5: :guilabel:`Select System Language`
@@ -189,22 +191,22 @@ Choose Language
Configure the Keyboard
======================
#. From the main menu, select :guilabel:`Configure the Keyboard`.
#. From the Main Menu, select :guilabel:`Configure the Keyboard`.
#. Select :kbd:`Enter`.
#. Select :kbd:`Enter`.
#. In :guilabel:`Select Keyboard`, navigate to the desired option.
#. In :guilabel:`Test keyboard`, type text to assure that the keys map to
your keyboard.
your keyboard.
#. Select :kbd:`Tab` to highlight :guilabel:`Confirm`.
#. Select :kbd:`Enter` to :kbd:`Confirm`.
.. note::
.. note::
Select :guilabel:`Cancel` to restore the previously defined value and return to the main menu.
Select :guilabel:`Cancel` to restore the previously defined value and return to the Main Menu.
.. figure:: figures/bare-metal-install-beta-6.png
:scale: 100 %
@@ -215,73 +217,69 @@ Configure the Keyboard
Configure Media
===============
#. From the main menu, select :guilabel:`Configure Media`.
#. From the Main Menu, select :guilabel:`Configure Media`.
.. note::
Available media appear by default.
#. Navigate to the desired media until highlighted.
Available media appear by default.
.. TODO: Add another screenshot showing contextual menu.
#. Navigate to the desired media until highlighted.
.. figure:: figures/bare-metal-install-beta-7.png
:scale: 100 %
:alt: Configure Media menu
Figure 7: :guilabel:`Configure Media menu`
.. note::
Use additional options as necessary during partitioning:
* Select :guilabel:`Revert` to cancel changes and revert to the
original configuration.
* Select :guilabel:`Rescan` to show media connected to the host
machine.
#. Select :kbd:`Enter` or :kbd:`Spacebar` to select it.
Two options for partitioning appear.
Two options for partitioning appear.
#. Select :guilabel:`Auto Partition` to accept the default configuration, or
:guilabel:`Add Partition` to add partitions manually. See instructions
below.
.. figure:: figures/bare-metal-install-beta-7.png
:scale: 100 %
:alt: Configure Media menu
Figure 7: :guilabel:`Configure Media menu`
.. note::
Use additional options as necessary during partitioning:
* Select :guilabel:`Revert` to cancel changes and revert to the
original configuration.
* Select :guilabel:`Rescan` to show media connected to the host
machine.
#. Select :kbd:`Enter` to :kbd:`Confirm`.
Choose one partitioning option below
------------------------------------
#. Choose one partitioning option and continue below:
* :ref:`auto-partition`
* :ref:`add-partition`
* :ref:`auto-partition`
* :ref:`add-partition`
.. _auto-partition:
.. _auto-partition:
Auto Partition
==============
Auto partition will configure the media to meet the minimum requirements.
Auto partition will configure the media to meet the minimum requirements.
#. Select :guilabel:`Auto Partition` as shown in Figure 8.
#. Select :guilabel:`Auto Partition` as shown in Figure 8.
#. Select :kbd:`Enter` or :kbd:`Spacebar` to highlight
:guilabel:`Auto partition`.
#. Select :kbd:`Enter` or :kbd:`Spacebar` to highlight
:guilabel:`Auto partition`.
#. Select :kbd:`Enter` to :kbd:`Confirm`.
This action returns you to the Main Menu.
This action returns you to the Main Menu.
.. figure:: figures/bare-metal-install-beta-8.png
:scale: 100 %
:alt: Auto Partition
:alt: Auto Partition
Figure 8: :guilabel:`Auto Partition`
#. You may skip to the :ref:`telemetry-tui` section below.
#. You may skip to the :ref:`telemetry-tui` section below.
.. _add-partition:
.. _add-partition:
Add Partition
=============
@@ -289,104 +287,104 @@ Add Partition
Minimum requirements
====================
To add partitions manually, assign partitions per the minimum
To add partitions manually, assign partitions per the minimum
requirements shown in Table 1:
.. list-table:: **Table 1. Disk Partition Setup**
:widths: 33, 33, 33
:header-rows: 1
* - FileSystem
* - FileSystem
- Mount Point
- Minimum size
* - ``VFAT``
- /boot
- 150M
* - ``swap``
-
-
- 256MB
* - ``root``
- /
- *Size depends upon use case/desired bundles.*
- *Size depends upon use case/desired bundles.*
#. In the Configure Media menu, select :guilabel:`Add Partition`.
#. In the Configure Media menu, select :guilabel:`Add Partition`.
.. note:: To change an existing partition, navigate to the partition,
.. note:: To change an existing partition, navigate to the partition,
and select :guilabel:`Enter`.
boot partition
--------------
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
select the file system type. See Figure 9.
#. In :guilabel:`Mount Point`, enter the ``/boot`` partition.
#. In :guilabel:`Mount Point`, enter the ``/boot`` partition.
#. In :guilabel:`Size`, enter an appropriate size (e.g., 150M) per Table 1.
#. Navigate to :guilabel:`Confirm` until highlighted.
#. Navigate to :guilabel:`Confirm` until highlighted.
#. Select :guilabel:`Enter` to confirm.
#. Select :guilabel:`Enter` to confirm.
.. figure:: figures/bare-metal-install-beta-9.png
:scale: 100 %
:alt: boot partition
:alt: boot partition
Figure 9: :guilabel:`boot partition`
swap partition
--------------
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
select the file system type. See Figure 10.
#. In :guilabel:`Mount Point`, the field remains blank.
#. In :guilabel:`Mount Point`, the field remains blank.
.. note::
.. note::
The Mount Point field is disabled.
The Mount Point field is disabled.
#. In :guilabel:`Size`, enter an appropriate size (e.g., 256MB) per Table 1.
#. Navigate to :guilabel:`Add`.
#. Navigate to :guilabel:`Add`.
#. Select :guilabel:`Enter` to continue.
#. Select :guilabel:`Enter` to continue.
.. figure:: figures/bare-metal-install-beta-10.png
:scale: 100 %
:alt: swap partition
:alt: swap partition
Figure 10: :guilabel:`swap partition`
root partition
--------------
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
select the file system type. See Figure 11.
#. In :guilabel:`Mount Point`, enter ``/root``.
#. In :guilabel:`Size`, enter an appropriate size per Table 1.
.. note::
.. note::
The remaining available space shows by default.
The remaining available space shows by default.
#. Navigate to :guilabel:`Add` until highlighted.
#. Navigate to :guilabel:`Add` until highlighted.
#. Select :guilabel:`Enter` to continue.
#. Select :guilabel:`Enter` to continue.
You are returned to the :guilabel:`Configure media` menu.
You are returned to the :guilabel:`Configure media` menu.
#. Select :guilabel:`Confirm` to complete the media selection.
#. Select :guilabel:`Confirm` to complete the media selection.
.. figure:: figures/bare-metal-install-beta-11.png
:scale: 100 %
:alt: root partition
:alt: root partition
Figure 11: :guilabel:`root partition`
You are returned to the :guilabel:`Configure media` menu. Your partitions should appear similar to those shown in Figure 12.
You are returned to the :guilabel:`Configure media` menu. Your partitions should appear similar to those shown in Figure 12.
.. figure:: figures/bare-metal-install-beta-12.png
:scale: 100 %
@@ -396,20 +394,20 @@ root partition
#. Navigate to :guilabel:`Confirm` until highlighted.
#. Select :guilabel:`Enter` to complete adding partitions.
#. Select :guilabel:`Enter` to complete adding partitions.
.. _telemetry-tui:
.. _telemetry-tui:
Telemetry
=========
To fulfill the :guilabel:`Required options`, choose whether or not to
participate in `telemetry`_. :ref:`telemetrics` is a |CL| feature that
reports failures and crashes to the |CL| development team for improvements.
For more detailed information, see our :ref:`telemetry-about` page.
To fulfill the :guilabel:`Required options`, choose whether or not to
participate in `telemetry`_. :ref:`telemetrics` is a |CL| feature that
reports failures and crashes to the |CL| development team for improvements.
For more detailed information, visit our :ref:`telemetry-about` page.
#. In the main menu, navigate to :guilabel:`Telemetry` and select
:kbd:`Enter`.
#. In the Main Menu, navigate to :guilabel:`Telemetry` and select
:kbd:`Enter`.
#. Select :kbd:`Tab` to highlight your choice.
@@ -417,82 +415,86 @@ For more detailed information, see our :ref:`telemetry-about` page.
.. figure:: figures/bare-metal-install-beta-13.png
:scale: 100 %
:alt: Enable Telemetry
:alt: Enable Telemetry
Figure 13: :guilabel:`Enable Telemetry`
Figure 13: :guilabel:`Enable Telemetry`
Skip to finish installation
===========================
After selecting values for all :guilabel:`Required options`, you may skip directly to :ref:`finish-cl-installation`
After selecting values for all :guilabel:`Required options`, you may skip
directly to :ref:`finish-cl-installation`
If you wish to configure network interfaces or proxy settings,
or add bundles, or add/manage users, continue to the next section.
In the Main Menu, select the :guilabel:`Advanced options` if you wish to
configure network interfaces or proxy settings, add bundles, add/manage
users, and more. Continue to the next section.
.. _advanced-options-installation:
Advanced Options
****************
.. _configure-network-interfaces:
Configure Network Interfaces
Configure Network Interfaces
============================
By default, |CL| is configured to automatically detect the host network
interface using DHCP. However, if you want to use a static IP address or if
you do not have a DHCP server on your network, follow these instructions to
manually configure the network interface. Otherwise, default network
interface settings are automatically applied.
interface settings are automatically applied.
.. note::
If DHCP is available, no user selection may be required.
.. note::
If DHCP is available, no user selection may be required.
#. Navigate to :guilabel:`Configure Network Interfaces` and
select :kbd:`Enter`.
select :kbd:`Enter`.
#. Navigate to the network :guilabel:`interface` you wish to change.
#. Navigate to the network :guilabel:`interface` you wish to change.
#. When the desired :guilabel:`interface` is highlighted, select
#. When the desired :guilabel:`interface` is highlighted, select
:guilabel:`Enter` to edit.
.. note:: Multiple network interfaces may appear.
.. note:: Multiple network interfaces may appear.
.. figure:: figures/bare-metal-install-beta-14.png
:scale: 100 %
:alt: Configure Network Interfaces
Figure 14: :guilabel:`Configure Network Interfaces`
Figure 14: :guilabel:`Configure Network Interfaces`
#. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom).
#. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom).
Optional: Navigate to the checkbox :guilabel:`Automatic / dhcp` and select
:kbd:`Spacebar` to deselect.
:kbd:`Spacebar` to deselect.
.. figure:: figures/bare-metal-install-beta-15.png
:scale: 100 %
:alt: Network interface configuration
Figure 15: :guilabel:`Network interface configuration`
Figure 15: :guilabel:`Network interface configuration`
#. Navigate to the appropriate fields and assign the desired
network configuration.
network configuration.
#. To save settings, navigate to :guilabel:`Confirm` and select
#. To save settings, navigate to :guilabel:`Confirm` and select
:kbd:`Enter`.
.. note::
.. note::
To revert to previous settings, navigate to the :guilabel:`Cancel`
and select :kbd:`Enter`.
#. Upon confirming network configuration, the :guilabel:`Testing Networking`
dialogue appears. Assure the result shows success. If a failure occurs,
your changes will not be saved.
dialogue appears. Assure the result shows success. If a failure occurs,
your changes will not be saved.
#. Upon confirmation, you are returned to :guilabel:`Network interface`
settings.
settings.
#. Navigate to and select :guilabel:`Main Menu`.
#. Navigate to and select :guilabel:`Main Menu`.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
@@ -500,27 +502,27 @@ Proxy
=====
|CL| automatically attempts to detect proxy settings, as described in
`Autoproxy`_. If you need to manually assign proxy settings, follow this
instruction.
`Autoproxy`_. If you need to manually assign proxy settings, follow this
instruction.
#. From the Advanced options menu, navigate to :guilabel:`Proxy`, and
#. From the Advanced options menu, navigate to :guilabel:`Proxy`, and
select :kbd:`Enter`.
#. Navigate to the field :guilabel:`HTTPS Proxy`.
#. Navigate to the field :guilabel:`HTTPS Proxy`.
.. figure:: figures/bare-metal-install-beta-16.png
:scale: 100 %
:alt: Configure the network proxy
Figure 16: :guilabel:`Configure the network proxy`
Figure 16: :guilabel:`Configure the network proxy`
#. Enter the desired proxy address and port using conventional syntax,
such as: http://address:port.
such as: \http://address:port.
#. Navigate to :guilabel:`Confirm` and select :kbd:`Enter`.
#. To revert to previous settings, navigate to :guilabel:`Cancel`
and select :guilabel:`Cancel`.
#. To revert to previous settings, navigate to :guilabel:`Cancel`
and select :guilabel:`Cancel`.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
@@ -529,43 +531,44 @@ Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Test Network Settings
=====================
To manually assure network connectivity before installing |CL|,
To manually assure network connectivity before installing |CL|,
select :guilabel:`Test Network Settings` and select :guilabel:`Enter`.
A progress bar appears as shown in Figure 17.
A progress bar appears as shown in Figure 17.
.. figure:: figures/bare-metal-install-beta-17.png
:scale: 100 %
:alt: Testing Networking dialogue
Figure 17: :guilabel:`Testing Networking dialogue`
Figure 17: :guilabel:`Testing Networking dialogue`
.. note::
Any changes made to network settings are automatically tested
during configuration.
.. note::
Any changes made to network settings are automatically tested
during configuration.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Bundle Selection
================
#. On the Advanced menu, select :guilabel:`Bundle selection`
#. On the Advanced menu, select :guilabel:`Bundle Selection`
#. Navigate to the desired bundle using :kbd:`Tab` or :kbd:`Up/Down` arrows.
#. Select :kbd:`Spacebar` to select the checkbox for each desired bundle.
#. Select :kbd:`Spacebar` to select the checkbox for each desired bundle.
.. figure:: figures/bare-metal-install-beta-18.png
:scale: 100 %
:alt: Bundle selection
:alt: Bundle Selection
Figure 18: :guilabel:`Bundle selection`
Figure 18: :guilabel:`Bundle Selection`
..TODO:Add new screenshot that shows more bundles.
#. Select :kbd:`Confirm` or :kbd:`Cancel`.
You are returned to the :guilabel:`Advanced options` menu.
You are returned to the :guilabel:`Advanced options` menu.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
@@ -575,199 +578,202 @@ User Manager
Add New User
------------
#. In Advanced Options, select :guilabel:`User Manager`.
#. In Advanced Options, select :guilabel:`User Manager`.
#. Select :guilabel:`Add New User` as shown in Figure 19.
#. Select :guilabel:`Add New User` as shown in Figure 19.
.. figure:: figures/bare-metal-install-beta-19.png
:scale: 100 %
:alt: Add New User, User Name
:alt: Add New User, User Name
Figure 19: :guilabel:`Add New User`
Figure 19: :guilabel:`Add New User`
#. Optional: Enter a :guilabel:`User Name`.
.. note:
.. note:
The User Name must be alphanumeric and can include spaces, commas, or hyphens. Maximum length is 64 characters.
The User Name must be alphanumeric and can include spaces, commas, or hyphens. Maximum length is 64 characters.
.. figure:: figures/bare-metal-install-beta-20.png
:scale: 100 %
:alt: User Name
:alt: User Name
Figure 20: :guilabel:`User Name`
Figure 20: :guilabel:`User Name`
#. Enter a :guilabel:`Login`.
.. note::
.. note::
The User Login must be alphanumeric and can include hyphens and underscores. Maximum length is 31 characters.
The User Login must be alphanumeric and can include hyphens and underscores. Maximum length is 31 characters.
#. Enter a :guilabel:`Password`.
.. note:
.. note:
Minimum length is 8 characters. Maximum length is 255 characters.
#. In :guilabel:`Retype`, enter the same password.
#. In :guilabel:`Retype`, enter the same password.
#. Optional: Navigate to the :guilabel:`Administrative` checkbox and select
:kbd:`Spacebar` to assign administrative rights to the user.
#. Optional: Navigate to the :guilabel:`Administrative` checkbox and select
:kbd:`Spacebar` to assign administrative rights to the user.
.. note::
.. note::
Selecting this option enables sudo privileges for the user.
Selecting this option enables sudo privileges for the user.
.. figure:: figures/bare-metal-install-beta-21.png
:scale: 100 %
:alt: Administrative rights
Figure 21: :guilabel:`Administrative rights`
#. Select :kbd:`Confirm`.
Figure 21: :guilabel:`Administrative rights`
#. Select :kbd:`Confirm`.
.. note::
If desired, select :guilabel:`Reset` to reset the form.
If desired, select :guilabel:`Reset` to reset the form.
#. In :guilabel:`User Manager`, navigate to :guilabel:`Confirm`.
#. With :guilabel:`Confirm` highlighted, select :kbd:`Enter`.
Modify / Delete User
--------------------
#. In User Manager, select :guilabel:`Tab` to highlight the user, as shown
in Figure 22.
#. In User Manager, select :guilabel:`Tab` to highlight the user, as shown
in Figure 22.
#. Select :kbd:`Enter` to modify the user.
#. Select :kbd:`Enter` to modify the user.
.. figure:: figures/bare-metal-install-beta-22.png
:scale: 100 %
:alt: Modify User
Figure 22: :guilabel:`Modify User`
Figure 22: :guilabel:`Modify User`
#. Modify user details as desired.
#. Modify user details as desired.
Optional: To delete the user, navigate to the :guilabel:`Delete`
button and select :kbd:`Enter`
Optional: To delete the user, navigate to the :guilabel:`Delete`
button and select :kbd:`Enter`
#. Navigate to :kbd:`Confirm` until highlighted.
#. Navigate to :kbd:`Confirm` until highlighted.
.. note::
Optional: Select :guilabel:`Reset` to rest the form.
Optional: Select :guilabel:`Reset` to rest the form.
#. Select :guilabel:`Confirm` to save the changes you made.
#. Select :guilabel:`Confirm` to save the changes you made.
You are returned to the User Manager menu.
You are returned to the User Manager menu.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Kernel Command Line
Kernel Command Line
===================
For more advanced users, |CL| provides the ability to add, or remove, kernel
arguments. If you want to append a new argument, enter the argument here.
This argument will be used every time you install a new kernel or update a
new kernel.
For advanced users, |CL| provides the ability to add or remove kernel
arguments. If you want to append a new argument, enter the argument here.
This argument will be used every time you install or update a
new kernel.
#. In Advanced Options, select :guilabel:`Tab` to highlight
:guilabel:`Kernel Command Line`.
#. In Advanced Options, select :guilabel:`Tab` to highlight
:guilabel:`Kernel Command Line`.
#. Select :kbd:`Enter`.
#. Select :kbd:`Enter`.
#. To add or remove arguments, choose one of the following options.
#. To add or remove arguments, choose one of the following options.
.. figure:: figures/bare-metal-install-beta-23.png
:scale: 100 %
:alt: kernel command line
Figure 23: :guilabel:`kernel command line`
Figure 23: :guilabel:`kernel command line`
#. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`.
#. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`.
#. To remove an existing argument, enter the argument in
:guilabel:`Remove Arguments`.
#. To remove an existing argument, enter the argument in
:guilabel:`Remove Arguments`.
#. Select :kbd:`Confirm`.
#. Select :kbd:`Confirm`.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Kernel Selection
Kernel Selection
================
#. Select a kernel option. By default, the latest kernel release is
selected.
selected.
.. note::
.. note::
To choose a different kernel, such as Long Term Support (LTS), shown in Figure 24, navigate to it using :guilabel:`Tab`.
To choose a different kernel, such as Long Term Support (LTS), shown in Figure 24, navigate to it using :guilabel:`Tab`.
.. figure:: figures/bare-metal-install-beta-24.png
:scale: 100 %
:alt: Kernel selection
Figure 24: :guilabel:`Kernel selection`
Figure 24: :guilabel:`Kernel selection`
#. Select :kbd:`Spacebar` to select the desired option.
#. Select :kbd:`Confirm`.
#. Select :kbd:`Spacebar` to select the desired option.
#. Select :kbd:`Confirm`.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Swupd Mirror
============
If you have your own custom mirror of |CL|, you may want to add that URL.
If you have your own custom mirror of |CL|, you can add its URL.
#. In Advanced Options, select :guilabel:`Swupd Mirror`.
#. In Advanced Options, select :guilabel:`Swupd Mirror`.
#. To add a local swupd mirror, enter a valid URL in :guilabel:`Mirror URL:`
#. Select :kbd:`Confirm`.
#. Select :kbd:`Confirm`.
.. figure:: figures/bare-metal-install-beta-25.png
:scale: 100 %
:alt: Swupd Mirror
Figure 25: :guilabel:`Swupd Mirror`
Figure 25: :guilabel:`Swupd Mirror`
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Assign Hostname
===============
#. In Advanced Options, select :guilabel:`Assign Hostname`.
#. In Advanced Options, select :guilabel:`Assign Hostname`.
#. In :guilabel:`Hostname`, enter the hostname only (excluding the domain).
.. note::
Hostname does not allow empty spaces. Hostname must start with an
alphanumeric character but may also contain hyphens. Maximum length of
63 characters.
alphanumeric character but may also contain hyphens. Maximum length of
63 characters.
.. figure:: figures/bare-metal-install-beta-26.png
:scale: 100 %
:alt: Assign Hostname
Figure 26: :guilabel:`Assign Hostname`
Figure 26: :guilabel:`Assign Hostname`
#. Navigate to :kbd:`Confirm` until highlighted.
#. Navigate to :kbd:`Confirm` until highlighted.
#. Select :kbd:`Confirm`.
#. Select :kbd:`Confirm`.
Optional: :ref:`Skip to finish installation <finish-cl-installation>`.
Automatic OS Updates
====================
In the rare case that you need to disable automatic software updates,
In the rare case that you need to disable automatic software updates,
follow the onscreen instructions.
#. In Advanced Options, select :guilabel:`Automatic OS Updates`.
#. In Advanced Options, select :guilabel:`Automatic OS Updates`.
#. Select the desired option.
#. Select the desired option.
.. figure:: figures/bare-metal-install-beta-27.png
:scale: 100 %
@@ -775,7 +781,7 @@ follow the onscreen instructions.
Figure 27: :guilabel:`Automatic OS Updates`
You are returned to the :guilabel:`Main Menu`.
You are returned to the :guilabel:`Main Menu`.
.. _finish-cl-installation:
@@ -783,16 +789,16 @@ Finish installation
*******************
#. When you are satisfied with your installation configuration, navigate to
:guilabel:`Install` and select :kbd:`Enter`.
:guilabel:`Install` and select :kbd:`Enter`.
.. note::
.. note::
Whenever installation is finished, a ``reboot`` button appears.
Whenever installation is finished, a ``reboot`` button appears.
#. Select ``reboot``.
#. Select ``reboot``.
#. When the system reboots, remove any installation media present.
**Congratulations!**
You have successfully installed |CL| on bare metal using the new installer.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 519 KiB

After

Width:  |  Height:  |  Size: 537 KiB

@@ -1,4 +1,4 @@
.. _deploy-at-scale.rst:
.. _deploy-at-scale:
Deploy at Scale
###############
@@ -0,0 +1,71 @@
.. _assign-static-ip:
Assign a static IP address to a network interface
#################################################
Introduction
************
By default, your |CL-ATTR| system automatically gets an IP address from your
network via DHCP. If you do not have a DHCP server on your network or simply
want to use a static IP address, follow the steps in this guide.
Process
*******
#. Create this directory structure:
.. code-block:: bash
sudo mkdir -p /etc/systemd/network
#. Identify the interface to be assigned the static IP address:
.. code-block:: bash
ip addr
The system returns the following:
.. code-block:: console
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: wlp1s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 4a:98:8d:e5:43:15 brd ff:ff:ff:ff:ff:ff
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
link/ether f4:4d:30:68:96:20 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.2/24 brd 10.54.74.255 scope global dynamic eno1
valid_lft 6766sec preferred_lft 6766sec
inet6 fe80::f64d:30ff:fe68:9620/64 scope link
valid_lft forever preferred_lft forever
In this example, we will use the `eno1` interface.
#. Create the :file:`70-static.network` file and add the following:
.. code-block:: bash
sudo $EDITOR /etc/systemd/network/70-static.network
[Match]
Name=[interface name]
[Network]
Address=[IP address]/24
DHCP=yes # to get DNS info, etc.
Replace [interface name] and [IP address] with your specific settings.
#. Restart the networkd service:
.. code-block:: bash
sudo systemctl restart systemd-networkd
**Congratulations!** You have successfully assigned a static IP address.
@@ -0,0 +1,18 @@
.. _autoproxy:
Autoproxy
#########
.. rst-class:: content-collapse
About
=====
.. include:: autoproxy_about.txt
.. rst-class:: content-collapse
Guide
=====
.. include:: autoproxy_guide.txt
@@ -0,0 +1,43 @@
The |CL-ATTR| is the first Linux distribution to support autoproxy. The OS
can discover a Proxy Auto-Config (PAC) script and use it to automatically
resolve the proxy needed for a given connection. With Autoproxy, you can use
|CL| inside any proxy environment without having to manually
configure the proxies.
Corporate and private networks can be very complex, needing to restrict and
control network connections for security reasons. The typical side effects
are limited or blocked connectivity and requiring manual configuration of
proxies to perform the most mundane tasks such as cloning a repo or checking
for updates. With Clear Linux, all of the work is done behind the scenes to
effortlessly use your network and have connections “just work”.
This feature removes massive complications in network connectivity due to
proxy issues. You can automate tasks like unit testing without worrying
about the proxy not being set and you can remove unset proxies from the
equation when dealing with network unavailability across systems.
How Autoproxy works
-------------------
We designed autoproxy around general tools provided by nearly any Linux
distribution with a few minor additions and modifications. We leveraged the
DHCP and network information provided from systemd and created a
PAC-discovery daemon. The daemon uses the information to resolve a URL for a
PAC file. The daemon then passes the URL into PACrunner*. PACrunner
downloads the PAC file and uses the newly implemented Duktape* engine to
parse it.
.. figure:: figures/autoproxy_0.png
:width: 400px
Figure 1: Autoproxy Flow
From that point on, any cURL* or network requests query PACrunner for the
correct proxy to use. We modified the cURL library to communicate with
PACrunner over DBus. However, cURL will ignore PACrunner and run normally if
no PAC file is loaded or if you set any proxies manually. Thus, your
environment settings are respected and no time is wasted trying to resolve a
proxy.
More importantly: all these steps happen in the background, very quickly, and
with no user interaction.
@@ -0,0 +1,77 @@
Autoproxy allows |CL| to operate seamlessly behind a proxy
because, :ref:`swupd-guide` and other |CL| tools are implemented on
top of libcurl. Tools that do not use libcurl, like git, must
be configured independently.
If you encounter problems with autoproxy functioning, use
:command:`pacdiscovery` and :command:`FindProxyForURL` to
help troubleshoot assuming a familiarity with PAC files and WPAD.
.. note::
Learn more about WPAD, PAC files, and PAC functions at `findproxyforurl`_.
.. _findproxyforurl: http://findproxyforurl.com/
Running :command:`pacdiscovery` with no arguments will immediately indicate
1. if there is a problem resolving the :command:`WPAD` host name resolution:
.. code:: console
$ pacdiscovery
failed getaddrinfo: No address associated with hostname
Unable to find wpad host
2. or if the :command:`pacrunner` service is disabled (masked).
.. code:: console
$ pacdiscovery
PAC url: http://autoproxy.your.domain.com/wpad.dat
Failed to create proxy config: Unit pacrunner.service is masked.
Unmask the :command:`pacrunner` service by running:
.. code:: console
$ systemctl unmask pacrunner.service
:command:`FindProxyForURL` with :command:`busctl` can also indicate if the
:command:`pacrunner.service` is masked.
.. code:: console
$ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com"
Unit pacrunner.service is masked.
dig wpad, dig wpad.<domain>
:command:`FindProxyForURL` returns the URL and port of the proxy server when
an external URL and host are provided as arguments.
.. code:: console
$ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com"
s "PROXY proxy.your.domain.com:<port>"
If a proxy server is not avialable, or if :command:`pacrunner` is running
without a PAC file, :command:`FindProxyForURL` will return "DIRECT".
.. code:: console
$ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com"
s "DIRECT"
Once :command:`pacdiscovery` is able to look up :command:`WPAD`, restart the
:command:`pacrunner` service:
.. code:: console
$ systemctl stop pacrunner
$ systemctl restart pacdiscovery
.. note::
A "domain" or "search" entry in :file:`/etc/resolv.conf` is required for short
name lookups to resolve. The :file:`resolv.conf` man page has additional
details.
Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

@@ -0,0 +1,176 @@
.. _kernel-modules:
Add kernel modules
##################
Kernel modules are additional pieces of software capable of being inserted
into the Linux kernel to add functionality, such as a hardware driver.
Kernel modules may already be part of the Linux source tree (in-tree) or may
come from an external source, such as directly from a vendor (out-of-tree).
In cases where drivers beyond those enabled by default in |CL-ATTR| are
needed it may be necessary to:
.. contents:: :local:
:depth: 1
:backlinks: top
Check if the module is available through |CL|
=============================================
Using an existing module is significantly easier to maintain and retains
signature verification of the |CL| kernel. For more information on |CL|
security practices, see the :ref:`security` page.
|CL| comes with many upstream kernel modules available for use. If
you require a kernel module, be sure to check whether it is already available in |CL| first.
You can search for kernel module file names, which end with the :file:`.ko`
file extension, using the :command:`swupd search` command. For example:
:command:`sudo swupd search ${module_name}.ko`.
See :ref:`swupd-search` for more information.
Request the module be added to |CL|
===================================
If the kernel module you need is already open source
(e.g. in the Linux upstream) and likely to be useful to others,
consider submitting a request to add or enable in the |CL| kernel.
Make enhancement requests to the |CL| distribution `on GitHub`_ .
Build and load an out-of-tree module
====================================
In some cases you may need an out-of-tree kernel module that is not
available through |CL|.
You can build and load out-of-tree kernel modules, however you must:
* disable secure boot
* disable kernel module integrity checking
* build the module against new versions of the Linux kernel
.. note::
Any time the kernel is upgraded on your Clear Linux system, you will
need to rebuild your out-of-tree modules.
This approach works well for individual development or testing.
For a more scalable and customizable approach, consider using the
`mixer tool`_ to provide a custom kernel and updates.
Build kernel module
-------------------
#. From a |CL| system, ensure you are running the *native* kernel.
Currently only the native kernel is enabled to build and load
out-of-tree modules.
.. code-block:: bash
$ uname -r
4.XX.YY-ZZZZ.native
Ensure *.native* is in the kernel name
#. Install the `linux-dev` bundle to obtain the kernel headers, which are
required for compiling kernel modules.
.. code-block:: bash
sudo swupd bundle-add linux-dev
#. Follow instructions from the kernel module source code to compile the
kernel module.
Load kernel module
------------------
#. Disable Secure Boot in your system's UEFI settings, if you have enabled
it. The loading of new out-of-tree modules modifies the signatures Secure
Boot relies on for trust.
#. Disable signature checking for the kernel by modifying the kernel boot
parameters and reboot the system.
All kernel modules from |CL| have been signed to enforce kernel security.
However, out-of-tree modules break this chain of trust so this mechanism
needs to be disabled.
.. code-block:: bash
sudo mkdir -p /etc/kernel/cmdline.d
echo "module.sig_unenforce" | sudo tee /etc/kernel/cmdline.d/allow-unsigned-modules.conf
#. Update the boot manager and reboot the system to implement the changed
kernel parameters.
.. code-block:: bash
sudo clr-boot-manager update
sudo reboot
.. note::
:command:`clr-boot-manager update` does not return any
console output if successful.
#. After rebooting, out-of-tree modules can be manually loaded with
:command:`insmod`.
.. code-block:: bash
sudo insmod ${path_to_module}
Optional: Use `modprobe` to specify module options and aliases
--------------------------------------------------------------
Use :command:`modprobe` to load a module and set options.
Because :command:`modprobe` can add or remove more than one module, due to
modules having dependencies, a method of specifying what options are
to be used with individual modules is useful. This can be done with
configuration files under the :file:`/etc/modprobe.d` directory.
.. code-block:: bash
sudo mkdir /etc/modprobe.d
All files underneath the :file:`/etc/modprobe.d` directory
that end with the :file:`.conf` extension specify module options to use when
loading. This can also be used to create convenient aliases for modules or
they can override the normal loading behavior altogether for those with
special requirements.
You can find more info on module loading in the modprobe.d manual page:
.. code-block:: bash
man modprobe.d
Optional: Configure kernel modules to load at boot
--------------------------------------------------
Use the :file:`/etc/modules-load.d` configuration directory to
specify kernel modules to load automatically at boot.
.. code-block:: bash
sudo mkdir /etc/modules-load.d
All files underneath the :file:`/etc/modules-load.d` directory
that end with the :file:`.conf` extension contain a list of module names
of aliases (one per line) to load at boot.
You can find more info on module loading in the modules-load.d manual page:
.. code-block:: bash
man modules-load.d
.. _`on GitHub`: https://github.com/clearlinux/distribution
.. _`mixer tool`: https://clearlinux.org/features/mixer-tool
@@ -10,11 +10,13 @@ completed.
.. toctree::
:maxdepth: 2
autoproxy
developer-workstation
swupd-search
enable-user-space
swupd-guide
bulk-provision
kernel-modules
mixer
mixin
validate-signatures
@@ -25,3 +27,4 @@ completed.
download-verify-decompress-mac
download-verify-decompress-windows
autospec
assign-static-ip
@@ -57,7 +57,14 @@ used for illustrative purposes. You may use any image of |CL| you choose.
.. code-block:: console
openssl smime -verify -in clear-$(curl https://download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig -inform der -content sha512sum.out -CAfile ClearLinuxRoot.pem
openssl smime -verify -purpose any -in clear-$(curl https://download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig -inform der -content sha512sum.out -CAfile ClearLinuxRoot.pem
.. note::
The `-purpose any` option is required when using OpenSSL 1.1. If using
an earlier version of OpenSSL, omit this option to perform signature
validation. The `openssl version` command may be used to determine the
version of OpenSSL in use.
#. The output should contain ``Verification successful``. If the output
contains ``bad_signature`` anywhere, then the image is not trustworthy.
@@ -101,7 +108,14 @@ these steps manually when performing a ``swupd update``.
.. code-block:: console
openssl smime -verify -in Manifest.MoM.sig -inform der -content Manifest.MoM -CAfile Swupd_Root.pem
openssl smime -verify -purpose any -in Manifest.MoM.sig -inform der -content Manifest.MoM -CAfile Swupd_Root.pem
.. note::
The `-purpose any` option is required when using OpenSSL 1.1. If using
an earlier version of OpenSSL, omit this option to perform signature
validation. The `openssl version` command may be used to determine the
version of OpenSSL in use.
.. note::
+235
View File
@@ -0,0 +1,235 @@
.. _dlrs:
Deep Learning Reference Stack
#############################
This tutorial shows you how to run benchmarking workloads in |CL-ATTR| using
TensorFlow\* and Kubeflow with the Deep Learning Reference Stack.
The Deep Learning Reference Stack is available in two versions.
The first is `Eigen`_, which includes `TensorFlow`_ optimized for Intel®
architecture. The second is `Intel MKL-DNN`_, which includes the TensorFlow
framework optimized using Intel® Math Kernel Library for Deep Neural
Networks (Intel® MKL-DNN) primitives.
.. contents:: :local:
:depth: 1
Release notes
=============
View current `release notes`_ for the Deep Learning Reference Stack.
View current `benchmark results`_ for the Deep Learning Reference Stack.
.. note::
Performance test numbers in the Deep Learning Reference Stack were obtained using `runc` as the runtime.
Prerequisites
=============
* |CL| installed on host system. If not installed, :ref:`bare-metal-install`
* `containers-basic` bundle
* `cloud-native-basic` bundle
In |CL|, `containers-basic` provides Docker\*, which is required for
TensorFlow benchmarking. Use the :command:`swupd` utility to check if
`containers-basic` and `cloud-native-basic` are present:
.. code-block:: bash
sudo swupd bundle-list
If you need to install the `containers-basic` or `cloud-native-basic`, enter:
.. code-block:: bash
sudo swupd bundle-add containers-basic cloud-native-basic
To ensure that kubernetes is correctly installed and configured,
:ref:`kubernetes`.
We have validated these steps against the following software package
versions:
* |CL| 26240--lowest version permissible.
* Docker 18.06.1
* Kubernetes 1.11.3
* Go 1.11.12
TensorFlow single and multi-node benchmarks
============================================
This section describes running the `TensorFlow benchmarks`_ in single node.
For multi-node testing, replicate these steps for each node. These steps
provide a template to run other benchmarks, provided that they can invoke
TensorFlow.
#. Download and run either the `Eigen`_ or the `Intel MKL-DNN`_ docker image
from `Docker Hub`_.
.. note::
You will enter the following commands in the running container.
Replace <docker_name> with the name of the image.
#. Clone the benchmark repository:
.. code-block:: bash
docker exec -t <docker_name> bash -c ‘git clone http://github.com/tensorflow/benchmarks -b cnn_tf_v1.12_compatible’
#. Next, execute the benchmark script to run the benchmark.
.. code-block:: bash
docker exec -i <docker_name> bash -c ‘python benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC ’.
.. note::
You can replace the model with one of your choice supported by the
TensorFlow benchmarks.
Kubeflow multi-node benchmarks
==============================
The benchmark workload will run in a Kubernetes cluster. We will use
`Kubeflow`_ for the Machine Learning workload deployment on three nodes.
Kubernetes setup
****************
Follow the instructions in the :ref:`kubernetes` tutorial to get set up on
|CL|. The kubernetes community also has
`instructions for creating a cluster`_.
Kubernetes networking
*********************
We used `flannel`_ as the network provider for these tests. If you are
comfortable with another network layer, refer to the Kubernetes
`networking documentation`_ for setup.
Images
******
We need to add `launcher.py` to our docker image to include the Deep
Learning Reference Stack and put the benchmarks repo in the correct
location. From the docker image, run the following:
.. code-block:: bash
mkdir -p /opt
git clone https://github.com/tensorflow/benchmarks.git /opt/tf-benchmarks
cp launcher.py /opt
chmod u+x /opt/*
Your entry point now becomes "/opt/launcher.py".
This will build an image which can be consumed directly by TFJob from
kubeflow. We are working to create these images as part of our release
cycle.
ksonnet\*
*********
Kubeflow uses ksonnet* to manage deployments, so we need to install that before setting up Kubeflow. On |CL|, follow these steps:
.. code-block:: bash
swupd bundle-add go-basic-dev
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
go get github.com/ksonnet/ksonnet
cd $GOPATH/src/github.com/ksonnet/ksonnet
make install
After the ksonnet installation is complete, ensure that binary `ks` is
accessible across the environment.
Kubeflow
********
Once you have Kubernetes running on your nodes, you can setup `Kubeflow`_ by
following these instructions from their `quick start guide`_.
.. code-block:: bash
export KUBEFLOW_SRC=$HOME/kflow
export KUBEFLOW_TAG=”v0.3.2”
export KFAPP=”kflow_app”
export K8S_NAMESPACE=”kubeflow”
mkdir ${KUBEFLOW_SRC}
cd ${KUBEFLOW_SRC}
ks init ${KFAPP}
cd ${KFAPP}
ks registry add kubeflow github.com/kubeflow/kubeflow/tree/${KUBEFLOW_TAG}/kubeflow
ks pkg install kubeflow/core
Now you have all the required kubeflow packages, and you can deploy the primary one for our purposes: tf-job-operator.
.. code-block:: bash
ks env rm default
kubectl create namespace ${K8S_NAMESPACE}
ks env add default --namespace "${K8S_NAMESPACE}"
ks generate tf-job-operator tf-job-operator
ks apply default -c tf-job-operator
This creates the CustomResourceDefinition(CRD) endpoint to launch a TFJob.
Run a TFJob
===========
#. Select this link for the `ksonnet registries for deploying TFJobs`_.
#. Install the TFJob componets as follows:
.. code-block:: bash
ks registry add dlrs-tfjob github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
ks pkg install dlrs-tfjob/dlrs-bench
#. Next, generate Kubernetes manifests for the workloads and apply them to
create and run them using these commands
.. code-block:: bash
ks generate dlrs-resnet50 dlaasresnet50 --name=dlrsresnet50
ks generate dlrs-alexnet dlaasalexnet --name=dlrsalexnet
ks apply default -c dlrsresnet50
ks apply default -c dlrsalexnet
This will replicate and deploy three test setups in your Kubernetes cluster.
Results of Running this Tutorial
================================
You need to parse the logs of the Kubernetes pod to get the performance
numbers. The pods will still be around post completion and will be in
‘Completed’ state. You can get the logs from any of the pods to inspect the
benchmark results. More information about `Kubernetes logging`_ is available from the Kubernetes community.
.. _TensorFlow: https://www.tensorflow.org/
.. _Kubeflow: https://www.kubeflow.org/
.. _Docker Hub: https://hub.docker.com/
.. _TensorFlow benchmarks: https://www.tensorflow.org/guide/performance/benchmarks
.. _instructions for creating a cluster: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
.. _flannel: https://github.com/coreos/flannel
.. _networking documentation: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network
.. _quick start guide: https://www.kubeflow.org/docs/started/getting-started/
.. _Eigen: https://hub.docker.com/r/clearlinux/stacks-dlaas-oss/
.. _Intel MKL-DNN: https://hub.docker.com/r/clearlinux/stacks-dlaas-mkl/
.. _release notes: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs
.. _ksonnet registries for deploying TFJobs: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
.. _Kubernetes logging: https://kubernetes.io/docs/concepts/cluster-administration/logging/
.. _benchmark results: https://clearlinux.org/stacks/deep-learning-reference-stack
+49 -91
View File
@@ -4,23 +4,19 @@ Run Kubernetes\* on |CL-ATTR|
#############################
This tutorial describes how to install, configure, and run the
`Kubernetes container orchestration system`_ on |CL-ATTR| using different
container engines and runtimes.
`Kubernetes container orchestration system`_ on |CL-ATTR| using CRI+O and
kata-runtime.
Kubernetes\* is an open source system for automating deployment, scaling, and
management of containerized applications. It groups containers that make up
an application into logical units for easy management and discovery.
Runc and Kata Containers\* kata-runtime adhere to :abbr:`OCI (Open Container Initiative*)`
guidelines and work seamlessly with Kubernetes. `Kata Containers`_ provide
strong isolation for untrusted workloads or multi-tenant scenarios. Runc and
Kata Containers can be allocated on a per-pod basis so you can mix and match
both on the same host to suit your needs.
This tutorial describes the following combinations:
* Kubernetes with Docker and runc
* Kubernetes with CRI-O and kata-runtime
Kata Containers\* kata-runtime adheres to
:abbr:`OCI (Open Container Initiative*)` guidelines and work seamlessly with
Kubernetes. `Kata Containers`_ provide strong isolation for untrusted
workloads or multi-tenant scenarios. Kata Containers can be
allocated on a per-pod basis so you can mix and match both on the same host
to suit your needs.
Prerequisites
*************
@@ -40,13 +36,12 @@ Before you install any new packages, update |CL| with the following command:
Install Kubernetes and CRI runtimes
***********************************
Kubernetes and a set of supported :abbr:`CRI (Container Runtime Interface)`
runtimes are included in the `cloud-native-basic`_ bundle. To install the
framework, enter the following command:
Kubernetes, a set of supported :abbr:`CRI (Container Runtime Interface)`
runtimes, and networking plugins, are included in the `cloud-native-basic`_ and `containers-basic`_ bundles. To install this framework, enter the following command:
.. code-block:: bash
sudo swupd bundle-add cloud-native-basic
sudo swupd bundle-add cloud-native-basic containers-basic
Configure Kubernetes
********************
@@ -105,7 +100,7 @@ deployment and your security needs.
On systems with limited resources, some performance degradation may
be observed while swap is disabled.
#. Switch to root to modify `hostname`:
#. Switch to root to modify the `hosts` file:
.. code-block:: bash
@@ -127,49 +122,7 @@ deployment and your security needs.
Configure and run Kubernetes
****************************
This section describes how to configure and run Kubernetes with:
* Docker and runc
* CRI-O and kata-runtime
Configure and run Docker + runc
===============================
#. Enable the Docker service:
.. code-block:: bash
sudo systemctl enable docker.service
#. Create (or edit if it exists) the file
:file:`/etc/systemd/system/docker.service.d/51-runtime.conf` and include the following lines:
.. code-block:: bash
[Service]
Environment="DOCKER_DEFAULT_RUNTIME=--default-runtime runc"
#. Create (or edit if it exists) the file :file:`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and include the following lines:
.. code-block:: bash
[Service]
Environment="KUBELET_EXTRA_ARGS="
#. Enter the commands:
.. code-block:: bash
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl restart kubelet
#. Initialize the master control plane with the command:
.. code-block:: bash
sudo kubeadm init --ignore-preflight-errors=SystemVerification
This section describes how to configure and run Kubernetes with CRI-O and kata-runtime.
Configure and run CRI-O + kata-runtime
======================================
@@ -211,45 +164,52 @@ If you choose the `flannel` add-on, then you must add the following to the
--pod-network-cidr 10.244.0.0/16
If you are using CRI-O and `flannel` and you want to use Kata Containers, edit
the :file:`/etc/crio/crio.conf` file to add:
If you are using CRI-O and `flannel` and you want to use Kata Containers, edit the :file:`/etc/crio/crio.conf` file to add:
.. code-block:: bash
[crio.runtime]
manage_network_ns_lifecycle = true
**Notes about Weave Net add-on**
.. TODO: Should Weave and Calico be mentioned below, or is Flannel enought?
If you choose the `Weave Net` add-on, then you must make the following changes
because it installs itself in the :file:`/opt/cni/bin` directory.
Create a symlink for the network overlays:
If you are using Docker and `Weave Net`, edit the :file:`kubeadm.conf` file to
add:
.. code-block:: bash
.. code-block:: bash
sudo ln -s /usr/libexec/cni /opt/cni/bin
Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
.. note::
If you are using CRI-O and `Weave Net`, you must complete the following steps.
|CL| installs CNI plugins that are part of the `cloud-native-basic` bundle to :file:`/usr/libexec/cni`. The directory is required because `swupd verify` uses it, if necessary, to repair a system to a known good state.
#. Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from:
.. TODO: Verify whether to omit this section per mythi, eadamsintel.
.. code-block:: bash
.. **Notes about Weave Net add-on**
plugin_dir = "/usr/libexec/cni/"
.. If you choose the `Weave Net` add-on, then you must make the following
.. changes because it installs itself in the :file:`/opt/cni/bin` directory.
to:
.. For using CRI-O and `Weave Net`, you must complete the following
.. steps.
.. code-block:: bash
.. #. Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from:
plugin_dir = "/opt/cni/bin"
.. .. code-block:: bash
#. Add the `loopback` CNI plugin to the plugin path with the command:
.. plugin_dir = "/usr/libexec/cni/"
.. code-block:: bash
.. to:
sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback
.. .. code-block:: bash
.. plugin_dir = "/opt/cni/bin"
.. #. Add the `loopback` CNI plugin to the plugin path with the command:
.. .. code-block:: bash
.. sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback
Use your cluster
@@ -257,8 +217,8 @@ Use your cluster
Once your master control plane is successfully initialized, instructions on
how to use your cluster and its *IP*, *token*, and *hash* values are
displayed. It is important that you record the cluster values because they are
needed when joining worker nodes to the cluster. Some values have a valid
displayed. It is important that you record the cluster values because they
are needed when joining worker nodes to the cluster. Some values have a valid
period. The values are presented in a format similar to:
.. code-block:: bash
@@ -268,10 +228,10 @@ period. The values are presented in a format similar to:
**Congratulations!**
You've successfully installed and set up Kubernetes in |CL| using Docker and
runc or CRI-O and kata-runtime. You are now ready to follow on-screen
instructions to deploy a pod network to the cluster and join worker nodes
with the displayed token and IP information.
You've successfully installed and set up Kubernetes in |CL| using CRI-O and
kata-runtime. You are now ready to follow on-screen instructions to deploy a
pod network to the cluster and join worker nodes with the displayed token
and IP information.
Related topics
**************
@@ -314,8 +274,7 @@ Proxy configuration (optional)
******************************
If you use a proxy server, you must set your proxy environment variables and
create an appropriate proxy configuration file for both CRI-O and Docker
services. Consult your IT department if you are behind a corporate proxy for
create an appropriate proxy configuration file for both CRI-O services. Consult your IT department if you are behind a corporate proxy for
the appropriate values. Ensure that your local IP is **explicitly included**
in the environment variable *NO_PROXY*. (Setting *localhost* is not enough.)
@@ -324,7 +283,7 @@ commands as a shell script to configure all of these services in one step:
.. code-block:: bash
services=('crio' 'docker')
services=('crio')
for s in "${services[@]}"; do
sudo mkdir -p "/etc/systemd/system/${s}.service.d/"
cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf"
@@ -383,9 +342,6 @@ Troubleshooting
.. code-block:: bash
/* Kubernetes with Docker + runc */
sudo -E kubeadm init --ignore-preflight-errors=SystemVerification
/* Kubernetes with CRI-O + kata-runtime */
sudo -E kubeadm init --cri-socket=/run/crio/crio.sock
@@ -396,6 +352,8 @@ Troubleshooting
.. _Software Update documentation: https://clearlinux.org/documentation/clear-linux/concepts/swupd-about#updating
.. _containers-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/containers-basic
.. _cloud-native-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/cloud-native-basic
.. _preflight check: https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/#preflight-checks
@@ -31,3 +31,4 @@ Explore our tutorials to discover what you can do with |CL|!
kata_migration
kubernetes
greengrass
dlrs
+1 -3
View File
@@ -1,12 +1,10 @@
.. |CL| replace:: Clear Linux OS
.. |CLOSIA| replace:: Clear Linux* OS
.. |CL-ATTR| replace:: Clear Linux* OS
.. |CL-PRJ| replace:: Clear Linux* project
.. |CC| replace:: Clear Containers
.. |CAUTION-BACKUP-USB| replace::
.. |CAUTION-BACKUP-USB| replace::
Burning an image formats the USB drive, thus destroying all existing content. Backup your data before proceeding.