diff --git a/source/clear-linux/concepts/figures/mixer-about-1.png b/source/clear-linux/concepts/figures/mixer-about-1.png new file mode 100644 index 00000000..8120a068 Binary files /dev/null and b/source/clear-linux/concepts/figures/mixer-about-1.png differ diff --git a/source/clear-linux/concepts/figures/mixer-about-2.png b/source/clear-linux/concepts/figures/mixer-about-2.png new file mode 100644 index 00000000..83f03375 Binary files /dev/null and b/source/clear-linux/concepts/figures/mixer-about-2.png differ diff --git a/source/clear-linux/concepts/mixer-about.rst b/source/clear-linux/concepts/mixer-about.rst index 1caf3764..485ea846 100644 --- a/source/clear-linux/concepts/mixer-about.rst +++ b/source/clear-linux/concepts/mixer-about.rst @@ -3,21 +3,80 @@ Mixer ##### -Mixing refers to composing a custom, versioned image of |CL-ATTR| for a -specific use case. While upstream |CL| provides options to install bundles -for various capabilities, some developers and OSVs may wish to either -augment the operating system with functionality from their own packages -or modify the structure of current bundles. Mixing offers a method to add -capabilities while retaining the ability to stay up to date with an upstream -version of |CL|. +|CL-ATTR| is a powerful, modular, and customizable OS. Upstream |CL| offers +many images to support different environments and use-cases. There are +hundreds of bundles that will meet most, if not all, of your OS and software +needs. -Mixing is a multi-step process that starts with installing the mixer bundle -using swupd. You will also need a working knowledge of -:abbr:`RPMs (RPM Package Manager files)` and how |CL| bundles work. For a -detailed guide on how to create a |CL| mix, visit :ref:`mixer`. +However, if you need additional customization or content, |CL| provides the +mixer tool. Depending on your needs, the mixer tool allows you to: + +1. :ref:`create-mix` to create a distinct derivative of the |CL| that + contains your custom software. +2. :ref:`create-mixin` to add custom bundles but also keep updating the OS + from upstream. + +.. _create-mix: + +Create a mix +============ + +When creating a mix, you can + +* Use any existing upstream bundles with no modification. +* Redefine what goes into existing bundles. +* Create completely new, custom bundles with your own custom packages. + +With mixer you are not required to incorporate every upstream release into +your mix. You decide which upstream versions to update your derivative to, as +illustrated in Figure 1. + +.. figure:: figures/mixer-about-1.png + :scale: 75% + :alt: Creating a custom mix. + + Figure 1: With a custom mix, you add your custom bundle and decide which + upstream versions to update your mix to, on your own release cycle. + +Creating your own mix forks away from the |CL| upstream and requires that you +act as your own OSV. There is a greater level of responsibility, requiring +more infrastructure and processes to adopt. However, with this approach, you +have a higher degree of control and customization of your custom |CL|. + + +.. _create-mixin: + +Create a mixin +============== + +The second option is to use the :command:`mixin` tool, a light wrapper for +mixer, to create custom bundles and sideload them into your upstream version +of |CL|. A mixin is useful when you need to add custom or 3rd +party content but want to keep on the upstream update cycle, as shown in +Figure 2. You can also create new bundles using upstream packages. + +.. figure:: figures/mixer-about-2.png + :scale: 75% + :alt: Creating a custom mix. + + Figure 2: With a mixin you can add custom bundles, but stay on upstream. + +Mixin is primarily intended for end users. It is easier to adopt as it does +not require breaking from upstream or acting as an OSV. With mixin + +* You are responsible for maintaining and testing your custom bundle(s). +* You retain access to all upstream bundles and updates. +* You can easily revert your system back to the upstream version. Related topics ============== -* :ref:`swupd-about` +|CL| provides flexibility in how you customize your OS. Learn more about +mixer and related topics to decide which customization approach is best for +you. + +* :ref:`mixer` +* :ref:`mixin` * :ref:`bundles-about` +* :ref:`swupd-about` +* :ref:`deploy-at-scale` diff --git a/source/clear-linux/get-started/bare-metal-install-beta/bare-metal-install-beta.rst b/source/clear-linux/get-started/bare-metal-install-beta/bare-metal-install-beta.rst index 5cf526d4..07cce2a5 100644 --- a/source/clear-linux/get-started/bare-metal-install-beta/bare-metal-install-beta.rst +++ b/source/clear-linux/get-started/bare-metal-install-beta/bare-metal-install-beta.rst @@ -3,10 +3,10 @@ Install |CL-ATTR| from the live desktop beta image ################################################## -The live desktop beta image allows you to boot |CL-ATTR| into a GNOME -desktop without modifying the host system. Using the live image, you can -explore the possibilities of developing with |CL|. You can also launch the -new installer and install |CL| on your target system. +The live desktop beta image allows you to boot |CL-ATTR| into a GNOME +desktop without modifying the host system. Using the live image, you can +explore the possibilities of developing with |CL|. You can also launch the +new installer and install |CL| on your target system. .. contents:: :local: :depth: 1 @@ -14,19 +14,19 @@ new installer and install |CL| on your target system. System requirements ******************* -Assure that your target system supports the installation: +Assure that your target system supports the installation: * :ref:`system-requirements` * :ref:`compatibility-check` -Preliminary steps +Preliminary steps ***************** -#. `Navigate to the image directory`_. +#. `Navigate to the image directory`_. #. Download the file :file:`clear--live-desktop-beta.img.xz` - .. note:: + .. note:: ```` is the latest |CL| auto-numbered release. @@ -42,15 +42,15 @@ Install from live image *********************** After you've downloaded and burned the live desktop image on a -USB drive, follow these steps. +USB drive, follow these steps. #. Insert the USB drive into an available USB slot. #. Power on the system. -#. Open the system BIOS setup menu by pressing the F2 key. +#. Open the system BIOS setup menu by pressing the F2 key. - .. note:: + .. note:: Your BIOS setup menu entry point may vary. @@ -64,9 +64,9 @@ USB drive, follow these steps. Launch the installer ==================== -#. After the |CL| live desktop image boots, select the :guilabel:`Activities` menu in the upper left. +#. After the |CL| live desktop image boots, select the :guilabel:`Activities` menu in the upper left. -#. Select the icon, :guilabel:`Install Clear Linux OS`, shown in Figure 1. +#. Select the icon, :guilabel:`Install Clear Linux OS`, shown in Figure 1. .. figure:: figures/bare-metal-install-beta-1.png :scale: 50 % @@ -82,37 +82,39 @@ Launch the installer Figure 2: Clear Linux OS Installer -.. _minimum-installation-requirements: +.. _minimum-installation-requirements: Minimum installation requirements ********************************* -To fulfill minimum installation requirements, complete -:guilabel:`Required options`. All other menu items are optional. +To fulfill minimum installation requirements, complete the +:guilabel:`Required options`. :guilabel:`Advanced options` are optional. Additional user help is described below. .. note:: - * In the Main menu, the :kbd:`Install` button is highlighted - **only after** you complete :ref:`required-options-installation`. + * The :kbd:`Install` button is only highlighted **after** you complete the + :ref:`required-options-installation`, and after you enter all required + values in submenus of :ref:`required-options-installation` and + :ref:`advanced-options-installation`. * You must choose whether or not to participate in :ref:`telemetrics` - before you can :ref:`finish-cl-installation`. + before you can finish installation. - * You may wish to :ref:`test-network-settings` before you - :ref:`configure-network-interfaces`. Assure that a ``Success`` message is received. + * You may wish to :ref:`test-network-settings` before you + :ref:`configure-network-interfaces`. Assure that a ``Success`` message is received before installation. -Main menu +Main Menu ********* -In the |CL| Installer main menu, two tabs appear: +In the |CL| Installer Main Menu, two tabs appear: :guilabel:`[R] Required options` and :guilabel:`[A] Advanced options`. Navigate between tabs using the arrow keys or these shortcut keys: * :kbd:`Shift+A` for :guilabel:`[A] Advanced options` * :kbd:`Shift+R` for :guilabel:`[R] Required options` -To meet the minimum requirements, enter your choices in the -:guilabel:`Required options`. After confirmation, your selections appear +To meet the minimum requirements, enter your choices in the +:guilabel:`Required options`. After confirmation, your selections appear below menu options in the Main Menu, beside the :guilabel:`>>` chevron. Navigation @@ -124,13 +126,13 @@ Navigation * Select :kbd:`Cancel` or :kbd:`Esc` to cancel your choice. - The |CL| Installer main menu appears as shown in Figure 3. + The |CL| Installer Main Menu appears as shown in Figure 3. .. figure:: figures/bare-metal-install-beta-3.png :scale: 100 % - :alt: Clear Linux OS Installer main menu + :alt: Clear Linux OS Installer Main Menu - Figure 3: :guilabel:`Clear Linux OS Installer main menu` + Figure 3: :guilabel:`Clear Linux OS Installer Main Menu` .. _required-options-installation: @@ -140,7 +142,7 @@ Required options Choose Timezone =============== -#. From the main menu, select :guilabel:`Choose Timezone`. +#. From the Main Menu, select :guilabel:`Choose Timezone`. #. Select :kbd:`Enter` to continue. @@ -154,7 +156,7 @@ Choose Timezone .. note:: Select :guilabel:`Cancel` to restore the previously defined value and - return to the main menu. + return to the Main Menu. .. figure:: figures/bare-metal-install-beta-4.png :scale: 100 % @@ -165,23 +167,23 @@ Choose Timezone Choose Language =============== -#. From the main menu, select :guilabel:`Choose Language`. +#. From the Main Menu, select :guilabel:`Choose Language`. #. Select :kbd:`Enter`. #. In :guilabel:`Select System Language`, navigate to your desired language. -#. Select :kbd:`Tab` to highlight :kbd:`Confirm`. +#. Select :kbd:`Tab` to highlight :kbd:`Confirm`. #. Select :kbd:`Enter` to :kbd:`Confirm`. - - .. note:: - Select :guilabel:`Cancel` to restore the previously defined value and return to the main menu. + .. note:: + + Select :guilabel:`Cancel` to restore the previously defined value and return to the Main Menu. .. figure:: figures/bare-metal-install-beta-5.png :scale: 100 % - :alt: Select System Language + :alt: Select System Language Figure 5: :guilabel:`Select System Language` @@ -189,22 +191,22 @@ Choose Language Configure the Keyboard ====================== -#. From the main menu, select :guilabel:`Configure the Keyboard`. +#. From the Main Menu, select :guilabel:`Configure the Keyboard`. -#. Select :kbd:`Enter`. +#. Select :kbd:`Enter`. #. In :guilabel:`Select Keyboard`, navigate to the desired option. #. In :guilabel:`Test keyboard`, type text to assure that the keys map to - your keyboard. + your keyboard. #. Select :kbd:`Tab` to highlight :guilabel:`Confirm`. #. Select :kbd:`Enter` to :kbd:`Confirm`. - .. note:: + .. note:: - Select :guilabel:`Cancel` to restore the previously defined value and return to the main menu. + Select :guilabel:`Cancel` to restore the previously defined value and return to the Main Menu. .. figure:: figures/bare-metal-install-beta-6.png :scale: 100 % @@ -215,73 +217,69 @@ Configure the Keyboard Configure Media =============== -#. From the main menu, select :guilabel:`Configure Media`. +#. From the Main Menu, select :guilabel:`Configure Media`. .. note:: - - Available media appear by default. -#. Navigate to the desired media until highlighted. + Available media appear by default. - .. TODO: Add another screenshot showing contextual menu. +#. Navigate to the desired media until highlighted. + + .. figure:: figures/bare-metal-install-beta-7.png + :scale: 100 % + :alt: Configure Media menu + + Figure 7: :guilabel:`Configure Media menu` + + .. note:: + + Use additional options as necessary during partitioning: + + * Select :guilabel:`Revert` to cancel changes and revert to the + original configuration. + * Select :guilabel:`Rescan` to show media connected to the host + machine. #. Select :kbd:`Enter` or :kbd:`Spacebar` to select it. - Two options for partitioning appear. + Two options for partitioning appear. #. Select :guilabel:`Auto Partition` to accept the default configuration, or :guilabel:`Add Partition` to add partitions manually. See instructions below. - .. figure:: figures/bare-metal-install-beta-7.png - :scale: 100 % - :alt: Configure Media menu - - Figure 7: :guilabel:`Configure Media menu` - - .. note:: - - Use additional options as necessary during partitioning: - - * Select :guilabel:`Revert` to cancel changes and revert to the - original configuration. - * Select :guilabel:`Rescan` to show media connected to the host - machine. - #. Select :kbd:`Enter` to :kbd:`Confirm`. -Choose one partitioning option below ------------------------------------- +#. Choose one partitioning option and continue below: -* :ref:`auto-partition` -* :ref:`add-partition` + * :ref:`auto-partition` + * :ref:`add-partition` -.. _auto-partition: +.. _auto-partition: Auto Partition ============== -Auto partition will configure the media to meet the minimum requirements. +Auto partition will configure the media to meet the minimum requirements. -#. Select :guilabel:`Auto Partition` as shown in Figure 8. +#. Select :guilabel:`Auto Partition` as shown in Figure 8. -#. Select :kbd:`Enter` or :kbd:`Spacebar` to highlight - :guilabel:`Auto partition`. +#. Select :kbd:`Enter` or :kbd:`Spacebar` to highlight + :guilabel:`Auto partition`. #. Select :kbd:`Enter` to :kbd:`Confirm`. - This action returns you to the Main Menu. + This action returns you to the Main Menu. .. figure:: figures/bare-metal-install-beta-8.png :scale: 100 % - :alt: Auto Partition + :alt: Auto Partition Figure 8: :guilabel:`Auto Partition` +#. You may skip to the :ref:`telemetry-tui` section below. -#. You may skip to the :ref:`telemetry-tui` section below. - -.. _add-partition: +.. _add-partition: Add Partition ============= @@ -289,104 +287,104 @@ Add Partition Minimum requirements ==================== -To add partitions manually, assign partitions per the minimum +To add partitions manually, assign partitions per the minimum requirements shown in Table 1: .. list-table:: **Table 1. Disk Partition Setup** :widths: 33, 33, 33 :header-rows: 1 - * - FileSystem + * - FileSystem - Mount Point - Minimum size * - ``VFAT`` - /boot - 150M * - ``swap`` - - + - - 256MB * - ``root`` - / - - *Size depends upon use case/desired bundles.* + - *Size depends upon use case/desired bundles.* -#. In the Configure Media menu, select :guilabel:`Add Partition`. +#. In the Configure Media menu, select :guilabel:`Add Partition`. - .. note:: To change an existing partition, navigate to the partition, + .. note:: To change an existing partition, navigate to the partition, and select :guilabel:`Enter`. boot partition -------------- -#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to +#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to select the file system type. See Figure 9. -#. In :guilabel:`Mount Point`, enter the ``/boot`` partition. +#. In :guilabel:`Mount Point`, enter the ``/boot`` partition. #. In :guilabel:`Size`, enter an appropriate size (e.g., 150M) per Table 1. -#. Navigate to :guilabel:`Confirm` until highlighted. +#. Navigate to :guilabel:`Confirm` until highlighted. -#. Select :guilabel:`Enter` to confirm. +#. Select :guilabel:`Enter` to confirm. .. figure:: figures/bare-metal-install-beta-9.png :scale: 100 % - :alt: boot partition + :alt: boot partition Figure 9: :guilabel:`boot partition` swap partition -------------- -#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to +#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to select the file system type. See Figure 10. -#. In :guilabel:`Mount Point`, the field remains blank. +#. In :guilabel:`Mount Point`, the field remains blank. - .. note:: + .. note:: - The Mount Point field is disabled. + The Mount Point field is disabled. #. In :guilabel:`Size`, enter an appropriate size (e.g., 256MB) per Table 1. -#. Navigate to :guilabel:`Add`. +#. Navigate to :guilabel:`Add`. -#. Select :guilabel:`Enter` to continue. +#. Select :guilabel:`Enter` to continue. .. figure:: figures/bare-metal-install-beta-10.png :scale: 100 % - :alt: swap partition + :alt: swap partition Figure 10: :guilabel:`swap partition` root partition -------------- -#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to +#. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to select the file system type. See Figure 11. #. In :guilabel:`Mount Point`, enter ``/root``. #. In :guilabel:`Size`, enter an appropriate size per Table 1. - .. note:: + .. note:: - The remaining available space shows by default. + The remaining available space shows by default. -#. Navigate to :guilabel:`Add` until highlighted. +#. Navigate to :guilabel:`Add` until highlighted. -#. Select :guilabel:`Enter` to continue. +#. Select :guilabel:`Enter` to continue. - You are returned to the :guilabel:`Configure media` menu. + You are returned to the :guilabel:`Configure media` menu. -#. Select :guilabel:`Confirm` to complete the media selection. +#. Select :guilabel:`Confirm` to complete the media selection. .. figure:: figures/bare-metal-install-beta-11.png :scale: 100 % - :alt: root partition + :alt: root partition Figure 11: :guilabel:`root partition` - You are returned to the :guilabel:`Configure media` menu. Your partitions should appear similar to those shown in Figure 12. + You are returned to the :guilabel:`Configure media` menu. Your partitions should appear similar to those shown in Figure 12. .. figure:: figures/bare-metal-install-beta-12.png :scale: 100 % @@ -396,20 +394,20 @@ root partition #. Navigate to :guilabel:`Confirm` until highlighted. -#. Select :guilabel:`Enter` to complete adding partitions. +#. Select :guilabel:`Enter` to complete adding partitions. -.. _telemetry-tui: +.. _telemetry-tui: Telemetry ========= -To fulfill the :guilabel:`Required options`, choose whether or not to -participate in `telemetry`_. :ref:`telemetrics` is a |CL| feature that -reports failures and crashes to the |CL| development team for improvements. -For more detailed information, see our :ref:`telemetry-about` page. +To fulfill the :guilabel:`Required options`, choose whether or not to +participate in `telemetry`_. :ref:`telemetrics` is a |CL| feature that +reports failures and crashes to the |CL| development team for improvements. +For more detailed information, visit our :ref:`telemetry-about` page. -#. In the main menu, navigate to :guilabel:`Telemetry` and select - :kbd:`Enter`. +#. In the Main Menu, navigate to :guilabel:`Telemetry` and select + :kbd:`Enter`. #. Select :kbd:`Tab` to highlight your choice. @@ -417,82 +415,86 @@ For more detailed information, see our :ref:`telemetry-about` page. .. figure:: figures/bare-metal-install-beta-13.png :scale: 100 % - :alt: Enable Telemetry + :alt: Enable Telemetry - Figure 13: :guilabel:`Enable Telemetry` + Figure 13: :guilabel:`Enable Telemetry` Skip to finish installation =========================== -After selecting values for all :guilabel:`Required options`, you may skip directly to :ref:`finish-cl-installation` +After selecting values for all :guilabel:`Required options`, you may skip +directly to :ref:`finish-cl-installation` -If you wish to configure network interfaces or proxy settings, -or add bundles, or add/manage users, continue to the next section. +In the Main Menu, select the :guilabel:`Advanced options` if you wish to +configure network interfaces or proxy settings, add bundles, add/manage +users, and more. Continue to the next section. + +.. _advanced-options-installation: Advanced Options **************** .. _configure-network-interfaces: -Configure Network Interfaces +Configure Network Interfaces ============================ By default, |CL| is configured to automatically detect the host network interface using DHCP. However, if you want to use a static IP address or if you do not have a DHCP server on your network, follow these instructions to manually configure the network interface. Otherwise, default network -interface settings are automatically applied. +interface settings are automatically applied. -.. note:: - - If DHCP is available, no user selection may be required. +.. note:: + + If DHCP is available, no user selection may be required. #. Navigate to :guilabel:`Configure Network Interfaces` and - select :kbd:`Enter`. + select :kbd:`Enter`. -#. Navigate to the network :guilabel:`interface` you wish to change. +#. Navigate to the network :guilabel:`interface` you wish to change. -#. When the desired :guilabel:`interface` is highlighted, select +#. When the desired :guilabel:`interface` is highlighted, select :guilabel:`Enter` to edit. - .. note:: Multiple network interfaces may appear. + .. note:: Multiple network interfaces may appear. .. figure:: figures/bare-metal-install-beta-14.png :scale: 100 % :alt: Configure Network Interfaces - Figure 14: :guilabel:`Configure Network Interfaces` + Figure 14: :guilabel:`Configure Network Interfaces` -#. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom). +#. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom). Optional: Navigate to the checkbox :guilabel:`Automatic / dhcp` and select - :kbd:`Spacebar` to deselect. + :kbd:`Spacebar` to deselect. .. figure:: figures/bare-metal-install-beta-15.png :scale: 100 % :alt: Network interface configuration - Figure 15: :guilabel:`Network interface configuration` + Figure 15: :guilabel:`Network interface configuration` #. Navigate to the appropriate fields and assign the desired - network configuration. + network configuration. -#. To save settings, navigate to :guilabel:`Confirm` and select +#. To save settings, navigate to :guilabel:`Confirm` and select :kbd:`Enter`. - .. note:: + .. note:: To revert to previous settings, navigate to the :guilabel:`Cancel` and select :kbd:`Enter`. #. Upon confirming network configuration, the :guilabel:`Testing Networking` - dialogue appears. Assure the result shows success. If a failure occurs, - your changes will not be saved. + dialogue appears. Assure the result shows success. If a failure occurs, + your changes will not be saved. #. Upon confirmation, you are returned to :guilabel:`Network interface` - settings. + settings. -#. Navigate to and select :guilabel:`Main Menu`. +#. Navigate to and select :guilabel:`Main Menu`. Optional: :ref:`Skip to finish installation `. @@ -500,27 +502,27 @@ Proxy ===== |CL| automatically attempts to detect proxy settings, as described in -`Autoproxy`_. If you need to manually assign proxy settings, follow this -instruction. +`Autoproxy`_. If you need to manually assign proxy settings, follow this +instruction. -#. From the Advanced options menu, navigate to :guilabel:`Proxy`, and +#. From the Advanced options menu, navigate to :guilabel:`Proxy`, and select :kbd:`Enter`. - -#. Navigate to the field :guilabel:`HTTPS Proxy`. - + +#. Navigate to the field :guilabel:`HTTPS Proxy`. + .. figure:: figures/bare-metal-install-beta-16.png :scale: 100 % :alt: Configure the network proxy - Figure 16: :guilabel:`Configure the network proxy` + Figure 16: :guilabel:`Configure the network proxy` #. Enter the desired proxy address and port using conventional syntax, - such as: http://address:port. + such as: \http://address:port. #. Navigate to :guilabel:`Confirm` and select :kbd:`Enter`. -#. To revert to previous settings, navigate to :guilabel:`Cancel` - and select :guilabel:`Cancel`. +#. To revert to previous settings, navigate to :guilabel:`Cancel` + and select :guilabel:`Cancel`. Optional: :ref:`Skip to finish installation `. @@ -529,43 +531,44 @@ Optional: :ref:`Skip to finish installation `. Test Network Settings ===================== -To manually assure network connectivity before installing |CL|, +To manually assure network connectivity before installing |CL|, select :guilabel:`Test Network Settings` and select :guilabel:`Enter`. -A progress bar appears as shown in Figure 17. +A progress bar appears as shown in Figure 17. .. figure:: figures/bare-metal-install-beta-17.png :scale: 100 % :alt: Testing Networking dialogue - Figure 17: :guilabel:`Testing Networking dialogue` + Figure 17: :guilabel:`Testing Networking dialogue` -.. note:: - - Any changes made to network settings are automatically tested - during configuration. +.. note:: + + Any changes made to network settings are automatically tested + during configuration. Optional: :ref:`Skip to finish installation `. Bundle Selection ================ -#. On the Advanced menu, select :guilabel:`Bundle selection` +#. On the Advanced menu, select :guilabel:`Bundle Selection` #. Navigate to the desired bundle using :kbd:`Tab` or :kbd:`Up/Down` arrows. -#. Select :kbd:`Spacebar` to select the checkbox for each desired bundle. - +#. Select :kbd:`Spacebar` to select the checkbox for each desired bundle. .. figure:: figures/bare-metal-install-beta-18.png :scale: 100 % - :alt: Bundle selection + :alt: Bundle Selection - Figure 18: :guilabel:`Bundle selection` + Figure 18: :guilabel:`Bundle Selection` + +..TODO:Add new screenshot that shows more bundles. #. Select :kbd:`Confirm` or :kbd:`Cancel`. - You are returned to the :guilabel:`Advanced options` menu. + You are returned to the :guilabel:`Advanced options` menu. Optional: :ref:`Skip to finish installation `. @@ -575,199 +578,202 @@ User Manager Add New User ------------ -#. In Advanced Options, select :guilabel:`User Manager`. +#. In Advanced Options, select :guilabel:`User Manager`. -#. Select :guilabel:`Add New User` as shown in Figure 19. +#. Select :guilabel:`Add New User` as shown in Figure 19. .. figure:: figures/bare-metal-install-beta-19.png :scale: 100 % - :alt: Add New User, User Name + :alt: Add New User, User Name - Figure 19: :guilabel:`Add New User` + Figure 19: :guilabel:`Add New User` - #. Optional: Enter a :guilabel:`User Name`. - .. note: + .. note: - The User Name must be alphanumeric and can include spaces, commas, or hyphens. Maximum length is 64 characters. + The User Name must be alphanumeric and can include spaces, commas, or hyphens. Maximum length is 64 characters. .. figure:: figures/bare-metal-install-beta-20.png :scale: 100 % - :alt: User Name + :alt: User Name - Figure 20: :guilabel:`User Name` + Figure 20: :guilabel:`User Name` #. Enter a :guilabel:`Login`. - .. note:: + .. note:: - The User Login must be alphanumeric and can include hyphens and underscores. Maximum length is 31 characters. + The User Login must be alphanumeric and can include hyphens and underscores. Maximum length is 31 characters. #. Enter a :guilabel:`Password`. - .. note: + .. note: Minimum length is 8 characters. Maximum length is 255 characters. -#. In :guilabel:`Retype`, enter the same password. +#. In :guilabel:`Retype`, enter the same password. -#. Optional: Navigate to the :guilabel:`Administrative` checkbox and select - :kbd:`Spacebar` to assign administrative rights to the user. +#. Optional: Navigate to the :guilabel:`Administrative` checkbox and select + :kbd:`Spacebar` to assign administrative rights to the user. - .. note:: + .. note:: - Selecting this option enables sudo privileges for the user. + Selecting this option enables sudo privileges for the user. .. figure:: figures/bare-metal-install-beta-21.png :scale: 100 % :alt: Administrative rights - Figure 21: :guilabel:`Administrative rights` - -#. Select :kbd:`Confirm`. + Figure 21: :guilabel:`Administrative rights` + +#. Select :kbd:`Confirm`. .. note:: - If desired, select :guilabel:`Reset` to reset the form. + If desired, select :guilabel:`Reset` to reset the form. + +#. In :guilabel:`User Manager`, navigate to :guilabel:`Confirm`. + +#. With :guilabel:`Confirm` highlighted, select :kbd:`Enter`. Modify / Delete User -------------------- -#. In User Manager, select :guilabel:`Tab` to highlight the user, as shown - in Figure 22. +#. In User Manager, select :guilabel:`Tab` to highlight the user, as shown + in Figure 22. -#. Select :kbd:`Enter` to modify the user. +#. Select :kbd:`Enter` to modify the user. .. figure:: figures/bare-metal-install-beta-22.png :scale: 100 % :alt: Modify User - Figure 22: :guilabel:`Modify User` + Figure 22: :guilabel:`Modify User` -#. Modify user details as desired. +#. Modify user details as desired. - Optional: To delete the user, navigate to the :guilabel:`Delete` - button and select :kbd:`Enter` + Optional: To delete the user, navigate to the :guilabel:`Delete` + button and select :kbd:`Enter` + +#. Navigate to :kbd:`Confirm` until highlighted. -#. Navigate to :kbd:`Confirm` until highlighted. - .. note:: - Optional: Select :guilabel:`Reset` to rest the form. + Optional: Select :guilabel:`Reset` to rest the form. -#. Select :guilabel:`Confirm` to save the changes you made. +#. Select :guilabel:`Confirm` to save the changes you made. -You are returned to the User Manager menu. +You are returned to the User Manager menu. Optional: :ref:`Skip to finish installation `. -Kernel Command Line +Kernel Command Line =================== -For more advanced users, |CL| provides the ability to add, or remove, kernel -arguments. If you want to append a new argument, enter the argument here. -This argument will be used every time you install a new kernel or update a -new kernel. +For advanced users, |CL| provides the ability to add or remove kernel +arguments. If you want to append a new argument, enter the argument here. +This argument will be used every time you install or update a +new kernel. -#. In Advanced Options, select :guilabel:`Tab` to highlight - :guilabel:`Kernel Command Line`. +#. In Advanced Options, select :guilabel:`Tab` to highlight + :guilabel:`Kernel Command Line`. -#. Select :kbd:`Enter`. +#. Select :kbd:`Enter`. -#. To add or remove arguments, choose one of the following options. +#. To add or remove arguments, choose one of the following options. .. figure:: figures/bare-metal-install-beta-23.png :scale: 100 % :alt: kernel command line - Figure 23: :guilabel:`kernel command line` + Figure 23: :guilabel:`kernel command line` -#. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`. +#. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`. -#. To remove an existing argument, enter the argument in - :guilabel:`Remove Arguments`. +#. To remove an existing argument, enter the argument in + :guilabel:`Remove Arguments`. -#. Select :kbd:`Confirm`. +#. Select :kbd:`Confirm`. Optional: :ref:`Skip to finish installation `. -Kernel Selection +Kernel Selection ================ #. Select a kernel option. By default, the latest kernel release is - selected. + selected. - .. note:: + .. note:: + + To choose a different kernel, such as Long Term Support (LTS), shown in Figure 24, navigate to it using :guilabel:`Tab`. - To choose a different kernel, such as Long Term Support (LTS), shown in Figure 24, navigate to it using :guilabel:`Tab`. - .. figure:: figures/bare-metal-install-beta-24.png :scale: 100 % :alt: Kernel selection - Figure 24: :guilabel:`Kernel selection` + Figure 24: :guilabel:`Kernel selection` -#. Select :kbd:`Spacebar` to select the desired option. - -#. Select :kbd:`Confirm`. +#. Select :kbd:`Spacebar` to select the desired option. + +#. Select :kbd:`Confirm`. Optional: :ref:`Skip to finish installation `. Swupd Mirror ============ -If you have your own custom mirror of |CL|, you may want to add that URL. +If you have your own custom mirror of |CL|, you can add its URL. -#. In Advanced Options, select :guilabel:`Swupd Mirror`. +#. In Advanced Options, select :guilabel:`Swupd Mirror`. #. To add a local swupd mirror, enter a valid URL in :guilabel:`Mirror URL:` -#. Select :kbd:`Confirm`. +#. Select :kbd:`Confirm`. .. figure:: figures/bare-metal-install-beta-25.png :scale: 100 % :alt: Swupd Mirror - Figure 25: :guilabel:`Swupd Mirror` + Figure 25: :guilabel:`Swupd Mirror` Optional: :ref:`Skip to finish installation `. Assign Hostname =============== -#. In Advanced Options, select :guilabel:`Assign Hostname`. +#. In Advanced Options, select :guilabel:`Assign Hostname`. #. In :guilabel:`Hostname`, enter the hostname only (excluding the domain). .. note:: Hostname does not allow empty spaces. Hostname must start with an - alphanumeric character but may also contain hyphens. Maximum length of - 63 characters. + alphanumeric character but may also contain hyphens. Maximum length of + 63 characters. .. figure:: figures/bare-metal-install-beta-26.png :scale: 100 % :alt: Assign Hostname - Figure 26: :guilabel:`Assign Hostname` + Figure 26: :guilabel:`Assign Hostname` -#. Navigate to :kbd:`Confirm` until highlighted. +#. Navigate to :kbd:`Confirm` until highlighted. -#. Select :kbd:`Confirm`. +#. Select :kbd:`Confirm`. Optional: :ref:`Skip to finish installation `. Automatic OS Updates ==================== -In the rare case that you need to disable automatic software updates, +In the rare case that you need to disable automatic software updates, follow the onscreen instructions. -#. In Advanced Options, select :guilabel:`Automatic OS Updates`. +#. In Advanced Options, select :guilabel:`Automatic OS Updates`. -#. Select the desired option. +#. Select the desired option. .. figure:: figures/bare-metal-install-beta-27.png :scale: 100 % @@ -775,7 +781,7 @@ follow the onscreen instructions. Figure 27: :guilabel:`Automatic OS Updates` -You are returned to the :guilabel:`Main Menu`. +You are returned to the :guilabel:`Main Menu`. .. _finish-cl-installation: @@ -783,16 +789,16 @@ Finish installation ******************* #. When you are satisfied with your installation configuration, navigate to - :guilabel:`Install` and select :kbd:`Enter`. + :guilabel:`Install` and select :kbd:`Enter`. - .. note:: + .. note:: - Whenever installation is finished, a ``reboot`` button appears. + Whenever installation is finished, a ``reboot`` button appears. -#. Select ``reboot``. +#. Select ``reboot``. #. When the system reboots, remove any installation media present. - + **Congratulations!** You have successfully installed |CL| on bare metal using the new installer. diff --git a/source/clear-linux/get-started/bare-metal-install-beta/figures/bare-metal-install-beta-2.png b/source/clear-linux/get-started/bare-metal-install-beta/figures/bare-metal-install-beta-2.png index 442988f6..d3d4cbb6 100644 Binary files a/source/clear-linux/get-started/bare-metal-install-beta/figures/bare-metal-install-beta-2.png and b/source/clear-linux/get-started/bare-metal-install-beta/figures/bare-metal-install-beta-2.png differ diff --git a/source/clear-linux/guides/deploy-at-scale.rst b/source/clear-linux/guides/deploy-at-scale.rst index 2f0bced2..5b3867df 100644 --- a/source/clear-linux/guides/deploy-at-scale.rst +++ b/source/clear-linux/guides/deploy-at-scale.rst @@ -1,4 +1,4 @@ -.. _deploy-at-scale.rst: +.. _deploy-at-scale: Deploy at Scale ############### diff --git a/source/clear-linux/guides/maintenance/assign-static-ip.rst b/source/clear-linux/guides/maintenance/assign-static-ip.rst new file mode 100644 index 00000000..444fbf21 --- /dev/null +++ b/source/clear-linux/guides/maintenance/assign-static-ip.rst @@ -0,0 +1,71 @@ +.. _assign-static-ip: + +Assign a static IP address to a network interface +################################################# + +Introduction +************ + +By default, your |CL-ATTR| system automatically gets an IP address from your +network via DHCP. If you do not have a DHCP server on your network or simply +want to use a static IP address, follow the steps in this guide. + +Process +******* + +#. Create this directory structure: + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/network + +#. Identify the interface to be assigned the static IP address: + + .. code-block:: bash + + ip addr + + The system returns the following: + + .. code-block:: console + + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host + valid_lft forever preferred_lft forever + + 2: wlp1s0: mtu 1500 qdisc mq state DOWN group default qlen 1000 + link/ether 4a:98:8d:e5:43:15 brd ff:ff:ff:ff:ff:ff + + 3: eno1: mtu 1500 qdisc fq state UP group default qlen 1000 + link/ether f4:4d:30:68:96:20 brd ff:ff:ff:ff:ff:ff + inet 10.0.1.2/24 brd 10.54.74.255 scope global dynamic eno1 + valid_lft 6766sec preferred_lft 6766sec + inet6 fe80::f64d:30ff:fe68:9620/64 scope link + valid_lft forever preferred_lft forever + + In this example, we will use the `eno1` interface. + +#. Create the :file:`70-static.network` file and add the following: + + .. code-block:: bash + + sudo $EDITOR /etc/systemd/network/70-static.network + + [Match] + Name=[interface name] + [Network] + Address=[IP address]/24 + DHCP=yes # to get DNS info, etc. + + Replace [interface name] and [IP address] with your specific settings. + +#. Restart the networkd service: + + .. code-block:: bash + + sudo systemctl restart systemd-networkd + +**Congratulations!** You have successfully assigned a static IP address. diff --git a/source/clear-linux/guides/maintenance/autoproxy.rst b/source/clear-linux/guides/maintenance/autoproxy.rst new file mode 100644 index 00000000..b5131ab7 --- /dev/null +++ b/source/clear-linux/guides/maintenance/autoproxy.rst @@ -0,0 +1,18 @@ +.. _autoproxy: + +Autoproxy +######### + +.. rst-class:: content-collapse + +About +===== + +.. include:: autoproxy_about.txt + +.. rst-class:: content-collapse + +Guide +===== + +.. include:: autoproxy_guide.txt \ No newline at end of file diff --git a/source/clear-linux/guides/maintenance/autoproxy_about.txt b/source/clear-linux/guides/maintenance/autoproxy_about.txt new file mode 100644 index 00000000..80dcaa4e --- /dev/null +++ b/source/clear-linux/guides/maintenance/autoproxy_about.txt @@ -0,0 +1,43 @@ +The |CL-ATTR| is the first Linux distribution to support autoproxy. The OS +can discover a Proxy Auto-Config (PAC) script and use it to automatically +resolve the proxy needed for a given connection. With Autoproxy, you can use +|CL| inside any proxy environment without having to manually +configure the proxies. + +Corporate and private networks can be very complex, needing to restrict and +control network connections for security reasons. The typical side effects +are limited or blocked connectivity and requiring manual configuration of +proxies to perform the most mundane tasks such as cloning a repo or checking +for updates. With Clear Linux, all of the work is done behind the scenes to +effortlessly use your network and have connections “just work”. + +This feature removes massive complications in network connectivity due to +proxy issues. You can automate tasks like unit testing without worrying +about the proxy not being set and you can remove unset proxies from the +equation when dealing with network unavailability across systems. + +How Autoproxy works +------------------- + +We designed autoproxy around general tools provided by nearly any Linux +distribution with a few minor additions and modifications. We leveraged the +DHCP and network information provided from systemd and created a +PAC-discovery daemon. The daemon uses the information to resolve a URL for a +PAC file. The daemon then passes the URL into PACrunner*. PACrunner +downloads the PAC file and uses the newly implemented Duktape* engine to +parse it. + +.. figure:: figures/autoproxy_0.png + :width: 400px + + Figure 1: Autoproxy Flow + +From that point on, any cURL* or network requests query PACrunner for the +correct proxy to use. We modified the cURL library to communicate with +PACrunner over DBus. However, cURL will ignore PACrunner and run normally if +no PAC file is loaded or if you set any proxies manually. Thus, your +environment settings are respected and no time is wasted trying to resolve a +proxy. + +More importantly: all these steps happen in the background, very quickly, and +with no user interaction. \ No newline at end of file diff --git a/source/clear-linux/guides/maintenance/autoproxy_guide.txt b/source/clear-linux/guides/maintenance/autoproxy_guide.txt new file mode 100644 index 00000000..3422a747 --- /dev/null +++ b/source/clear-linux/guides/maintenance/autoproxy_guide.txt @@ -0,0 +1,77 @@ +Autoproxy allows |CL| to operate seamlessly behind a proxy +because, :ref:`swupd-guide` and other |CL| tools are implemented on +top of libcurl. Tools that do not use libcurl, like git, must +be configured independently. + +If you encounter problems with autoproxy functioning, use +:command:`pacdiscovery` and :command:`FindProxyForURL` to +help troubleshoot assuming a familiarity with PAC files and WPAD. + +.. note:: + + Learn more about WPAD, PAC files, and PAC functions at `findproxyforurl`_. + +.. _findproxyforurl: http://findproxyforurl.com/ + +Running :command:`pacdiscovery` with no arguments will immediately indicate + +1. if there is a problem resolving the :command:`WPAD` host name resolution: + + .. code:: console + + $ pacdiscovery + failed getaddrinfo: No address associated with hostname + Unable to find wpad host + +2. or if the :command:`pacrunner` service is disabled (masked). + + .. code:: console + + $ pacdiscovery + PAC url: http://autoproxy.your.domain.com/wpad.dat + Failed to create proxy config: Unit pacrunner.service is masked. + +Unmask the :command:`pacrunner` service by running: + +.. code:: console + + $ systemctl unmask pacrunner.service + +:command:`FindProxyForURL` with :command:`busctl` can also indicate if the +:command:`pacrunner.service` is masked. + +.. code:: console + + $ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + Unit pacrunner.service is masked. + dig wpad, dig wpad. + +:command:`FindProxyForURL` returns the URL and port of the proxy server when +an external URL and host are provided as arguments. + +.. code:: console + + $ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + s "PROXY proxy.your.domain.com:" + +If a proxy server is not avialable, or if :command:`pacrunner` is running +without a PAC file, :command:`FindProxyForURL` will return "DIRECT". + +.. code:: console + + $ busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + s "DIRECT" + +Once :command:`pacdiscovery` is able to look up :command:`WPAD`, restart the +:command:`pacrunner` service: + +.. code:: console + + $ systemctl stop pacrunner + $ systemctl restart pacdiscovery + +.. note:: + + A "domain" or "search" entry in :file:`/etc/resolv.conf` is required for short + name lookups to resolve. The :file:`resolv.conf` man page has additional + details. \ No newline at end of file diff --git a/source/clear-linux/guides/maintenance/figures/autoproxy_0.png b/source/clear-linux/guides/maintenance/figures/autoproxy_0.png new file mode 100644 index 00000000..7a02cd81 Binary files /dev/null and b/source/clear-linux/guides/maintenance/figures/autoproxy_0.png differ diff --git a/source/clear-linux/guides/maintenance/kernel-modules.rst b/source/clear-linux/guides/maintenance/kernel-modules.rst new file mode 100644 index 00000000..5a937990 --- /dev/null +++ b/source/clear-linux/guides/maintenance/kernel-modules.rst @@ -0,0 +1,176 @@ +.. _kernel-modules: + +Add kernel modules +################## + +Kernel modules are additional pieces of software capable of being inserted +into the Linux kernel to add functionality, such as a hardware driver. +Kernel modules may already be part of the Linux source tree (in-tree) or may +come from an external source, such as directly from a vendor (out-of-tree). + +In cases where drivers beyond those enabled by default in |CL-ATTR| are +needed it may be necessary to: + +.. contents:: :local: + :depth: 1 + :backlinks: top + +Check if the module is available through |CL| +============================================= + +Using an existing module is significantly easier to maintain and retains +signature verification of the |CL| kernel. For more information on |CL| +security practices, see the :ref:`security` page. + +|CL| comes with many upstream kernel modules available for use. If +you require a kernel module, be sure to check whether it is already available in |CL| first. + +You can search for kernel module file names, which end with the :file:`.ko` +file extension, using the :command:`swupd search` command. For example: +:command:`sudo swupd search ${module_name}.ko`. +See :ref:`swupd-search` for more information. + +Request the module be added to |CL| +=================================== + +If the kernel module you need is already open source +(e.g. in the Linux upstream) and likely to be useful to others, +consider submitting a request to add or enable in the |CL| kernel. + +Make enhancement requests to the |CL| distribution `on GitHub`_ . + +Build and load an out-of-tree module +==================================== + +In some cases you may need an out-of-tree kernel module that is not +available through |CL|. + +You can build and load out-of-tree kernel modules, however you must: + +* disable secure boot +* disable kernel module integrity checking +* build the module against new versions of the Linux kernel + +.. note:: + + Any time the kernel is upgraded on your Clear Linux system, you will + need to rebuild your out-of-tree modules. + +This approach works well for individual development or testing. +For a more scalable and customizable approach, consider using the +`mixer tool`_ to provide a custom kernel and updates. + +Build kernel module +------------------- + +#. From a |CL| system, ensure you are running the *native* kernel. + Currently only the native kernel is enabled to build and load + out-of-tree modules. + + .. code-block:: bash + + $ uname -r + 4.XX.YY-ZZZZ.native + Ensure *.native* is in the kernel name + +#. Install the `linux-dev` bundle to obtain the kernel headers, which are + required for compiling kernel modules. + + .. code-block:: bash + + sudo swupd bundle-add linux-dev + +#. Follow instructions from the kernel module source code to compile the + kernel module. + + +Load kernel module +------------------ + +#. Disable Secure Boot in your system's UEFI settings, if you have enabled + it. The loading of new out-of-tree modules modifies the signatures Secure + Boot relies on for trust. + + +#. Disable signature checking for the kernel by modifying the kernel boot + parameters and reboot the system. + + All kernel modules from |CL| have been signed to enforce kernel security. + However, out-of-tree modules break this chain of trust so this mechanism + needs to be disabled. + + .. code-block:: bash + + sudo mkdir -p /etc/kernel/cmdline.d + echo "module.sig_unenforce" | sudo tee /etc/kernel/cmdline.d/allow-unsigned-modules.conf + +#. Update the boot manager and reboot the system to implement the changed + kernel parameters. + + .. code-block:: bash + + sudo clr-boot-manager update + sudo reboot + + .. note:: + + :command:`clr-boot-manager update` does not return any + console output if successful. + + +#. After rebooting, out-of-tree modules can be manually loaded with + :command:`insmod`. + + .. code-block:: bash + + sudo insmod ${path_to_module} + + +Optional: Use `modprobe` to specify module options and aliases +-------------------------------------------------------------- + +Use :command:`modprobe` to load a module and set options. + +Because :command:`modprobe` can add or remove more than one module, due to +modules having dependencies, a method of specifying what options are +to be used with individual modules is useful. This can be done with +configuration files under the :file:`/etc/modprobe.d` directory. + +.. code-block:: bash + + sudo mkdir /etc/modprobe.d + +All files underneath the :file:`/etc/modprobe.d` directory +that end with the :file:`.conf` extension specify module options to use when +loading. This can also be used to create convenient aliases for modules or +they can override the normal loading behavior altogether for those with +special requirements. + +You can find more info on module loading in the modprobe.d manual page: + +.. code-block:: bash + + man modprobe.d + +Optional: Configure kernel modules to load at boot +-------------------------------------------------- + +Use the :file:`/etc/modules-load.d` configuration directory to +specify kernel modules to load automatically at boot. + +.. code-block:: bash + + sudo mkdir /etc/modules-load.d + +All files underneath the :file:`/etc/modules-load.d` directory +that end with the :file:`.conf` extension contain a list of module names +of aliases (one per line) to load at boot. + +You can find more info on module loading in the modules-load.d manual page: + +.. code-block:: bash + + man modules-load.d + +.. _`on GitHub`: https://github.com/clearlinux/distribution +.. _`mixer tool`: https://clearlinux.org/features/mixer-tool diff --git a/source/clear-linux/guides/maintenance/maintenance.rst b/source/clear-linux/guides/maintenance/maintenance.rst index 0751cd2b..3d89fa54 100644 --- a/source/clear-linux/guides/maintenance/maintenance.rst +++ b/source/clear-linux/guides/maintenance/maintenance.rst @@ -10,11 +10,13 @@ completed. .. toctree:: :maxdepth: 2 + autoproxy developer-workstation swupd-search enable-user-space swupd-guide bulk-provision + kernel-modules mixer mixin validate-signatures @@ -25,3 +27,4 @@ completed. download-verify-decompress-mac download-verify-decompress-windows autospec + assign-static-ip diff --git a/source/clear-linux/guides/maintenance/validate-signatures.rst b/source/clear-linux/guides/maintenance/validate-signatures.rst index 65668884..54bbfe4d 100644 --- a/source/clear-linux/guides/maintenance/validate-signatures.rst +++ b/source/clear-linux/guides/maintenance/validate-signatures.rst @@ -57,7 +57,14 @@ used for illustrative purposes. You may use any image of |CL| you choose. .. code-block:: console - openssl smime -verify -in clear-$(curl https://download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig -inform der -content sha512sum.out -CAfile ClearLinuxRoot.pem + openssl smime -verify -purpose any -in clear-$(curl https://download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig -inform der -content sha512sum.out -CAfile ClearLinuxRoot.pem + + .. note:: + + The `-purpose any` option is required when using OpenSSL 1.1. If using + an earlier version of OpenSSL, omit this option to perform signature + validation. The `openssl version` command may be used to determine the + version of OpenSSL in use. #. The output should contain ``Verification successful``. If the output contains ``bad_signature`` anywhere, then the image is not trustworthy. @@ -101,7 +108,14 @@ these steps manually when performing a ``swupd update``. .. code-block:: console - openssl smime -verify -in Manifest.MoM.sig -inform der -content Manifest.MoM -CAfile Swupd_Root.pem + openssl smime -verify -purpose any -in Manifest.MoM.sig -inform der -content Manifest.MoM -CAfile Swupd_Root.pem + + .. note:: + + The `-purpose any` option is required when using OpenSSL 1.1. If using + an earlier version of OpenSSL, omit this option to perform signature + validation. The `openssl version` command may be used to determine the + version of OpenSSL in use. .. note:: diff --git a/source/clear-linux/tutorials/dlrs.rst b/source/clear-linux/tutorials/dlrs.rst new file mode 100644 index 00000000..f856f97e --- /dev/null +++ b/source/clear-linux/tutorials/dlrs.rst @@ -0,0 +1,235 @@ +.. _dlrs: + +Deep Learning Reference Stack +############################# + +This tutorial shows you how to run benchmarking workloads in |CL-ATTR| using +TensorFlow\* and Kubeflow with the Deep Learning Reference Stack. + +The Deep Learning Reference Stack is available in two versions. +The first is `Eigen`_, which includes `TensorFlow`_ optimized for Intel® +architecture. The second is `Intel MKL-DNN`_, which includes the TensorFlow +framework optimized using Intel® Math Kernel Library for Deep Neural +Networks (Intel® MKL-DNN) primitives. + +.. contents:: :local: + :depth: 1 + +Release notes +============= + +View current `release notes`_ for the Deep Learning Reference Stack. + +View current `benchmark results`_ for the Deep Learning Reference Stack. + +.. note:: + + Performance test numbers in the Deep Learning Reference Stack were obtained using `runc` as the runtime. + +Prerequisites +============= + +* |CL| installed on host system. If not installed, :ref:`bare-metal-install` +* `containers-basic` bundle +* `cloud-native-basic` bundle + +In |CL|, `containers-basic` provides Docker\*, which is required for +TensorFlow benchmarking. Use the :command:`swupd` utility to check if +`containers-basic` and `cloud-native-basic` are present: + +.. code-block:: bash + + sudo swupd bundle-list + +If you need to install the `containers-basic` or `cloud-native-basic`, enter: + +.. code-block:: bash + + sudo swupd bundle-add containers-basic cloud-native-basic + +To ensure that kubernetes is correctly installed and configured, +:ref:`kubernetes`. + +We have validated these steps against the following software package +versions: + +* |CL| 26240--lowest version permissible. +* Docker 18.06.1 +* Kubernetes 1.11.3 +* Go 1.11.12 + +TensorFlow single and multi-node benchmarks +============================================ + +This section describes running the `TensorFlow benchmarks`_ in single node. +For multi-node testing, replicate these steps for each node. These steps +provide a template to run other benchmarks, provided that they can invoke +TensorFlow. + +#. Download and run either the `Eigen`_ or the `Intel MKL-DNN`_ docker image + from `Docker Hub`_. + + .. note:: + + You will enter the following commands in the running container. + + Replace with the name of the image. + +#. Clone the benchmark repository: + + .. code-block:: bash + + docker exec -t bash -c ‘git clone http://github.com/tensorflow/benchmarks -b cnn_tf_v1.12_compatible’ + +#. Next, execute the benchmark script to run the benchmark. + + .. code-block:: bash + + docker exec -i bash -c ‘python benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC ’. + +.. note:: + + You can replace the model with one of your choice supported by the + TensorFlow benchmarks. + +Kubeflow multi-node benchmarks +============================== + +The benchmark workload will run in a Kubernetes cluster. We will use +`Kubeflow`_ for the Machine Learning workload deployment on three nodes. + +Kubernetes setup +**************** + +Follow the instructions in the :ref:`kubernetes` tutorial to get set up on +|CL|. The kubernetes community also has +`instructions for creating a cluster`_. + +Kubernetes networking +********************* + +We used `flannel`_ as the network provider for these tests. If you are +comfortable with another network layer, refer to the Kubernetes +`networking documentation`_ for setup. + +Images +****** + +We need to add `launcher.py` to our docker image to include the Deep +Learning Reference Stack and put the benchmarks repo in the correct +location. From the docker image, run the following: + +.. code-block:: bash + + mkdir -p /opt + git clone https://github.com/tensorflow/benchmarks.git /opt/tf-benchmarks + cp launcher.py /opt + chmod u+x /opt/* + +Your entry point now becomes "/opt/launcher.py". + +This will build an image which can be consumed directly by TFJob from +kubeflow. We are working to create these images as part of our release +cycle. + +ksonnet\* +********* + +Kubeflow uses ksonnet* to manage deployments, so we need to install that before setting up Kubeflow. On |CL|, follow these steps: + +.. code-block:: bash + + swupd bundle-add go-basic-dev + export GOPATH=$HOME/go + export PATH=$PATH:$GOPATH/bin + go get github.com/ksonnet/ksonnet + cd $GOPATH/src/github.com/ksonnet/ksonnet + make install + +After the ksonnet installation is complete, ensure that binary `ks` is +accessible across the environment. + +Kubeflow +******** + +Once you have Kubernetes running on your nodes, you can setup `Kubeflow`_ by +following these instructions from their `quick start guide`_. + +.. code-block:: bash + + export KUBEFLOW_SRC=$HOME/kflow + export KUBEFLOW_TAG=”v0.3.2” + export KFAPP=”kflow_app” + export K8S_NAMESPACE=”kubeflow” + + mkdir ${KUBEFLOW_SRC} + cd ${KUBEFLOW_SRC} + ks init ${KFAPP} + cd ${KFAPP} + ks registry add kubeflow github.com/kubeflow/kubeflow/tree/${KUBEFLOW_TAG}/kubeflow + ks pkg install kubeflow/core + +Now you have all the required kubeflow packages, and you can deploy the primary one for our purposes: tf-job-operator. + +.. code-block:: bash + + ks env rm default + kubectl create namespace ${K8S_NAMESPACE} + ks env add default --namespace "${K8S_NAMESPACE}" + ks generate tf-job-operator tf-job-operator + ks apply default -c tf-job-operator + +This creates the CustomResourceDefinition(CRD) endpoint to launch a TFJob. + +Run a TFJob +=========== + +#. Select this link for the `ksonnet registries for deploying TFJobs`_. + + #. Install the TFJob componets as follows: + + .. code-block:: bash + + ks registry add dlrs-tfjob github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob + + ks pkg install dlrs-tfjob/dlrs-bench + +#. Next, generate Kubernetes manifests for the workloads and apply them to + create and run them using these commands + + .. code-block:: bash + + ks generate dlrs-resnet50 dlaasresnet50 --name=dlrsresnet50 + ks generate dlrs-alexnet dlaasalexnet --name=dlrsalexnet + ks apply default -c dlrsresnet50 + ks apply default -c dlrsalexnet + +This will replicate and deploy three test setups in your Kubernetes cluster. + +Results of Running this Tutorial +================================ +You need to parse the logs of the Kubernetes pod to get the performance +numbers. The pods will still be around post completion and will be in +‘Completed’ state. You can get the logs from any of the pods to inspect the +benchmark results. More information about `Kubernetes logging`_ is available from the Kubernetes community. + +.. _TensorFlow: https://www.tensorflow.org/ +.. _Kubeflow: https://www.kubeflow.org/ +.. _Docker Hub: https://hub.docker.com/ +.. _TensorFlow benchmarks: https://www.tensorflow.org/guide/performance/benchmarks +.. _instructions for creating a cluster: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/ +.. _flannel: https://github.com/coreos/flannel +.. _networking documentation: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network +.. _quick start guide: https://www.kubeflow.org/docs/started/getting-started/ + +.. _Eigen: https://hub.docker.com/r/clearlinux/stacks-dlaas-oss/ +.. _Intel MKL-DNN: https://hub.docker.com/r/clearlinux/stacks-dlaas-mkl/ + +.. _release notes: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs + +.. _ksonnet registries for deploying TFJobs: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob + + +.. _Kubernetes logging: https://kubernetes.io/docs/concepts/cluster-administration/logging/ + +.. _benchmark results: https://clearlinux.org/stacks/deep-learning-reference-stack diff --git a/source/clear-linux/tutorials/kubernetes.rst b/source/clear-linux/tutorials/kubernetes.rst index 794e92e6..5f491ad6 100644 --- a/source/clear-linux/tutorials/kubernetes.rst +++ b/source/clear-linux/tutorials/kubernetes.rst @@ -4,23 +4,19 @@ Run Kubernetes\* on |CL-ATTR| ############################# This tutorial describes how to install, configure, and run the -`Kubernetes container orchestration system`_ on |CL-ATTR| using different -container engines and runtimes. +`Kubernetes container orchestration system`_ on |CL-ATTR| using CRI+O and +kata-runtime. Kubernetes\* is an open source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. -Runc and Kata Containers\* kata-runtime adhere to :abbr:`OCI (Open Container Initiative*)` -guidelines and work seamlessly with Kubernetes. `Kata Containers`_ provide -strong isolation for untrusted workloads or multi-tenant scenarios. Runc and -Kata Containers can be allocated on a per-pod basis so you can mix and match -both on the same host to suit your needs. - -This tutorial describes the following combinations: - -* Kubernetes with Docker and runc -* Kubernetes with CRI-O and kata-runtime +Kata Containers\* kata-runtime adheres to +:abbr:`OCI (Open Container Initiative*)` guidelines and work seamlessly with +Kubernetes. `Kata Containers`_ provide strong isolation for untrusted +workloads or multi-tenant scenarios. Kata Containers can be +allocated on a per-pod basis so you can mix and match both on the same host +to suit your needs. Prerequisites ************* @@ -40,13 +36,12 @@ Before you install any new packages, update |CL| with the following command: Install Kubernetes and CRI runtimes *********************************** -Kubernetes and a set of supported :abbr:`CRI (Container Runtime Interface)` -runtimes are included in the `cloud-native-basic`_ bundle. To install the -framework, enter the following command: +Kubernetes, a set of supported :abbr:`CRI (Container Runtime Interface)` +runtimes, and networking plugins, are included in the `cloud-native-basic`_ and `containers-basic`_ bundles. To install this framework, enter the following command: .. code-block:: bash - sudo swupd bundle-add cloud-native-basic + sudo swupd bundle-add cloud-native-basic containers-basic Configure Kubernetes ******************** @@ -105,7 +100,7 @@ deployment and your security needs. On systems with limited resources, some performance degradation may be observed while swap is disabled. -#. Switch to root to modify `hostname`: +#. Switch to root to modify the `hosts` file: .. code-block:: bash @@ -127,49 +122,7 @@ deployment and your security needs. Configure and run Kubernetes **************************** -This section describes how to configure and run Kubernetes with: - -* Docker and runc -* CRI-O and kata-runtime - -Configure and run Docker + runc -=============================== - -#. Enable the Docker service: - - .. code-block:: bash - - sudo systemctl enable docker.service - -#. Create (or edit if it exists) the file - :file:`/etc/systemd/system/docker.service.d/51-runtime.conf` and include the following lines: - - .. code-block:: bash - - [Service] - Environment="DOCKER_DEFAULT_RUNTIME=--default-runtime runc" - -#. Create (or edit if it exists) the file :file:`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and include the following lines: - - .. code-block:: bash - - [Service] - Environment="KUBELET_EXTRA_ARGS=" - -#. Enter the commands: - - .. code-block:: bash - - sudo systemctl daemon-reload - sudo systemctl restart docker - sudo systemctl restart kubelet - -#. Initialize the master control plane with the command: - - .. code-block:: bash - - sudo kubeadm init --ignore-preflight-errors=SystemVerification - +This section describes how to configure and run Kubernetes with CRI-O and kata-runtime. Configure and run CRI-O + kata-runtime ====================================== @@ -211,45 +164,52 @@ If you choose the `flannel` add-on, then you must add the following to the --pod-network-cidr 10.244.0.0/16 -If you are using CRI-O and `flannel` and you want to use Kata Containers, edit -the :file:`/etc/crio/crio.conf` file to add: +If you are using CRI-O and `flannel` and you want to use Kata Containers, edit the :file:`/etc/crio/crio.conf` file to add: .. code-block:: bash [crio.runtime] manage_network_ns_lifecycle = true -**Notes about Weave Net add-on** +.. TODO: Should Weave and Calico be mentioned below, or is Flannel enought? -If you choose the `Weave Net` add-on, then you must make the following changes -because it installs itself in the :file:`/opt/cni/bin` directory. +Create a symlink for the network overlays: -If you are using Docker and `Weave Net`, edit the :file:`kubeadm.conf` file to -add: +.. code-block:: bash -.. code-block:: bash + sudo ln -s /usr/libexec/cni /opt/cni/bin - Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin" +.. note:: -If you are using CRI-O and `Weave Net`, you must complete the following steps. + |CL| installs CNI plugins that are part of the `cloud-native-basic` bundle to :file:`/usr/libexec/cni`. The directory is required because `swupd verify` uses it, if necessary, to repair a system to a known good state. -#. Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from: +.. TODO: Verify whether to omit this section per mythi, eadamsintel. - .. code-block:: bash +.. **Notes about Weave Net add-on** - plugin_dir = "/usr/libexec/cni/" +.. If you choose the `Weave Net` add-on, then you must make the following +.. changes because it installs itself in the :file:`/opt/cni/bin` directory. - to: +.. For using CRI-O and `Weave Net`, you must complete the following +.. steps. - .. code-block:: bash +.. #. Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from: - plugin_dir = "/opt/cni/bin" +.. .. code-block:: bash -#. Add the `loopback` CNI plugin to the plugin path with the command: +.. plugin_dir = "/usr/libexec/cni/" - .. code-block:: bash +.. to: - sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback +.. .. code-block:: bash + +.. plugin_dir = "/opt/cni/bin" + +.. #. Add the `loopback` CNI plugin to the plugin path with the command: + +.. .. code-block:: bash + +.. sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback Use your cluster @@ -257,8 +217,8 @@ Use your cluster Once your master control plane is successfully initialized, instructions on how to use your cluster and its *IP*, *token*, and *hash* values are -displayed. It is important that you record the cluster values because they are -needed when joining worker nodes to the cluster. Some values have a valid +displayed. It is important that you record the cluster values because they +are needed when joining worker nodes to the cluster. Some values have a valid period. The values are presented in a format similar to: .. code-block:: bash @@ -268,10 +228,10 @@ period. The values are presented in a format similar to: **Congratulations!** -You've successfully installed and set up Kubernetes in |CL| using Docker and -runc or CRI-O and kata-runtime. You are now ready to follow on-screen -instructions to deploy a pod network to the cluster and join worker nodes -with the displayed token and IP information. +You've successfully installed and set up Kubernetes in |CL| using CRI-O and +kata-runtime. You are now ready to follow on-screen instructions to deploy a +pod network to the cluster and join worker nodes with the displayed token +and IP information. Related topics ************** @@ -314,8 +274,7 @@ Proxy configuration (optional) ****************************** If you use a proxy server, you must set your proxy environment variables and -create an appropriate proxy configuration file for both CRI-O and Docker -services. Consult your IT department if you are behind a corporate proxy for +create an appropriate proxy configuration file for both CRI-O services. Consult your IT department if you are behind a corporate proxy for the appropriate values. Ensure that your local IP is **explicitly included** in the environment variable *NO_PROXY*. (Setting *localhost* is not enough.) @@ -324,7 +283,7 @@ commands as a shell script to configure all of these services in one step: .. code-block:: bash - services=('crio' 'docker') + services=('crio') for s in "${services[@]}"; do sudo mkdir -p "/etc/systemd/system/${s}.service.d/" cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf" @@ -383,9 +342,6 @@ Troubleshooting .. code-block:: bash - /* Kubernetes with Docker + runc */ - sudo -E kubeadm init --ignore-preflight-errors=SystemVerification - /* Kubernetes with CRI-O + kata-runtime */ sudo -E kubeadm init --cri-socket=/run/crio/crio.sock @@ -396,6 +352,8 @@ Troubleshooting .. _Software Update documentation: https://clearlinux.org/documentation/clear-linux/concepts/swupd-about#updating +.. _containers-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/containers-basic + .. _cloud-native-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/cloud-native-basic .. _preflight check: https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/#preflight-checks diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index e5d51254..24f3a3d7 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -31,3 +31,4 @@ Explore our tutorials to discover what you can do with |CL|! kata_migration kubernetes greengrass + dlrs diff --git a/source/substitutions.txt b/source/substitutions.txt index 7a156671..24330b4c 100644 --- a/source/substitutions.txt +++ b/source/substitutions.txt @@ -1,12 +1,10 @@ .. |CL| replace:: Clear Linux OS -.. |CLOSIA| replace:: Clear Linux* OS - .. |CL-ATTR| replace:: Clear Linux* OS .. |CL-PRJ| replace:: Clear Linux* project .. |CC| replace:: Clear Containers -.. |CAUTION-BACKUP-USB| replace:: +.. |CAUTION-BACKUP-USB| replace:: Burning an image formats the USB drive, thus destroying all existing content. Backup your data before proceeding.