diff --git a/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md b/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md new file mode 100644 index 00000000..fb6e16a1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md @@ -0,0 +1,20 @@ +--- +name: Identify 'Guides' Type by Role -- For Guides only. +about: For new content in 'Guides', select 1 of 3 categories to which it belongs. + +--- + +New pages will appear here: https://clearlinux.org/documentation/clear-linux/guides + +1. Enter an 'x' in the category for the new "guide": +* [ ] Basics +* [ ] Developer +* [ ] Administrator + +Complete the field below, following the colon, that matches option selected above: + +I am a Clear Linux Beginner (Basics). I want to learn how to: + +I am a Clear Linux Developer. I want to learn how to: + +I am a Clear Linux Administrator. I want to learn how to: diff --git a/guides_feedback b/guides_feedback new file mode 100644 index 00000000..08ef7692 --- /dev/null +++ b/guides_feedback @@ -0,0 +1,12 @@ +1. Review existing 'guides' page: https://clearlinux.org/documentation/clear-linux/guides + +2. If you wish to add new content, use one of the following categories. Fill in the phrase after the colon, +based on the role to which the task applies. + +I am a Clear Linux Beginner, and I want to learn how to: + +I am a Clear Linux Developer, and I want to learn how to: + +I am a Clear Linux Administrator, and I want to learn how to: + + diff --git a/source/clear-linux/clear-linux.rst b/source/clear-linux/clear-linux.rst index 2c4ffe42..b0c4258a 100644 --- a/source/clear-linux/clear-linux.rst +++ b/source/clear-linux/clear-linux.rst @@ -3,16 +3,23 @@ Clear Linux\* Project for Intel® Architecture ############################################# -Welcome to the |CLOSIA| documentation pages. Our documentation is split into -five sections: Get started, Concepts, Guides, Tutorials, and Reference. -Under :ref:`get-started`, you can find information about installing |CL| on -bare metal, in a virtual environment, or as a live image on a USB stick. Under -the :ref:`concepts` section, you can find detailed technical information about -the |CL| features. The :ref:`guides` section contains step-by-step -instructions to complete common tasks. The :ref:`tutorials` section contains -step-by-step instructions to complete the installation and configuration of -tools needed for a specific use case. Lastly, the :ref:`reference` section -contains information providing additional context or details. +Welcome to the |CLOSIA| documentation pages. Our documentation is divided +into five sections: + +* :ref:`get-started` Information about installing |CL| on + bare metal, in a virtual environment, or as a live image on a USB stick + +* :ref:`concepts` Detailed technical information about + |CL| features and what differentiates it from other |CL| distros + +* :ref:`guides` Step-by-step instructions on how to complete + common tasks that help you leverage |CL| native features + +* :ref:`tutorials` Step-by-step instructions for applying specific use + cases with |CL| that often involve third-party tools + +* :ref:`reference` Technical descriptions that provide context or + important details about |CL| .. toctree:: :maxdepth: 2 diff --git a/source/clear-linux/concepts/autospec-about.rst b/source/clear-linux/concepts/autospec-about.rst new file mode 100644 index 00000000..6b1c3bc3 --- /dev/null +++ b/source/clear-linux/concepts/autospec-about.rst @@ -0,0 +1,112 @@ +.. _autospec-about: + +Autospec +######## + +.. _incl-autospec-overview: + +Overview +******** + +Whereas a standard RPM build process using ``rpmbuild`` requires a tarball +and ``spec`` file to start, ``autospec`` only requires a tarball and package +name. ``autospec`` analyzes the source code and :file:`Makefile` information +in order to generate a ``spec`` file for you. Although not required, you can +influence ``autospec`` by providing control files. + +.. code-block:: console + + buildreq_add + buildreq__ban + pkgconfig_add + pkgconfig_ban + requires_add + requires_ban + options.conf + build_pattern + +These files should be located in same directory as the resulting ``spec`` +file. + +.. note:: + + For a comprehensive list of control files, view the `autospec readme`_. + +.. _incl-autospec-overview-end: + +Control files are explained in Table 1. + +.. list-table:: **Table 1. Control Files** + :widths: 20 80 + :header-rows: 1 + + * - Filename + - Description + * - buildreq_add + - Each line in the file provides the name of a package to add as a + build dependency to the ``spec``. + * - buildreq_ban + - Each line in the file is a build dependency that under no + circumstance should be automatically added to the build dependencies. + This is useful to block automatic configuration routines adding + undesired functionality, or to omit any automatically discovered + dependencies during tarball scanning. + * - pkgconfig_add + - Each line in the file is assumed to be a pkgconfig() build + dependency. Add the pkg-config names here, as ``autospec`` will + automatically transform the names into their ``pkgconfig($name)`` + style when generating the ``spec``. + * - pkgconfig_ban + - Each line in this file is a pkgconfig() build dependency that should + not be added automatically to the build, much the same as + `` buildreq_ban``. As with ``pkgconfig_add``, these names are + automatically transformed by ``autospec`` into their correct + ``pkgconfig($name))`` style. + * - requires_add + - Each line in the file provides the name of a package to add as a + runtime dependency to the ``spec``. + * - requires_ban + - Each line in the file is a runtime dependency that under no + circumstance should be automatically added to the runtime + dependencies. This is useful to block automatic configuration + routines adding undesired functionality, or to omit any automatically + discovered dependencies during tarball scanning. + * - build_pattern + - In certain situations, the automatically detected build pattern may + not work for the given package. This one line file allows you to + override the build pattern that ``autospec`` will use. + * - options.conf + - Further control of the build can be achieved through the use of the + ``options.conf`` file. If this file does not exist it is created by + autospec with default values. If certain deprecated configuration + files exists autospec will use the value indicated by those files and + remove them. + +How autospec works +****************** + +Autospec attempts to infer the requirements of the ``spec`` file. If +autospec infers correctly, the control files (Table 1) will automatically +correct the build requirements. The control files are used to influence +the ``spec`` file generation. + +#. The :command:`make autospec` command generates a ``spec`` file from the + control files. + +#. ``autospec`` creates a ``build root`` with ``mock`` config. + +#. ``autospec`` attempts to build an RPM from the generated ``spec`` file. + +#. ``autospec`` detects any missed declarations in the ``spec`` file. + +.. note:: + + * If there are missed declarations, ``autospec`` creates another ``mock`` + ``chroot`` and starts building again at Step 1. + * If a build error occurs, ``autospec`` stops for user inspection. + * If no build errors occur, RPM packages are successfully built. + +``autospec`` continues to rebuild the package, based on new information +discovered from build failures until it has a valid ``spec`` file. + +.. _autospec readme: https://github.com/clearlinux/autospec diff --git a/source/clear-linux/concepts/concepts.rst b/source/clear-linux/concepts/concepts.rst index 612646b1..3f78eda3 100644 --- a/source/clear-linux/concepts/concepts.rst +++ b/source/clear-linux/concepts/concepts.rst @@ -13,5 +13,7 @@ details relevant to the |CL| features. swupd-about mixer-about bundles-about + autospec-about restart security + telemetry-about diff --git a/source/clear-linux/concepts/figures/telemetry-about-1.png b/source/clear-linux/concepts/figures/telemetry-about-1.png new file mode 100644 index 00000000..37463254 Binary files /dev/null and b/source/clear-linux/concepts/figures/telemetry-about-1.png differ diff --git a/source/clear-linux/concepts/restart.rst b/source/clear-linux/concepts/restart.rst index 69a39964..f3faabb2 100644 --- a/source/clear-linux/concepts/restart.rst +++ b/source/clear-linux/concepts/restart.rst @@ -6,7 +6,8 @@ Restart system services after an OS update The software life cycle describes how software is created, developed, and deployed, and includes how to replace or update software. A good OS provides tools for the entire software life cycle. These tools must include -ways to remove software components properly when replaced with something else. +ways to remove software components properly when replaced with something +else. Most of the work on software update code in |CL| was focused on adding new software to the system. We recommended that users reboot their system once in @@ -32,9 +33,9 @@ solutions such as the following: * Ask the user to restart the OS. Both solutions are acceptable for many OSes. However, |CL| updates software -automatically and users do not see notices from the updater unless they review -the journal. |CL| requires a completely different solution, with the following -requirements: +automatically and users do not see notices from the updater unless they +review the journal. |CL| requires a completely different solution, with the +following requirements: * Eliminate the guesswork about what to restart and under what circumstances. * Cannot restart everything. Many service daemons do not support an automatic @@ -81,7 +82,7 @@ Figure 1: Invoke :command:`clr-service-restart`. :command:`clr-service-restart` implements a whitelist to identify which daemons can be restarted. The system administrator can customize the default -|CL| OS whitelist using :option:`allow` or :option:`disallow` options for +|CL| OS whitelist using *allow* or *disallow* options for restarting system services. When a software update occurs, :command:`clr-service-restart` consults the whitelist to see if a service daemon is allowed to be restarted or not. See the options section for @@ -91,32 +92,33 @@ details. Options for clr-service-restart ******************************* -The :option:`allow` option identifies a daemon to restart after an OS software +The *allow* option identifies a daemon to restart after an OS software update. The :command:`clr-service-restart` daemon creates a symlink in :file:`/etc/clr-service-restart` as a record. The example below tells -:command:`clr-service-restart` to restart the :option:`tallow` daemon after an +:command:`clr-service-restart` to restart the *tallow* daemon after an OS software update. .. code-block:: bash sudo clr-service-restart allow tallow.service -The :option:`disallow` option tells :command:`clr-service-restart` not to +The *disallow* option tells :command:`clr-service-restart` not to restart the specified daemon even if the OS defaults permit the daemon to be restarted. The :command:`clr-service-restart` daemon creates a symlink in -:file:`/etc/clr-service-restart` that points to :file:`/dev/null` as a record. -The example below tells :command:`clr-service-restart` not to restart the -:option:`rngd` daemon after an OS software update. +:file:`/etc/clr-service-restart` that points to :file:`/dev/null` as a +record. The example below tells :command:`clr-service-restart` not to +restart the *rngd* daemon after an OS software update. .. code-block:: bash sudo clr-service-restart disallow rngd -The :option:`default` option makes :command:`clr-service-restart` revert back -to the OS defaults and delete any symlink in :file:`/etc/clr-service-restart`. -The example below tells :command:`clr-service-restart` to restart -:option:`rngd` automatically again, because :option:`rngd` is whitelisted for -automatic service restarts by default in |CL|. +The *default* option makes :command:`clr-service-restart` revert back +to the OS defaults and delete any symlink +in :file:`/etc/clr-service-restart`. The example below +tells :command:`clr-service-restart` to restart *rngd* automatically again, +because *rngd* is whitelisted for automatic service restarts by default +in |CL|. .. code-block:: bash @@ -132,23 +134,24 @@ services are restarted after an OS software update. To monitor :command:`clr-service-restart`, use one or both options described below. -:option:`-n` +.. option:: -n -This option makes :command:`clr-service-restart` perform no restarts. Instead -it displays the services that could potentially be restarted. When used, -:command:`clr-service-restart` outputs a list of messages showing: + This option makes :command:`clr-service-restart` perform no restarts. + Instead it displays the services that could potentially be restarted. + When used, :command:`clr-service-restart` outputs a list of messages + showing: -* Which service needs a restart. -* What unit it is. -* Why it needs a restart. -* Which command is required to restart the unit. + * Which service needs a restart. + * What unit it is. + * Why it needs a restart. + * Which command is required to restart the unit. -:option:`-a` +.. option:: -a -This option makes :command:`clr-service-restart` consider all system services, -not only the ones that are whitelisted. Because the default whitelist in |CL| -is relatively short, you can use this option to restart all impacted services -when you log in on the system. + This option makes :command:`clr-service-restart` consider all system + services, not only the ones that are whitelisted. Because the default + whitelist in |CL| is relatively short, you can use this option to + restart all impacted services when you log in on the system. If you pass both options (:option:`-a` and :option:`-n`), :command:`clr-service-restart` displays a complete list of system services @@ -166,12 +169,14 @@ telemetry record and sends it to the optional |CL| telemetry service if both conditions below are met: * If a unit fails to automatically restart after an OS update. -* If that unit resides in the system location :file:`/usr/lib/systemd/system`. +* If that unit resides in the system + location :file:`/usr/lib/systemd/system`. If you do not install the |CL| telemetrics bundle, the data is discarded. If you install the telemetrics bundle and you opt to send telemetry, then the -system unit name is sent to the |CL| telemetry service. We evaluate the report -and update the whitelist to remove services that are not safe to restart. +system unit name is sent to the |CL| telemetry service. We evaluate the +report and update the whitelist to remove services that are not safe to +restart. Conclusion ********** diff --git a/source/clear-linux/concepts/swupd-about.rst b/source/clear-linux/concepts/swupd-about.rst index 3f4b8452..167cc215 100644 --- a/source/clear-linux/concepts/swupd-about.rst +++ b/source/clear-linux/concepts/swupd-about.rst @@ -123,13 +123,13 @@ and core update take merely seconds. While we realize our definition of bundles makes sense to us, data center operators may have special needs and ideas. Therefore, we provide a -:ref:`mixer`. This tool allows users to customize and add bundles +:ref:`mixer tool `. This tool allows users to customize and add bundles or even add their own software, while keeping the operating system and its updates as the basis. Using this tool, system administrators can focus on the custom pieces their deployments require while staying on a controlled update stream. -To learn how to run an update of your system, visit our :ref:`update` page. +To learn how to run an update of your system, visit our :ref:`using swupd ` page. .. [1] The software update technology for Clear Linux* OS for Intel Architecture was first presented at the Linux Plumbers conference in 2012. diff --git a/source/clear-linux/concepts/telemetry-about.rst b/source/clear-linux/concepts/telemetry-about.rst new file mode 100644 index 00000000..b37e6d90 --- /dev/null +++ b/source/clear-linux/concepts/telemetry-about.rst @@ -0,0 +1,82 @@ +.. _telemetry-about: + +Telemetrics +########### + +One of the key features of |CLOSIA| is telemetry, which is used to +monitor system health. Telemetry enables developers to observe and proactively +address issues before end users are impacted. + +*Telemetrics* is a combination word made from: + +* *Telemetry* which is sensing and reporting data. +* *Analytics* which is using visualization and statistical inferencing to make + sense of the reported data. + +|CL| telemetry reports system-level debug/crash information using specialized probes. The +probes monitor system tasks such as :abbr:`swupd (software updater)`, kernel +oops, machine error checks, and BIOS error report table for unhandled hardware +failures. Telemetry enables real-time issue reporting to allow system +developers to quickly focus on an issue and monitor corrective actions. + +|CL| telemetry is fully customizable and can be used during software development +for debugging purposes. You can use **libtelemetry** in your code to create custom +telemetry records. You can also use **telem-record-gen** in script files or call +it from another program. + +Architecture +************ + +|CL| telemetry has two fundamental components, which are shown in figure 1: + +* Client: generates and delivers records to the backend server via the network. +* Backend: captures records sent from the client and displays the cumulative + content through a specialized interface. + + .. note:: + + If you want to capture your own records for analysis, you must set up + your own backend server. + +.. figure:: figures/telemetry-about-1.png + :scale: 75% + :alt: Clear Linux Telemetry Architecture. + + Figure 1: Clear Linux Telemetry Architecture. + +The telemetry client provides the front end of a complete telemetrics solution +and includes the following components: + +* **telemd**, a daemon that prepares the records to send to a telemetrics server or + spools the records on disk in case it cannot successfully deliver them. +* Probes that collect specific types of data from the operating system. +* **libtelemetry**, that telemetrics probes use to create telemetrics records and + send them to the telemd daemon for further processing. + +The telemetry backend provides the server-side component of a complete telemetrics solution and +consists of: + +* Nginx web server. +* Two Flask apps: + + * Collector, an ingestion web app for records received from telemetrics-client probes. + * TelemetryUI, a web app that exposes several views to visualize the telemetry data + and also provides a REST API to perform queries. + +* PostgreSQL as the underlying database server. + +The default telemetry backend server reports back to the |CL| development team +and is not viewable outside the Intel firewall. If you want to collect your +own records, then you must set up your own telemetry backend server. + +Next steps +********** + +To put this concept into practice, see the following resources: + +* :ref:`telemetry-enable` +* :ref:`telemetry-backend` +* `Telemetry feature description`_ + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst index 46a492f6..00c8cbe7 100644 --- a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst @@ -1,7 +1,7 @@ .. _bare-metal-install: -Install Clear Linux OS on bare metal -#################################### +Install Clear Linux OS on bare metal (automatic) +################################################ These instructions guide you through the installation of |CLOSIA| on bare metal using a bootable USB drive. @@ -44,7 +44,7 @@ Follow these steps to install |CL| on the target system: #. Power on the system. -#. Open the system BIOS setup menu, normally by pressing the :kbd:`F2` key. +#. Open the system BIOS setup menu by pressing the :kbd:`F2` key. Your BIOS setup menu entry point may vary. #. In the setup menu, enable the UEFI boot and set the USB drive as the first diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst index 99e48fae..8f750061 100644 --- a/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst @@ -1,7 +1,7 @@ .. _bare-metal-manual-install: -Bare metal manual installation guide -#################################### +Install Clear Linux OS on bare metal (manual) +############################################# This section contains the steps for a |CL| manual installation. It picks up where the :ref:`bare-metal-install` left off. diff --git a/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst index f646defa..0c5eb624 100755 --- a/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst @@ -1,7 +1,7 @@ .. _cgdisk-manual-install: Create partitions for Clear Linux\* using CGDISK -############################################### +################################################ As part of the |CL| manual installation processThese instructions guide you through the initial setup of your hard drive partitions using the :command:`cgdisk` utility . If you do not wish to continue creating your own @@ -291,4 +291,4 @@ partitions and select to format them. https://clearlinux.org/features/software-update .. _`Linux partitioning scheme`: - https://wiki.archlinux.org/index.php/partitioning#Partition_scheme \ No newline at end of file + https://wiki.archlinux.org/index.php/partitioning#Partition_scheme diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png index 7d366096..0a0b3e25 100644 Binary files a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png index ed43c853..4b7fa392 100644 Binary files a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png index 91a2d13d..3e73dc50 100644 Binary files a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png differ diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst index 76a551c5..4767c26a 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-mac` * :ref:`bootable-usb-windows` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-linux.rst :Start-after: verify-linux: diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst index 29ebe375..2aab17b2 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-linux` * :ref:`bootable-usb-windows` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-mac.rst :start-after: verify-mac: diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst index 424c7ebb..d4dbd75e 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-mac` * :ref:`bootable-usb-linux` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-windows.rst :Start-after: verify-windows: diff --git a/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png b/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png new file mode 100644 index 00000000..0b0ee7f6 Binary files /dev/null and b/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png differ diff --git a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst index 96819b32..d090222b 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst @@ -64,7 +64,7 @@ For additional help, see the `VMware Workstation Player guide`_. Clear Linux image types *********************** -.. include:: ../../guides/maintenance/image-types.rst +.. include:: ../../reference/image-types.rst :Start-after: image-types-content: Download the latest Clear Linux VMware image diff --git a/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst b/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst index 5bd5afcd..06dc38ee 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst @@ -74,7 +74,7 @@ For additional help, see the `VMware Workstation Player guide`_. Clear Linux image types *********************** -.. include:: ../../guides/maintenance/image-types.rst +.. include:: ../../reference/image-types.rst :Start-after: image-types-content: diff --git a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst index 92dcf185..e9f35d57 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst @@ -1,36 +1,33 @@ .. _vmware-esxi-preconfigured-cl-image: -Run preconfigured Clear Linux image as a VMware\* ESXi guest OS -############################################################### +Run preconfigured Clear Linux\* image as a VMware\* ESXi guest OS +################################################################# `VMware ESXi`_ is a type 1 bare-metal hypervisor which runs directly on top of server hardware. With VMware ESXi, you can create, configure, manage, -and run |CLOSIA| virtual machines in the cloud. +and run |CLOSIA| virtual machines at scale. This section shows you how to deploy a preconfigured |CL| VMware image on -VMware ESXi 6.5 Update 1 using these steps: +VMware ESXi 6.5. -#. Download the latest Clear Linux VMware image. -#. Verify the integrity of the |CL| image. -#. Uncompress the |CL| image. -#. Upload the Clear Linux image to the VMware server. -#. Convert the Clear Linux image to an ESXi-supported format. -#. Create and configure a new VM. -#. Power on the VM and boot |CL|. - -If you would prefer to perform a fresh installation of |CL| into a new VMware -ESXi :abbr:`VM (Virtual Machine)` instead, see :ref:`vmware-esxi-install-cl`. +If you would prefer to perform a manual installation of |CL| into a new +VMware ESXi :abbr:`VM (Virtual Machine)` instead, see +:ref:`vmware-esxi-install-cl`. Visit :ref:`image-types` to learn more about +the available images. .. note:: - VMware also offers a type 2 hypervisor called `VMware Workstation Player`_ which is designed for the desktop environment. + VMware also offers a type 2 hypervisor called `VMware Workstation Player` + which is designed for the desktop environment. + See :ref:`vmw-player-preconf` or see :ref:`vmw-player`. -Visit :ref:`image-types` to learn more about the available images. +.. contents:: :local: + :depth: 2 Download the latest Clear Linux VMware image ******************************************** -Get the latest |CL| VMware image from the `image`_ repository. +Get the latest |CL| VMware prebuilt image from the `image`_ repository. Look for :file:`clear-[version number]-vmware.vmdk.xz`. You can also use this command: @@ -50,59 +47,87 @@ For alternative instructions on other operating systems, see: Upload the Clear Linux image to the VMware server ************************************************* -#. Connect to the VMware server and login to an account with sufficient - permission to create and manage VMs. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Storage`. - See Figure 1. -#. Under the :guilabel:`Datastores` tab, click - the :guilabel:`Datastore browser` button. +Once the |CL| VMware prebuilt image has been downloaded and +uncompressed on your local system, it must be uploaded to a datastore +on the VMware ESXi server. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-1.png +The steps in this section can also be referenced from the `VMware documentation on Using the Datastore File Browser`_ + +#. Connect to the VMware ESXi server and login to an account with sufficient + permission to create and manage VMs. + +#. Under the :guilabel:`Navigator` window on the left side, + select :guilabel:`Storage`. + See Figure 1 + +#. Under the :guilabel:`Datastores` tab, click + the :guilabel:`Datastore browser` button. + + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-1.png :scale: 100 % :alt: VMware ESXi - Navigator > Storage Figure 1: VMware ESXi - Navigator > Storage -#. Click the :guilabel:`Create directory` button and name the directory - `Clear Linux VM`. See Figure 2. +#. Click the :guilabel:`Create directory` button and name the directory + `Clear Linux VM`. See Figure 2. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-2.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-2.png :scale: 100 % :alt: VMware ESXi - Datastore > Create directory Figure 2: VMware ESXi - Datastore > Create directory -#. Select the newly-created directory and click the :guilabel:`Upload` - button. See Figure 3. +#. Select the newly-created directory and click the :guilabel:`Upload` + button. See Figure 3. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-3.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-3.png :scale: 100 % :alt: VMware ESXi - Datastore > Upload VMware image Figure 3: VMware ESXi - Datastore > Upload VMware image -#. Select the uncompressed |CL| VMware image file - :file:`clear-[version number]-vmware.vmdk` and upload it. +#. Select the uncompressed |CL| VMware image file + :file:`clear-[version number]-vmware.vmdk` and upload it. Convert the Clear Linux image to an ESXi-supported format ********************************************************* -#. SSH into the VMware server and login to an account with root privileges. -#. Locate the uploaded image, which is typically found in `/vmfs/volumes/datastore1`. -#. Use the :command:`vmkfstools` command to perform the conversion, as - shown below: +Once the |CL| VMware prebuilt image has been uploaded to the VMware ESXi +datastore, it must be converted to a format for usable with VMware's ESXi +hypervisor. - .. code-block:: console +The steps in this section can also be referenced from the `VMware documentation on Cloning and converting virtual machine disks with vmkfstools`_ - # vmkfstools -i clear-[version number]-vmware.vmdk -d zeroedthick clear-[version number]-esxi.vmdk +#. SSH into the `vSphere Management Assistant`_ appliance that is managing + the ESXi host or connect to the vSphere hosting using the `vSphere CLI`_. + + .. note:: + + If there is no :abbr:`vMA (vSphere Management Assistant)` appliance or :abbr:`vCLI (vSphere CLI)` configured and available, + you can temporarily enable SSH directly on the ESXi host by referencing + the `VMware documentation on Enable the Secure Shell (SSH)`_ . - Two files should result from this: + As a security best practice, remember to disable SSH access after following the steps in this section. - * :file:`clear-[version number]-esxi-flat.vmdk` - * :file:`clear-[version number]-esxi.vmdk` - The :file:`clear-[version number]-esxi.vmdk` file will be used in the - next section when you create a new VM. +#. Locate the uploaded image, which is typically found in + :file:`/vmfs/volumes/datastore1`. + +#. Use the :command:`vmkfstools` command to perform the conversion, as + shown below: + + .. code-block:: console + + vmkfstools -i clear-[version number]-vmware.vmdk -d zeroedthick clear-[version number]-esxi.vmdk + + Two files should result from this: + + * :file:`clear-[version number]-esxi-flat.vmdk` + * :file:`clear-[version number]-esxi.vmdk` + + The :file:`clear-[version number]-esxi.vmdk` file will be used in the + next section when you create a new VM. Create and configure a new VM ***************************** @@ -111,138 +136,153 @@ In this section, you will create a new VM, configure its basic parameters such as number of CPUs, memory size, and then attach the converted |CL| VMware image. Also, in order to boot |CL|, you must enable UEFI support. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. See Figure 4. +#. Under the :guilabel:`Navigator` window, select + :guilabel:`Virtual Machines`. See Figure 4. -#. In the right window, click the :guilabel:`Create / Register VM` button. +#. In the right window, click the :guilabel:`Create / Register VM` button. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-4.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-4.png :scale: 100 % :alt: VMware ESXi - Navigator > Virtual Machines Figure 4: VMware ESXi - Navigator > Virtual Machines -#. On the :guilabel:`Select creation type` step: +#. On the :guilabel:`Select creation type` step: - #. Select the :guilabel:`Create a new virtual machine` option. See - Figure 5. + #. Select the :guilabel:`Create a new virtual machine` option. See + Figure 5. - #. Click the :guilabel:`Next` button. + #. Click the :guilabel:`Next` button. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-5.png - :scale: 100 % - :alt: VMware ESXi - Create a new virtual machine + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-5.png + :scale: 100 % + :alt: VMware ESXi - Create a new virtual machine - Figure 5: VMware ESXi - Create a new virtual machine + Figure 5: VMware ESXi - Create a new virtual machine -#. On the :guilabel:`Select a name and guest OS` step: +#. On the :guilabel:`Select a name and guest OS` step: - #. Give the new VM a name in the :guilabel:`Name` field. See Figure 6. - #. Set the :guilabel:`Compatability` option to :guilabel:`ESXi 6.5 virtual machine`. - #. Set the :guilabel:`Guest OS family` option to :guilabel:`Linux`. - #. Set the :guilabel:`Guest OS version` option to :guilabel:`Other 3.x or later Linux (64-bit)`. - #. Click the :guilabel:`Next` button. +#. Give the new VM a name in the :guilabel:`Name` field. See Figure 6. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-6.png - :scale: 100 % - :alt: VMware ESXi - Give a name and select guest OS type +#. Set the :guilabel:`Compatability` option to + :guilabel:`ESXi 6.5 virtual machine`. +#. Set the :guilabel:`Guest OS family` option to :guilabel:`Linux`. +#. Set the :guilabel:`Guest OS version` option to + :guilabel:`Other 3.x or later Linux (64-bit)`. +#. Click the :guilabel:`Next` button. - Figure 6: VMware ESXi - Give a name and select guest OS type + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-6.png + :scale: 100 % + :alt: VMware ESXi - Give a name and select guest OS type -#. On the :guilabel:`Select storage` step: + Figure 6: VMware ESXi - Give a name and select guest OS type - #. Accept the default option. - #. Click the :guilabel:`Next` button. +#. On the :guilabel:`Select storage` step: -#. On the :guilabel:`Customize settings` step: + #. Accept the default option. + #. Click the :guilabel:`Next` button. - #. Click the :guilabel:`Virtual Hardware` button. See Figure 7. - #. Expand the :guilabel:`CPU` setting and enable :guilabel:`Hardware virtualization` by - checking :guilabel:`Expose hardware assisted virtualization to the guest OS`. +#. On the :guilabel:`Customize settings` step: - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-7.png - :scale: 100 % - :alt: VMware ESXi - Enable hardware virtualization + #. Click the :guilabel:`Virtual Hardware` button. See Figure 7. + #. Expand the :guilabel:`CPU` setting and enable + :guilabel:`Hardware virtualization` by checking + :guilabel:`Expose hardware assisted virtualization to the guest OS`. - Figure 7: VMware ESXi - Enable hardware virtualization + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-7.png + :scale: 100 % + :alt: VMware ESXi - Enable hardware virtualization - #. Remove the default :guilabel:`Hard drive 1` setting by clicking - the `X` icon on the right side. See Figure 8. + Figure 7: VMware ESXi - Enable hardware virtualization - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-8.png - :scale: 100 % - :alt: VMware ESXi - Remove hard drive + #. Remove the default :guilabel:`Hard drive 1` setting by clicking + the `X` icon on the right side. See Figure 8. - Figure 8: VMware ESXi - Remove hard drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-8.png + :scale: 100 % + :alt: VMware ESXi - Remove hard drive - #. Since a preconfigured image will be used, - the :guilabel:`CD/DVD Drive 1` setting will not be needed. Disable it by unchecking the :guilabel:`Connect` checkbox. See Figure 9. + Figure 8: VMware ESXi - Remove hard drive - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-9.png - :scale: 100 % - :alt: VMware ESXi - Disconnect the CD/DVD drive + #. Since a preconfigured image will be used, + the :guilabel:`CD/DVD Drive 1` setting will not be needed. Disable it + by unchecking the :guilabel:`Connect` checkbox. See Figure 9. - Figure 9: VMware ESXi - Disconnect the CD/DVD drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-9.png + :scale: 100 % + :alt: VMware ESXi - Disconnect the CD/DVD drive - #. Attach the :file:`clear-[version number]-esxi.vmdk` file that was - converted from the preconfigured |CL| VMware image. + Figure 9: VMware ESXi - Disconnect the CD/DVD drive - #. Click the :guilabel:`Add hard disk` button and select the - :guilabel:`Existing hard drive` option. See Figure 10. + #. Attach the :file:`clear-[version number]-esxi.vmdk` file that was + converted from the preconfigured |CL| VMware image. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-10.png - :scale: 100 % - :alt: VMware ESXi - Add an existing hard drive + #. Click the :guilabel:`Add hard disk` button and select the + :guilabel:`Existing hard drive` option. See Figure 10. - Figure 10: VMware ESXi - Add an existing hard drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-10.png + :scale: 100 % + :alt: VMware ESXi - Add an existing hard drive - #. Select the converted :file:`clear-[version number]-esxi.vmdk` - file. Do not use the original unconverted :file:`clear-[version number]-vmware.vmdk` file. See Figure 11. + Figure 10: VMware ESXi - Add an existing hard drive - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-11.png - :scale: 100 % - :alt: VMware ESXi - Select the converted `vmdk` file + #. Select the converted :file:`clear-[version number]-esxi.vmdk` + file. Do not use the original unconverted + :file:`clear-[version number]-vmware.vmdk` file. See Figure 11. - Figure 11: VMware ESXi - Select the converted :file:`clear-[version number]-esxi.vmdk` file + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-11.png + :scale: 100 % + :alt: VMware ESXi - Select the converted `vmdk` file -#. |CL| needs UEFI support in order to boot. Enable UEFI boot support. + Figure 11: VMware ESXi - Select the converted + :file:`clear-[version number]-esxi.vmdk` file - #. Click the :guilabel:`VM Options` button. See Figure 12. - #. Expand the :guilabel:`Boot Options` setting. - #. For the :guilabel:`Firmware` setting, click the drop-down list to - the right of it and select the :guilabel:`EFI` option. +#. |CL| needs UEFI support in order to boot. Enable UEFI boot support. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-12.png - :scale: 100 % - :alt: VMware ESXi - Set boot firmware to EFI + #. Click the :guilabel:`VM Options` button. See Figure 12. + #. Expand the :guilabel:`Boot Options` setting. + #. For the :guilabel:`Firmware` setting, click the drop-down list to + the right of it and select the :guilabel:`EFI` option. - Figure 12: VMware ESXi - Set boot firmware to EFI + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-12.png + :scale: 100 % + :alt: VMware ESXi - Set boot firmware to EFI -#. Click the :guilabel:`Save` button. -#. Click the :guilabel:`Next` button. -#. Click the :guilabel:`Finish` button. + Figure 12: VMware ESXi - Set boot firmware to EFI + +#. Click the :guilabel:`Save` button. +#. Click the :guilabel:`Next` button. +#. Click the :guilabel:`Finish` button. Power on the VM and boot Clear Linux ************************************ After configuring the settings above, power on the VM. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. See Figure 13. -#. In the right window, select the newly-created VM. -#. Click the :guilabel:`Power on` button. -#. Click on the icon representing the VM to bring it into view and maximize - its window. +#. Under the :guilabel:`Navigator` window, select + :guilabel:`Virtual Machines`. See Figure 13. +#. In the right window, select the newly-created VM. +#. Click the :guilabel:`Power on` button. +#. Click on the icon representing the VM to bring it into view and maximize + its window. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-13.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-13.png :scale: 100 % :alt: VMware ESXi - Navigator > Virtual Machines > Power on VM Figure 13: VMware ESXi - Navigator > Virtual Machines > Power on VM Also see: -********* +========= * :ref:`vmware-esxi-install-cl` .. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html +.. _VMware documentation on Using the Datastore File Browser: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.html.hostclient.doc/GUID-7533A767-8396-4844-A3F2-206047D254EA.html +.. _vSphere Management Assistant: https://www.vmware.com/support/developer/vima/ +.. _vSphere CLI: https://www.vmware.com/support/developer/vcli/ +.. _VMware documentation on Cloning and converting virtual machine disks with vmkfstools: https://kb.vmware.com/kb/1028042 +.. _VMware documentation on Enable the Secure Shell (SSH): https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.html.hostclient.doc/GUID-B649CB74-832F-467B-B6A4-8BA67AD5C1F0.html +.. _VMware documentation on General ESXi Security Recommendations: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.security.doc/GUID-B39474AF-6778-499A-B8AB-E973BE6D4899.html .. _VMware Workstation Player: https://www.vmware.com/products/workstation-player.html .. _image: https://download.clearlinux.org/image diff --git a/source/clear-linux/guides/guides.rst b/source/clear-linux/guides/guides.rst index c45065fb..2be109e1 100644 --- a/source/clear-linux/guides/guides.rst +++ b/source/clear-linux/guides/guides.rst @@ -3,6 +3,13 @@ Guides ###### +Our Guides: + +* Provide a critical, fundamental understanding of |CL| features +* Show you how to leverage the full feature set of |CL| +* Enhance your productivity when using |CL| + + The following guides provide step-by-step instructions for tasks that come after completing the |CL| :ref:`installation `. diff --git a/source/clear-linux/guides/maintenance/autospec.rst b/source/clear-linux/guides/maintenance/autospec.rst new file mode 100644 index 00000000..dd4dd2d7 --- /dev/null +++ b/source/clear-linux/guides/maintenance/autospec.rst @@ -0,0 +1,318 @@ +.. _autospec: + +Build RPMs with autospec +######################## + +This guide shows you how to create RPMs with :ref:`autospec ` +, a tool that assists in automated creation and maintenance of RPM packaging +on |CLOSIA|. Additionally, you learn how to use these RPMs to create bundles +with :ref:`mixer `. + +Prerequisites +************* +This guide assumes that you have: + +* Created :ref:`a custom mix ` of |CL| and deployed it to a + to a target device + +* |CL| running on a host machine or virtual environment + + .. note:: + + To install |CL|, see: + + * :ref:`bare-metal-install` + * :ref:`virtual-machine-install` + +Install Clear Linux tooling framework +===================================== + +Our GitHub\* repository provides you with the resources you need +to create and maintain packages. + +#. On your host system, install this developer bundle. + + .. code-block:: bash + + sudo swupd bundle-add os-clr-on-clr + +#. Run this command to download the :file:`user-setup.sh` script. + + .. code-block:: bash + + curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh + +#. Make :file:`user-setup.sh` executable. + + .. code-block:: bash + + chmod +x user-setup.sh + +#. Run the script as an unprivileged user. + + .. code-block:: bash + + ./user-setup.sh + +#. After the script completes, log out and log in again to complete + the setup process. + + The `user-setup script`_ creates a folder called :file:`clearlinux`, which + contains the :file:`Makefile`, :file:`packages`, and :file:`projects` + subfolders. + + The :file:`projects` folder contains the main tools, `autospec` + and `common`, used for making packages in |CL|. + +Create a RPM with autospec +************************** + +.. include:: ../../concepts/autospec-about.rst + :start-after: incl-autospec-overview: + :end-before: incl-autospec-overview-end: + +For a detailed explanation of how ``autospec`` works on |CL|, visit our +:ref:`autospec-about` about page. For a general understanding of how RPMs +work, we recommend visiting the `rpm website`_ or the +`RPM Packaging Guide`_ . + +Building RPMs +============= + +Choose one of the following options to build RPMs and manage source +code: + +* :ref:`build-a-new-rpm` and spec file using ``make autospecnew``. + +* :ref:`build-source-code-with-existing-spec-file` (without changing the + spec file) using ``make build``. + +* :ref:`generate-a-new-spec-file` based on changes in the control files with + ``make autospec``. + +.. _build-a-new-rpm: + +Build a new RPM +=============== + +#. Navigate to the autospec workspace. + + .. code-block:: bash + + cd ~/clearlinux + +#. Enter the command: + + .. code-block:: bash + + make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" + NAME="helloclear" + + .. note:: + + For a local tarball, use for the *URL*: + file:// + +#. If build failures or dependency issues occur, continue below. + Otherwise, skip directly to `copy-rpm-packages-to-mixer`_. + + #. Navigate to the specific package. + + .. code-block:: bash + + cd ~/clearlinux/packages/[package-name] + + #. Respond to the build process output by editing control files to resolve + issues, which may include dependencies or exclusions. + See `autospec readme`_ + + #. Run this command: + + .. code-block:: bash + + make autospec + + Repeat the last two steps above until all errors are resolved and you + complete a successful build. + + Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. + +.. _build-source-code-with-existing-spec-file: + +Build source code with an existing spec file +============================================ + +If you only want to build the RPM using the spec file, use this method. This +method assumes that a spec file already exists. In this example, we run a +``make build`` on the ``dmidecode`` package. + +#. Navigate to the ``dmidecode`` package in clearlinux: + + .. code-block:: bash + + cd ~/clearlinux/packages/dmidecode/ + +#. To download the tarball and build, run the command: + + .. code-block:: bash + + make build + + Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. + +.. _generate-a-new-spec-file: + +Generate a new spec file with a pre-defined package +=================================================== + +In this method, you will modify an existing |CL| package called ``dmidecode`` +to create a custom RPM. You will make a simple change to this package, +change the revision to a new number that is higher than the |CL| OS version, +and rebuild the package. + +#. Navigate to clearlinux: + + .. code-block:: bash + + cd ~/clearlinux + +#. Copy the ``dmidecode`` package. + + .. code-block:: bash + + make clone_dmidecode + +#. Navigate into the *dmidecode* directory: + + .. code-block:: bash + + cd packages/dmidecode + +#. With an editor, open the :file:`excludes` file and add these lines: + + .. code-block:: bash + + /usr/bin/biosdecode + /usr/bin/ownership + /usr/bin/vpddecode + /usr/share/man/man8/biosdecode.8 + /usr/share/man/man8/ownership.8 + /usr/share/man/man8/vpddecode.8 + + .. note:: + + These files aren't needed by dmidecode, so we can remove them without + any issues. + +#. Save the file and exit. + +#. At :file:`~/clearlinux/packages/dmidecode`, build the modified + ``dmidecode`` package: + + .. code-block:: bash + + make autospec + + When the process completes, you will see new RPM packages in the + :file:`results/` folder. + +#. To view the new RPM packages, enter: + + .. code-block:: bash + + ls /clearlinux/packages/dmidecode/results/ + +Add a custom RPM to a mix and deploy to target +********************************************** + +We need a RPM repository to store our custom RPMs. This repository also +includes some metadata that allows programs such as ``yum`` and ``dnf`` to +follow and include any specified dependencies. This architecture enables us +to test custom RPMs before we integrate them in a mix. + +.. note:: + + Assure that you followed the :ref:`mixer` instruction and created + a location for **local RPM packages** using the *--local-rpms* flag + with the command: :command:`mixer init --local-rpms`. If you skipped + this step, return and complete it in :ref:`mixer` before proceeding. + +.. _copy-rpm-packages-to-mixer: + +Copy RPM packages to mixer and build bundle +============================================ + +Transfer the newly generated RPM packages to the ``mixer`` folder so +that it can include them as needed. + +.. note:: + + This guide assumes that you have a web server that hosts ``swupd`` update + content. + +#. Change directory into the mix workspace: + + .. code-block:: bash + + cd ~/mix + +#. Copy the contents from the results folder in the RPM packages to the + :file:`local-rpms` folder in the :file:`mix` folder: + + .. code-block:: bash + + cp ~/clearlinux/packages/dmidecode/results/*x86_64*rpm ~/mix/local-rpms/ + +#. Remove the debuginfo: + + .. code-block:: bash + + rm ~/mix/local-rpms/*debuginfo*x86_64* + +#. Generate the yum repo: + + .. code-block:: bash + + sudo mixer add-rpms + +#. Create a local bundle definition file to include the newly generated RPM + package in your mix. In our example, the ``[bundle-name]`` is + either ``dmidecode`` or ``helloclear``. + + .. code-block:: bash + + mixer bundle edit [bundle-name] + +#. Then add the new bundle to the mix. + + .. code-block:: bash + + mixer bundle add [bundle-name] + +#. Build the bundle and update content. + + .. code-block:: bash + + sudo mixer build all + +#. Log into the target device. + +#. On the target device, update and install the new bundle. + + .. code-block:: bash + + sudo swupd update + + sudo swupd bundle-add [bundle-name] + +**Congratulations!** + +You successfully built a RPM and created a mix with it. + +.. _rpm website: http://rpm.org + +.. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/ + +.. _user-setup script: https://github.com/clearlinux/common/blob/master/user-setup.sh + +.. _autospec readme: https://github.com/clearlinux/autospec diff --git a/source/clear-linux/guides/maintenance/bulk-provision.rst b/source/clear-linux/guides/maintenance/bulk-provision.rst index 60c57e0a..6badd952 100644 --- a/source/clear-linux/guides/maintenance/bulk-provision.rst +++ b/source/clear-linux/guides/maintenance/bulk-provision.rst @@ -92,7 +92,7 @@ Configuration within the web hosting directory of **ICIS**. The following example shows an Ister configuration file: - .. code-block:: json + .. code-block:: none template=http://192.168.1.1:60000/icis/static/ister/ister.json @@ -102,7 +102,7 @@ Configuration kernel parameter value. The following example shows an iPXE boot script with the ``isterconf`` parameter: - .. code-block:: json + .. code-block:: none #!ipxe kernel linux quiet init=/usr/lib/systemd/systemd-bootchart initcall_debug tsc=reliable no_timer_check noreplace-smp rw initrd=initrd isterconf=http://192.168.1.1:60000/icis/static/ister/ister.conf @@ -129,7 +129,7 @@ Configuration in the ``static`` directory within the web hosting directory of **ICIS**. The following example shows one such assignment: - .. code-block:: json + .. code-block:: none # MAC address,role 00:01:02:03:04:05,ciao @@ -138,7 +138,7 @@ Configuration :file:`config.txt` file, a default role for those MAC address may be defined as follows: - .. code-block:: json + .. code-block:: none # MAC address,role default,ciao diff --git a/source/clear-linux/guides/maintenance/developer-workstation.rst b/source/clear-linux/guides/maintenance/developer-workstation.rst new file mode 100644 index 00000000..d408aebf --- /dev/null +++ b/source/clear-linux/guides/maintenance/developer-workstation.rst @@ -0,0 +1,111 @@ +.. _developer-workstation: + +Developer Workstation +##################### + +Overview +******** + +*Developer Workstation* helps you find the :ref:`bundles-about` you need to +start your |CL| development project. + +Before continuing, we recommend that you learn how to use +:ref:`swupd `. Visit our :ref:`swupd-about` page to understand +how |CL| simplifies software versioning compared to other Linux\* +distributions. + +Workstation Setup +================= + +This guide helps you understand the minimum bundles required to get started. +After installing them, you can add more bundles relevant to your use case. +To run any process required for Clear Linux development, you can add the +large bundle :ref:`*os-clr-on-clr* `. However, given how +many packages this bundle contains, you may want instead to deploy a leaner +OS with only those bundles relevant to your project. Developer Workstation +responds to this need. + +Use Table 1, *Clear Linux Developer Profiles*, to identify the *minimum +required bundles* to get started developing based on your role or project. +While your role may not neatly fit in one of these categories, consider using Table 1 as a starting point. + +.. list-table:: **Table 1. Clear Linux Developer Profiles** + :widths: 20, 20, 20, 20 + :header-rows: 1 + + * - Clear Linux Bundle + - *Internet of Things (IoT)* + - *System Administrator* + - *Client/Cloud/Web Developer* + + * - `editors` + - ✓ + - ✓ + - ✓ + + * - `network-basic` + - ✓ + - ✓ + - ✓ + + * - `openssh-server` + - ✓ + - ✓ + - ✓ + + * - `webserver-basic` + - + - ✓ + - ✓ + + * - `application-server` + - + - ✓ + - ✓ + + * - `database-basic` + - + - ✓ + - ✓ + + * - `desktop-autostart` + - ✓ + - ✓ + - ✓ + + * - `dev-utils` + - + - + - ✓ + +`swupd` search +============== + +We recommend trying out :ref:`swupd search `, to learn the +commands to search for and add bundles relevant to your project. + +:ref:`swupd-search` shows you how to: + +* Use `swupd` to search for bundles +* Use `swupd` to add bundles + +Core Concepts +============= + +We recommend that you understand these core concepts in |CL| *before* +developing your project. + +* :ref:`Bundles ` +* :ref:`Software update ` +* :ref:`Mixer ` +* :ref:`Autospec ` + +Other resources for developers +----------------------------------- + +* `Developer Tooling Framework for Clear Linux`_ +* `Clear Linux Bundles`_ + +.. _Clear Linux Bundles: https://github.com/clearlinux/clr-bundles + +.. _Developer Tooling Framework for Clear Linux: https://github.com/clearlinux/common diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst index f497148c..e90c0170 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-linux: @@ -42,6 +42,9 @@ checksum, a warning is displayed with a message indicating the computed checksum does **not** match. Otherwise, the name of the image is printed on the screen followed by `OK`. +For a more in-depth discussion of image verification including checking the +certificate see :ref:`image-content-validation`. + Uncompress the Clear Linux image ******************************** diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst index d37346ef..d94be7b6 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-mac: diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst index 24d7c7fa..03b892fb 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-windows: diff --git a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png index db5a2f3f..a4692c9a 100755 Binary files a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png and b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png differ diff --git a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png new file mode 100644 index 00000000..90b1eb28 Binary files /dev/null and b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png differ diff --git a/source/clear-linux/guides/maintenance/image-types.rst b/source/clear-linux/guides/maintenance/image-types.rst deleted file mode 100644 index e3a119f5..00000000 --- a/source/clear-linux/guides/maintenance/image-types.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. _image-types: - -Clear Linux image types -####################### - -.. _image-types-content: - -|CLOSIA| offers many types of `images`_ for different platforms and environments. - -The naming convention of a |CL| image filename is: - - .. code-block:: console - - clear-[version number]-[image type].[compression type] - -The *[version number]* field specifies the version number. - -The *[image type]* field specifies the type of image and its corresponding -file format. - -The *[compression type]* field specifies the compression type. Two types of -compressions are used: GNU\* zip (*.gz*) and XZ (*.xz*). - -Table 1 lists the currently available images. - -.. list-table:: Table 1: Types of Clear Linux images - :widths: 20, 60 - :header-rows: 1 - - * - Image Type - - Description - * - azure.vhd - - Image for Microsoft\* Azure\* - * - azure-docker.vhd - - Image with Docker\* installed for Microsoft Azure - * - azure-machine-learning.vhd - - Image with the `machine-learning-basic` bundle installed for Microsoft - Azure - * - cloud.img - - Image for cloud deployment such as OpenStack\* - * - containers.img - - Optimized image used by Clear Containers runtime - * - hyperv.vhdx - - Image for Microsoft Hyper-V\* - * - hyperv-mini.vhdx - - Minimal image with fewer bundles for Microsoft Hyper-V - * - installer.img - - Interactive installer image - * - installer.iso - - ISO of the interactive installer image - * - kvm.img - - Image for booting in a simple VM with start_qemu.sh - * - live.img - - Live boot image - * - vmware.vmdk - - Image for VMware\* - -.. _images: https://download.clearlinux.org/image diff --git a/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst b/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst index 7b991faa..a676e615 100644 --- a/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst +++ b/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst @@ -1,126 +1,169 @@ .. _increase-virtual-disk-size: -Increase Clear Linux image virtual disk size -############################################ +Increase virtual disk size of a Clear Linux\* image +################################################### -Prebuilt |CLOSIA| images come in different sizes, ranging from 300 MB to 20 +|CLOSIA| prebuilt images come in different sizes, ranging from 300 MB to 20 GB. This guide describes how to increase the size of your prebuilt |CL| image if you need more capacity. -Determine the prebuilt image size -********************************* +.. contents:: This guide will cover: -There are two methods to find the virtual disk size of your prebuilt |CL| -image. +Determine the partition order and sizes of the prebuilt image +************************************************************* -The first method is to check the config.JSON file of the image, located in the -`releases`_ repository. For example, to find the size of the Hyper-V\* image -version number 20450, follow these steps: +There are two methods to find the order and sizes of partitions virtual disk +of your prebuilt |CL| image. -#. Go to the `releases`_ repository. -#. Drill down into the `20450 > clear > config > image` directory. -#. Open the :file:`hyperv-config.json` file. -#. Locate the `PartitionLayout` key. - The example shows 512 MB for the EFI partition, 32 MB for the swap - partition, and 8 GB for the root partition. +In both examples, the prebuilt Hyper-V image has a disk size of 8.5 GB with / +dev/sda3 being the partition for the root filesystem (/) - .. code-block:: console +Checking :command:`lsblk` on the VM +----------------------------------- - "PartitionLayout" : [ { "disk" : "hyperv.img", - "partition" : 1, - "size" : "512M", - "type" : "EFI" }, - { "disk" : "hyperv.img", - "partition" : 2, - "size" : "32M", - "type" : "swap" }, - { "disk" : "hyperv.img", - "partition" : 3, - "size" : "8G", - "type" : "linux" } ], - -The second method is to boot up your :abbr:`VM (Virtual Machine)` and +The first method is to boot up your :abbr:`VM (Virtual Machine)` and execute the :command:`lsblk` command as shown below: .. code-block:: bash - sudo lsblk + sudo lsblk -An example output: +An example output of the :command:`lsblk` command: .. code-block:: console - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - sda 8:0 0 8.5G 0 disk - ├─sdd1 8:1 0 512M 0 part - ├─sdd2 8:2 0 32M 0 part [SWAP] - └─sdd3 8:3 0 8G 0 part / + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + sda 8:0 0 8.5G 0 disk + ├─sda1 8:1 0 512M 0 part + ├─sda2 8:2 0 32M 0 part [SWAP] + └─sda3 8:3 0 8G 0 part / -In this example, both methods show the prebuilt Hyper-V image is about 8.5 GB. +An example of this can also be seen in Figure 1. + +Checking :file:`config.json` used to build the image +---------------------------------------------------- + +The second method to determine partition to check the :file:`config.json` +file used to create prebuilt image, located in the `releases`_ repository. +For example, to find the size of the Hyper-V\* image version number 20450, +follow these steps: + +#. Go to the `releases`_ repository. +#. Drill down into the `20450 > clear > config > image` directory. +#. Open the :file:`hyperv-config.json` file. +#. Locate the `PartitionLayout` key. + + The example shows 512 MB for the EFI partition, 32 MB for the swap + partition, and 8 GB for the root partition. + + + .. code-block:: console + + "PartitionLayout" : [ { "disk" : "hyperv.img", + "partition" : 1, + "size" : "512M", + "type" : "EFI" }, + { "disk" : "hyperv.img", + "partition" : 2, + "size" : "32M", + "type" : "swap" }, + { "disk" : "hyperv.img", + "partition" : 3, + "size" : "8G", + "type" : "linux" } ], Increase virtual disk size ************************** +Once you have determined the disk and partition to be increased, you are +ready to perform the actual increase of the disk, partition, and filesystem. -To increase the virtual disk size for a prebuilt image, perform the steps below: +Power off VM and increase virtual disk size: +-------------------------------------------- -#. Shut down your VM if it is running. -#. Use an appropriate hypervisor tool to increase the virtual disk size of - your VM. -#. Power up the VM. -#. Log in to an account with root privileges. -#. Open a terminal emulator. -#. Add the |CL| `storage-utils` bundle to install the `parted` and - `resize2fs` tools. +To increase the virtual disk size for a prebuilt image, perform the steps +below: - .. code-block:: bash +#. Shut down your VM if it is running. +#. Use the process defined by your hypervisor + or cloud provider to increase + the virtual disk size of your |CL| VM. +#. Power up the VM. - sudo swupd bundle-add storage-utils -#. Launch the `parted` tool. +Resize the partition of the virtual disk: +----------------------------------------- - .. code-block:: bash +#. Log in to an account with root privileges. +#. Open a terminal emulator. +#. Add the |CL| `storage-utils` bundle to install the + :command:`parted` and :command:`resize2fs` tools. - sudo parted + .. code-block:: bash -#. In the `parted` tool, perform these steps: + sudo swupd bundle-add storage-utils - #. Press :command:`p` to print the partitions table. - #. If the warning message below is displayed, enter :command:`Fix`. +#. Launch the `parted` tool. - .. code-block:: console + .. code-block:: bash - Warning: Not all of the space available to /dev/sda appears to be - used, you can fix the GPT to use all of the space (an extra ... - blocks) or continue with the current setting? + sudo parted - Fix/Ignore? +#. In the `parted` tool, perform these steps: - #. Enter :command:`resizepart [partition number]` where - *[partition number]* is the partition number to modify. - #. Enter :command:`yes` when prompted. - #. Enter the new `End` size. + #. Press :command:`p` to print the partitions table. + #. If the warning message below is displayed, enter :command:`Fix`. - .. note:: + .. code-block:: console - If you want a partition to take up the remaining disk space, then - enter the total size of the disk. When you print the partitions - table with the :command:`p` command, the total disk size is shown - after the `Disk` label. + Warning: Not all of the space available to /dev/sda appears to be + used, you can fix the GPT to use all of the space (an extra ... + blocks) or continue with the current setting? - #. Enter :command:`q` to exit `parted` when you are finished resizing the - image. + Fix/Ignore? -#. Enter :command:`sudo resize2fs -p /dev/[modified partition name]` where - *[modified partition name]* is the partition that was changed in `parted`. + #. Enter :command:`resizepart [partition number]` where + *[partition number]* is the partition number of the partition to modify. + #. Enter :command:`yes` when prompted. + #. Enter the new `End` size. -Figure 1 shows how to increase the size of a |CL| Hyper-V image from 8.5 -GB to 20 GB. Before the steps shown in Figure 1, we used the Hyper-V Manager -to increase the VM virtual disk size from 8.5 GB to 20 GB. + .. note:: -.. figure:: figures/increase-virtual-disk-size-1.png - :scale: 100 % - :alt: Increase root partition size example + If you want a partition to take up the remaining disk space, then + enter the total size of the disk. When you print the partitions + table with the :command:`p` command, the total disk size is shown + after the `Disk` label. - Figure 1: Increase root partition size example. + An example of this can be seen in Figure 1. -.. _releases: https://download.clearlinux.org/releases/ + #. Enter :command:`q` to exit `parted` when you are finished resizing the + image. + + Figure 1 depicts the described steps to resize the partition of the virtual disk from 8.5GB to 20GB. + + .. figure:: figures/increase-virtual-disk-size-1.png + :scale: 100 % + :alt: Increase root partition size + + Figure 1: Increase root partition size. + +Resize the filesytem +-------------------- + +#. Enter :command:`sudo resize2fs -p /dev/[modified partition name]` where + *[modified partition name]* is the partition that was changed in `parted`. + +#. Run the :command:`df -h` to verify that the filesystem size has + increased. + + Figure 2 depicts the described steps to resize the partition of the virtual disk from 8.5GB to 20GB. + + .. figure:: figures/increase-virtual-disk-size-2.png + :scale: 100 % + :alt: Increase root filesystem with resize2fs + + Figure 2: Increase root filesystem size after partition has been expanded. + +Congratulations! You have resized the disk, partition, and filesystem. At +this point, the increase in disk capacity is usable. + +.. _releases: https://download.clearlinux.org/releases/ \ No newline at end of file diff --git a/source/clear-linux/guides/maintenance/maintenance.rst b/source/clear-linux/guides/maintenance/maintenance.rst index 8775a909..09737c90 100644 --- a/source/clear-linux/guides/maintenance/maintenance.rst +++ b/source/clear-linux/guides/maintenance/maintenance.rst @@ -9,8 +9,10 @@ maintaining |CLOSIA| after :ref:`installation ` is completed. .. toctree:: :maxdepth: 2 + developer-workstation + swupd-search enable-user-space - update + swupd-guide bulk-provision mixer mixin @@ -21,4 +23,4 @@ maintaining |CLOSIA| after :ref:`installation ` is completed. download-verify-uncompress-linux download-verify-uncompress-mac download-verify-uncompress-windows - image-types + autospec diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 90e54dec..b6f7cb3c 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -16,35 +16,26 @@ To start working with the mixer tools, you need a recent image of |CL| with the `mixer` bundle installed. If the bundle is not yet installed, you can add it with the :command:`swupd bundle-add` command as follows: - .. code-block:: bash +.. code-block:: bash - sudo swupd bundle-add mixer + sudo swupd bundle-add mixer Current mixing workflow *********************** There are two different workflows to create your own mix. +First, if your mix only uses |CL| content, *skip step 5* below. +Second, if your mix includes your own +:abbr:`RPMs (RPM Package Manager files)`, follow all these steps. -The following workflow applies if the mix includes your own -:abbr:`RPMs (RPM Package Manager files)`: - -#. `Create a workspace`_ -#. `Generate the starting point for your mix`_ -#. `Edit builder.conf`_ -#. `Create or locate RPMs for the mix`_ -#. `Import RPMs into workspace`_ -#. `Create a local RPM repo`_ -#. `List, edit, create, add, remove, or validate bundles`_ -#. `Build the bundle chroots`_ -#. `Create an update`_ -#. `Create an image`_ - -Alternatively, the following workflow applies if the mix only uses |CL| -content. - +#. `Create ngninx web server to host mixer updates`_ #. `Create a workspace`_ #. `Generate the starting point for your mix`_ #. `Edit builder.conf`_ +#. `Create custom RPMs`_ + #. `Create or locate RPMs for the mix`_ + #. `Import RPMs into workspace`_ + #. `Create a local RPM repo`_ #. `List, edit, create, add, remove, or validate bundles`_ #. `Build the bundle chroots`_ #. `Create an update`_ @@ -53,6 +44,114 @@ content. The following sections contain detailed information on every step of these workflows. +.. _create-nginx-web-server: + +Create ngninx web server to host mixer updates +********************************************** + +Follow these steps to set up a HTTP service with ``nginx`` web +server, where you can host custom |CL| mixes. + +#. Install ``web-server-basic``: + + .. code-block:: bash + + sudo swupd bundle-add web-server-basic + +#. Make the directory where mixer updates will reside. + + .. code-block:: bash + + sudo mkdir -p /var/www + +#. Create a symbolic link. + + .. code-block:: bash + + sudo ln -sf $HOME/mixer/update/www /var/www/mixer + +#. Set up ``nginx`` configuration. + + .. code-block:: bash + + sudo mkdir -p /etc/nginx/conf.d + +#. Copy the default example configuration file. + + .. code-block:: bash + + sudo cp -f /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf + +#. Open ``mixer.conf`` with an editor. + + .. code-block:: bash + + sudo nano /etc/nginx/conf.d/mixer.conf + +#. Add the server configuration content below to ``mixer.conf``. + + .. code-block:: console + + server { + server_name localhost; + location / { + root /var/www/mixer; + autoindex on; + } + } + +#. Restart the daemon, enable nginx on boot, and start the service. + + .. code-block:: bash + + sudo systemctl daemon-reload + + sudo systemctl enable nginx + + sudo systemctl start nginx + +#. To verify the web server is running, enter in an Internet browser: + + .. code-block:: bash + + http://localhost + +#. Alternatively, verify the server is running by entering: + + .. code-block:: bash + + ps aux | grep nginx + + .. note:: + + If `nginx` is active, a list should appear showing one master process + and a few worker processes. + +Connect the URL to mixer +======================== + +Add the URL of the `nginx` server to builder.conf. Your |CL| clients connect +to this URL to find the update content. + +#. Get the IP address of your nginx server: + + .. code-block:: bash + + networkctl status + +#. In the mixer workspace, edit `builder.conf` to set the value for + CONTENTURL and VERSIONURL to the IP `Address` of your `nginx` server. + + .. code-block:: bash + + nano builder.conf + + .. note:: + + For example: + CONTNENTURL=http://192.168.25.52 + VERSIONURL=http://192.168.25.52 + Create a workspace ****************** @@ -91,7 +190,7 @@ For example: Additionally, to build a mix with your own custom RPMs, use the optional -:option:`--local-rpms` flag, for example: +*--local-rpms* flag, for example: .. code-block:: bash @@ -103,9 +202,8 @@ directories in your mix workspace and adds their paths to the generated the paths manually. For more information on using these directories or setting them up manually, see `Create or locate RPMs for the mix`_. -If all upstream |CL| bundles will be part of the mix, you can easily add them all -during initialization with the optional :option:`--all-upstream` flag. For -example: +If all upstream |CL| bundles will be part of the mix, you can easily add +them all during initialization with the optional *--all-upstream* flag. For example: .. code-block:: bash @@ -114,7 +212,7 @@ example: Finally, you may want to track the contents of your mixer workspace with a git repository. This is a great way to track changes to your mix's content or to revert to earlier versions if something goes wrong. Mixer can set this -up automatically with the optional :option:`--git` flag, for example: +up automatically with the optional *--git* flag, for example: .. code-block:: bash @@ -133,7 +231,7 @@ Edit builder.conf To configure the mixer tool, edit the :file:`builder.conf` as needed. The file :file:`builder.conf` is read automatically from the current -workspace directory. Use the :option:`--config` flag during initialization +workspace directory. Use the *--config* flag during initialization to specify an alternate path to the file as needed. The :file:`builder.conf` file has different sections, for example: @@ -197,8 +295,8 @@ The following variables require further explanation: the folder at all. * The `YUM_CONF` variable sets the path where mixer automatically generates - the :file:`.yum-mix.conf` yum configuration file. The yum configuration file - points the chroot-builder to the path where the RPMs are stored. + the :file:`.yum-mix.conf` yum configuration file. The yum configuration + file points the chroot-builder to the path where the RPMs are stored. * The `CERT` variable sets the path where mixer stores the :file:`Swupd_Root.pem` certificate file. The chroot-builder needs the @@ -206,9 +304,9 @@ The following variables require further explanation: security for content verification. The value of the `CERT` variable can point to a different certificate. The chroot-builder inserts the certificate specified in this value into the - :file:`/os-core-update/usr/share/clear/update-ca/` path. The software update - client uses this certificate to verify the :file:`Manifest.MoM` file's - signature. For now, we **strongly** recommend that you do not modify + :file:`/os-core-update/usr/share/clear/update-ca/` path. The software + update client uses this certificate to verify the :file:`Manifest.MoM` + file's signature. For now, we **strongly** recommend that you do not modify this line, as the certificate that :abbr:`swupd (Software Updater)` expects needs to have a very specific configuration to sign and verify properly. Mixer automatically generates the certificate, if you do not @@ -216,14 +314,14 @@ The following variables require further explanation: file to provide security for the updated content you create. * The `CONTENTURL` and `VERSIONURL` variables set the domain or IP address - where swupd looks for your update content and the corresponding version. You - must set these variables to the domain or IP-address of the server hosting the - update content. You can use any web server to host your update content. To learn - how to install and configure web server using |CL|, visit - :ref:`web-server-install`. For our example, the web update content within - the `SERVER_STATE_DIR` directory is located here: - :file:`/home/clr/mix/update/www`. If the web server is on the same machine as - this directory, you can create a symlink to the directory in your web + where swupd looks for your update content and the corresponding version. + You must set these variables to the domain or IP-address of the server + hosting the update content. You can use any web server to host your update + content. To learn how to install and configure web server using |CL|, see + :ref:`create-nginx-web-server`. For our example, the web update content + within the `SERVER_STATE_DIR` directory is located here: + :file:`/home/clr/mix/update/www`. If the web server is on the same machine + as this directory, you can create a symlink to the directory in your web server's document root to easily host the content. These URLs are embedded in images created for your mix. The `swupd-client` looks at these URLs to determine if a new version is available and the location @@ -242,8 +340,12 @@ The following variables require further explanation: .. note:: If you are working only with |CL| bundles, then skip to `List, edit, create, add, remove, or validate bundles`_. + +Create custom RPMs +****************** + Create or locate RPMs for the mix -********************************* +================================= If you create RPMs from scratch, you can use `autospec`, `mock`, `rpmbuild`, or similar tools to build them. If the RPMs are not built on |CL|, ensure @@ -252,7 +354,7 @@ there is no guarantee they will be compatible. For more information on building the RPMs properly, refer to our `build RPMs instructions`_. Import RPMs into workspace -************************** +========================== #. Create a :file:`local-rpms` directory in your workspace, for example, :file:`/home/clr/mix/local-rpms`. @@ -269,7 +371,7 @@ Mixer uses this directory to find the RPMs to build a local RPM repo for yum to use. Create a local RPM repo -*********************** +======================= #. Create an empty directory in your workspace named :file:`local-yum`. #. Add the path to your :file:`builder.conf` file: @@ -315,7 +417,7 @@ bundles. When listing bundles with this command, mixer automatically recurses through the includes to show every single bundle in the mix. If you see an unexpected bundle in the list, that bundle is probably included -in another bundle. Use the :option:`--tree` flag to get a better view of how +in another bundle. Use the *--tree* flag to get a better view of how a bundle ended up in the mix, for example: .. code-block:: bash @@ -357,7 +459,7 @@ with the following command: mixer bundle list local Both the local and upstream :command:`bundle list` commands accept the -:option:`--tree` flag to show a visual representation of the inclusion relationships +*--tree* flag to show a visual representation of the inclusion relationships between the bundles in the mix. Edit the bundles in the mix @@ -416,7 +518,8 @@ as part of the bundle. .. note:: The :command:`mixer bundle edit` command accepts multiple bundles at once. - Thus, you can create multiple new bundles in a single command, for example: + Thus, you can create multiple new bundles in a single command, for + example: .. code-block:: bash @@ -458,7 +561,7 @@ This command removes `bundle1` from the mix bundle list stored in your :file:`mixbundles` file. By default, this command does not remove the bundle definition file from your local bundles. To completely remove a bundle, including its local bundle definition file, use the following command with -the :option:`--local` flag: +the *--local* flag: .. code-block:: bash @@ -466,7 +569,7 @@ the :option:`--local` flag: By default, removing a local bundle file with this command removes the bundle from the mix as well. To only remove the local bundle definition file, use -the following command with the :option:`--mix=false` flag: +the following command with the *--mix=false* flag: .. code-block:: bash @@ -486,8 +589,8 @@ Validate the bundles in the mix Mixer performs basic validation on all bundles when used throughout the system. -Mixer checks the validity of the bundle's syntax and name. Optionally, you can -run this validation manually on `bundle1` with the following command: +Mixer checks the validity of the bundle's syntax and name. Optionally, you +can run this validation manually on `bundle1` with the following command: .. code-block:: bash @@ -496,7 +599,7 @@ run this validation manually on `bundle1` with the following command: .. note:: This command can be useful in many circumstances. One example is when importing already-existing local bundles from other projects. -If you use the optional :option:`--strict` flag, the command additionally +If you use the optional *--strict* flag, the command additionally checks if the rest of the bundle header fields can be parsed, if the bundle header fields are non-empty, and if the bundle header ``Title`` field and the bundle filename match. Perform a strict validation of `bundle1` with the @@ -521,7 +624,7 @@ git commit after you modify the mix bundle list or edit a bundle definition file. All the :command:`mixer bundle` commands in the previous sections support an -optional :option:`--git` flag. This flag automatically applies a git commit +optional *--git* flag. This flag automatically applies a git commit when the command completes, for example: .. code-block:: bash @@ -551,7 +654,7 @@ chroots. We have added a new chroot-builder to the mixer tool itself. While this is currently an experimental feature, you should use the new chroot-builder. To use the new chroot-builder, use the following command with the -:option:`--new-chroots` flag: +*--new-chroots* flag: .. code-block:: bash @@ -578,7 +681,7 @@ By default, mixer uses the legacy `swupd-server` to generate the update content. However, we have built a new implementation into the mixer tool itself. While this is currently an experimental feature, you should use the new `swupd-server`. To use the the new `swupd-server`, use the following -command with the :option:`--new-swupd` flag: +command with the *--new-swupd* flag: .. code-block:: bash @@ -601,7 +704,7 @@ mix version to another, with the following command: The pack-maker generates all delta packs for the bundles changed from `PAST_VERSION` to `MIX_VERSION`. If your `STATE_DIR` is in a different -location, specify the location with the :option:`-S` flag. Mixer cannot +location, specify the location with the *-S* flag. Mixer cannot create delta packs for the first build because the update is from version 0. Version 0 implicitly has no content. Thus, mixer can generate no deltas. @@ -651,7 +754,7 @@ With the `ister` tool configured, build the image with the following command: Mixer automatically looks for the :file:`release-image-config.json` file, but you can freely choose the filename. To use a different name, simply pass the -:option:`--template` flag when creating your image, for example: +*--template* flag when creating your image, for example: .. code-block:: bash @@ -659,7 +762,7 @@ you can freely choose the filename. To use a different name, simply pass the By default, `ister` uses the format version of the build machine it runs on. Therefore, if the format you are building differs from the format of the |CL| -OS you are building on, you must use the :option:`--format ` +OS you are building on, you must use the *--format * flag. Find the current format version of your OS with the following command: .. code-block:: bash @@ -677,14 +780,14 @@ Increment the mix version number for the next mix with the following command: This command automatically updates the mix version stored in the :file:`mixversion` file, incrementing it by 10. To increment by a different -amount, use the :option:`--increment` flag, for example: +amount, use the *--increment* flag, for example: .. code-block:: bash mixer versions update --increment 100 Alternatively, to set the mix version to a specific value, use the -:option:`--mix-version` flag, for example: +*--mix-version* flag, for example: .. code-block:: bash @@ -699,7 +802,7 @@ If you have been tracking your workspace with git, you can restore the mix to an earlier state. However, be careful when "rewriting history" if you have published the mix content to users already. -Use the following command with the the :option:`--upstream-version` flag to +Use the following command with the the *--upstream-version* flag to update the upstream version of |CL| used as a base for the mix: .. code-block:: bash @@ -746,7 +849,8 @@ modifications as needed, for example: .. code-block:: bash - sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + sudo mixer-pack-maker.sh --to --from -S / + home/clr/mix/update .. _mixer-format: diff --git a/source/clear-linux/guides/maintenance/swupd-guide.rst b/source/clear-linux/guides/maintenance/swupd-guide.rst new file mode 100644 index 00000000..e22a2695 --- /dev/null +++ b/source/clear-linux/guides/maintenance/swupd-guide.rst @@ -0,0 +1,183 @@ +.. _swupd-guide: + +Use swupd +######### + +*swupd* manages the software update capability of |CLOSIA|. It can check for +valid system updates and, if found, download and install them. It can also +perform verification of the system software. + +|CL| uses :ref:`bundles ` as the base abstraction for +installing functionality on top of the core operating system. Use the `swupd` +tool to install and remove bundles. + +This guide assumes: + +* The device is on a well-connected network. + +* The device is able to connect to the release infrastructure + http://update.clearlinux.org + +.. contents:: + :local: + :depth: 2 + +.. _man_swupd: + +For a full listing of commands and options please see the man page in |CL| + +.. code-block:: bash + + man swupd + +OS update and verification +************************** + +|CL| is designed to promote a regular update cadence. `swupd` helps to +make sure that process is simple and secure. + +View OS info +============ + +Current OS version and update server info: + +.. code-block:: bash + + swupd info + +.. code-block:: console + + Installed version: 23330 + Version URL: https://download.clearlinux.org/update/ + Content URL: https://cdn.download.clearlinux.org/update/ + +Enable or disable automatic updates +=================================== + +|CL| updates are automatic by default but can be set to occur only +on demand. To verify your current auto-update setting: + +.. code-block:: bash + + sudo swupd autoupdate + +Disable automatic updates: + +.. code-block:: bash + + sudo swupd autoupdate --disable + +To re-enable automatic: + +.. code-block:: bash + + sudo swupd autoupdate --enable + +Check for updates +================= + +.. code-block:: bash + + sudo swupd check-update + +Perform a manual update +======================= + +You can update to a specific version or accept the latest as the default with +no arguments. Initiate a manual update: + +.. code-block:: bash + + sudo swupd update -m 23330 + +.. note:: + + After a kernel update, reboot immediately for the enhancements to + take effect. + +System Software Verification +============================ + +`swupd` can determine whether system directories and files have been added +to, overwritten, removed, or modified (e.g., permissions). + +.. code-block:: bash + + sudo swupd verify + +All directories that are watched by `swupd` are verified according to +the manifest data and hash mismatches are flagged as follows: + +.. code-block:: console + + Verifying version 23300 + Verifying files + ...0% + Hash mismatch for file: /usr/bin/chardetect + ... + ... + Hash mismatch for file: /usr/lib/python3.6/site-packages/urllib3/util/wait.py + ...100% + Inspected 237180 files + 423 files did not match + Verify successful + +In this case, python packages that were installed on top of the default +install were flagged as mismatched. `swupd` can be directed to ignore +or fix issues based on :ref:`command line options `. + +Fixing hash mismatches +====================== + +`swupd` can correct any issues it detects. Additional directives can be +added including a white list of directories that will be ignored, if +desired. + +The following command will repair issues, remove unknown items, and +ignore files or directories matching `/usr/lib/python`: + +.. code-block:: bash + + sudo swupd verify --fix --picky --picky-whitelist=/usr/lib/python + +Bundles +******* + +Listing installed bundles +========================= + +You can list all of the bundles currently installed on the system + +.. code-block:: bash + + sudo swupd bundle-list --all + +Finding a bundle containing a binary +==================================== + +Run the following to display a list of bundles that contain a particular +binary. Note that it may be present in multiple bundles: + +.. code-block:: bash + + swupd search -b + +Adding a bundle +=============== + +Start by selecting a bundle from the list of :ref:`available-bundles`. In +this example we're adding dev-utils-dev, which is useful for development. + +.. code-block:: bash + + sudo swupd bundle-add dev-utils-dev + +Removing a bundle +================= + +Dependencies common to other bundles will not be removed + +.. code-block:: bash + + sudo swupd bundle-remove dev-utils-dev + diff --git a/source/clear-linux/guides/maintenance/swupd-search.rst b/source/clear-linux/guides/maintenance/swupd-search.rst new file mode 100644 index 00000000..2a066451 --- /dev/null +++ b/source/clear-linux/guides/maintenance/swupd-search.rst @@ -0,0 +1,144 @@ +.. _swupd-search: + +Use swupd search to find bundles +################################ + +.. contents:: + :local: + :depth: 2 + +This document shows you how to use `swupd search` to find and add +bundles. + +Assumptions +*********** + +This guide assumes you: + +* Possess a basic knowledge of :ref:`swupd ` +* Understand :ref:`how swupd differs ` from + other Linux\* distributions +* May use :ref:`mixer` to build your own |CL| OS + +How do I search for a bundle? +***************************** + +Use `swupd search` to locate the bundle where the application binary exists. + +Example: Kata Containers +======================== + +Containers have revolutionized the way we manage cloud infrastructure. +Traditional containers often share the same OS kernel, which raises +security concerns. Instead, with Kata Containers, each container has its own +kernel instance and runs on its own :abbr:`Virtual Machine (VM)`. Whether you're running 3 or 300 nodes on your cluster, Kata Containers provide a +lightweight, fast, and secure option for app/container management. + +In |CL|, you only need to add `this bundle`_ to use `Kata Containers`_: +`containers-virt`. Also, check out our tutorial: :ref:`kata`. + +We need to find *kata* containers in a bundle. How do we search for it? + +#. Enter :command:`swupd search`, followed by 'kata' as the search term: + + .. code-block:: bash + + sudo swupd search kata + + .. note:: + + `swupd search` downloads |CL| manifest data and searches for + matching paths. Enter only one term, or hyphenated term, per + search. Use the command :command:`man swupd` to learn more. + +#. Alternatively, if you want to search binaries only, add the `-b` + flag: + + .. code-block:: bash + + sudo swupd search -b kata + + .. note:: + + `-b` flag, or `--binary`, means: Restrict search to program binary paths. Omit this flag if you want a larger scope of search results. + + Only the base bundle is returned. In |CL|, *bundles* can contain + other *bundles* via `includes`. For more details, see `Bundle Definition Files`_ and its subdirectory *bundles*. + + If your search does not produce results on a specific term when using + the `-b` flag, abbreviate the search term. For example, if you search + for *kubernetes* and it does not show results, instead abbreviate the + term to *kube* to show results. + +#. Optionally, you can review our `bundles`_ or individual `packages`_ + +#. Using `sudo swupd search -b kata` shows a match for our use case. + + .. code-block:: console + + Bundle containers-virt (834 MB to install) + /usr/bin/kata-virtfs-lite-proxy-helper + /usr/bin/kata-runtime + /usr/bin/kata-qemu-lite-system-x86_64 + /usr/bin/kata-qemu-lite-pr-helper + /usr/bin/kata-qemu-lite-ga + /usr/bin/kata-collect-data.sh + + .. note:: + + If the bundle is already installed, *[installed]* appears in search results. If this doesn't apppear, the bundle needs to be installed. + +#. Add the bundle `containers-virt`: + + .. code-block:: bash + + sudo swupd bundle-add containers-virt + + .. note:: + + To add multiple bundles simply add a space followed by the bundle name. + +#. When prompted, enter your password. + +#. Upon successful installation, your console should show similar data: + + .. code-block:: console + + Downloading packs... + + Extracting containers-virt pack for version 24430 + ...50% + Extracting kernel-container pack for version 24430 + ...100% + Starting download of remaining update content. This may take a while... + ...100% + Finishing download of update content... + Installing bundle(s) files... + ...100% + Calling post-update helper scripts. + Successfully installed 1 bundle + +FAQ +=== + +Find answers to these common questions: + +* How do I install and *use* :ref:`Kata Containers ` on |CL|? + +* How do I :ref:`kata_migration`? + +* How do I show all :ref:`bundles available `? + +* How do I :ref:`update swupd`? + +* How do I :ref:`remove bundles`? + +.. _Kata Containers: https://clearlinux.org/blogs/clear-linux-os-announces-support-kata-containers + +.. _this bundle: https://github.com/clearlinux/clr-bundles/blob/master/bundles/containers-virt + +.. _Bundle Definition Files: https://github.com/clearlinux/clr-bundles + + .. _bundles: https://github.com/clearlinux/clr-bundles/tree/master/bundles + + .. _packages: https://github.com/clearlinux/clr-bundles/blob/master/packages diff --git a/source/clear-linux/guides/maintenance/telemetry-enable.rst b/source/clear-linux/guides/maintenance/telemetry-enable.rst index 3f6ed348..6d401226 100644 --- a/source/clear-linux/guides/maintenance/telemetry-enable.rst +++ b/source/clear-linux/guides/maintenance/telemetry-enable.rst @@ -5,11 +5,11 @@ Enable and disable telemetry in Clear Linux |CLOSIA| includes a telemetry solution as part of the OS that records events of interest and reports them back to the development team via the telemetrics -daemon, :command:`telemd`. This functionality is maintained in the -``telemetrics`` software bundle. +daemon, **telemd**. This functionality is maintained in the +**telemetrics** software bundle. .. note:: - The telemetry functionality adheres to `Intel's privacy policies`_ + The telemetry functionality adheres to `Intel privacy policies`_ regarding the collection and use of :abbr:`PII (Personally Identifiable Information)` and is open source. Specifically, no intentionally identifiable information about the user or system owner is collected. @@ -20,10 +20,10 @@ redirect where the records go if they wish to collect records for themselves. Install the telemetry software bundle ************************************* -During the initial installation of |CL| you are requested to join the +During the initial installation of |CL|, you are requested to join the stability enhancement program and allow |CLOSIA| to collect anonymous reports -to improve system stability. If you chose not to join this program at that -time then the telemetry software bundle is not added to your system. +to improve system stability. If you choose not to join this program, then the +telemetry software bundle is not added to your system. To install the telemetry bundle, enter the following command as either the root user or with :command:`sudo` privileges: @@ -104,9 +104,13 @@ To completely remove telemetrics from your system, use the command Additional resources ******************** -https://clearlinux.org/features/telemetry +* `Telemetry feature description`_ +* :ref:`Telemetry architecture` +* :ref:`telemetry-backend` +* https://github.com/clearlinux/telemetrics-client -https://github.com/clearlinux/telemetrics-client - -.. _`Intel's privacy policies`: +.. _`Intel privacy policies`: https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/guides/maintenance/update.rst b/source/clear-linux/guides/maintenance/update.rst deleted file mode 100644 index 5705fda3..00000000 --- a/source/clear-linux/guides/maintenance/update.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. _update: - -Update Clear Linux -################## - -This section describes how to update |CLOSIA|. - -Prerequisites -============= - -* The system is up and running the Clear Linux 15400 release or later. - -* The device is on a well-connected network. - -* The device is able to connect to the release infrastructure - http://update.clearlinux.org - -To verify the current release running on the system, enter: - -.. code-block:: console - - $ sudo swupd update -s - -How to update the system -======================== - -Starting with version 15400, Clear Linux supports auto-update. By default, it -is turned on. - -To verify the current auto-update setting: - - .. code-block:: console - - $ swupd autoupdate - -To enable auto-update: - - .. code-block:: console - - $ sudo swupd autoupdate --enable - -To disable auto-update: - - .. code-block:: console - - $ sudo swupd autoupdate --disable - -To force a manual update: - - .. code-block:: console - - $ sudo swupd update - -.. note:: - - When the swupd console output indicates a kernel update, reboot - immediately for the enhancements to take effect. - - -Additional information -====================== - -To see the man page listing additional swupd options, install the -`sysadmin-basic` bundle to access the `man` command and later access -the man page: - - .. code-block:: console - - $ sudo swupd bundle-add sysadmin-basic - $ man swupd diff --git a/source/clear-linux/guides/maintenance/validate-signatures.rst b/source/clear-linux/guides/maintenance/validate-signatures.rst index 190e4f59..1001369a 100644 --- a/source/clear-linux/guides/maintenance/validate-signatures.rst +++ b/source/clear-linux/guides/maintenance/validate-signatures.rst @@ -9,9 +9,11 @@ creates a chain of trust. A broken chain of trust, seen as an invalid signature, means the content is not valid. This guide covers how to validate the contents of an image, which is a manual -process, and describes the automatic process ``swupd`` performs internally to +process and is the same process ``swupd`` performs internally to validate an update. +.. _image-content-validation: + Image content validation ======================== diff --git a/source/clear-linux/guides/network/vnc.rst b/source/clear-linux/guides/network/vnc.rst index e2f1e21d..8de6f3ef 100644 --- a/source/clear-linux/guides/network/vnc.rst +++ b/source/clear-linux/guides/network/vnc.rst @@ -674,12 +674,12 @@ For `Method 3`: Figure 7 shows two VNC sessions (5901 and 5905) accepting connections from any host as specified by the `0.0.0.0`'s. This is before the `-localhost` option was used. - + .. figure:: figures/vnc/vnc-7.png :scale: 100 % :alt: VNC session accepting connection from any host - Figure 7: VNC sessions (5901 and 5905) accepting connections from any host + Figure 7: VNC sessions (5901 and 5905) accepting connections from any host Figure 8 shows two VNC sessions (5901 and 5905) only accepting connections from localhost as specified by `127.0.0.1`'s. This is after the `-localhost` option was used. diff --git a/source/clear-linux/reference/collaboration/documentation/contents.rst b/source/clear-linux/reference/collaboration/documentation/contents.rst new file mode 100644 index 00000000..e88c4d7c --- /dev/null +++ b/source/clear-linux/reference/collaboration/documentation/contents.rst @@ -0,0 +1,40 @@ +.. _contents: + + +Contents directive +################## + +For |CL| documentation that has three or more sections, use the `contents::` +directive as shown in the example below. This directive automatically captures the headings (and +subheadings if used) as specified in the value given after `:depth:`. Adding this directive to +longer documents allows users to quickly navigate to the desired section. + +.. contents:: :local: + :depth: 2 + +.. code-block:: bash + + .. contents:: :local: + :depth: 2 + +.. note:: + + Assure that you add `:local:` as the value. For more resources on this directive, + visit the `reStruturedText Directives`_ + +EXAMPLE: + +Clear Linux Guide Example +************************* + +Introduction +============ + +Step-by-Step +============ + +Launch +====== + + +.. _reStruturedText Directives: http://docutils.sourceforge.net/0.4/docs/ref/rst/directives.html#table-of-contents diff --git a/source/clear-linux/reference/collaboration/documentation/cross.rst b/source/clear-linux/reference/collaboration/documentation/cross.rst index 8b4d726d..4f5efc2a 100644 --- a/source/clear-linux/reference/collaboration/documentation/cross.rst +++ b/source/clear-linux/reference/collaboration/documentation/cross.rst @@ -15,16 +15,18 @@ Internal cross-references An internal cross-reference is a reference to a location within the |CLOSIA| documentation. Use explicit markup labels and the ``:ref:`` role to create cross references to headings, figures, and code examples as needed. Every -file must have a label before the title identical to the file's name in order -to be able to add cross-references without having to open the file. +file must have a label before the title, which is identical to the file's +name, in order to be be cross-referenced within the entire documentation. -The labels' naming conventions are: +Labels' naming conventions: + +* Ensure the label is unique throughout the documentation * Use only full words. * Use \- to link multiple words. -* Use only as many words as necessary to ensure the label is unique. +* Use only as many words as necessary. These are some examples of proper labels: @@ -81,7 +83,8 @@ This creates a link to the :ref:`label-of-target` using the text of the heading. This creates a link to the :ref:`target ` using the word -'target' instead of the heading. +'target' instead of the heading. We use the term 'target' here similar to +the way'anchor' is used in HTML. .. note:: @@ -92,7 +95,7 @@ This creates a link to the :ref:`target ` using the word External References ******************* -External references or hyperlinks can be added easily with ReST. Only +External references or hyperlinks can be added easily with reST. Only hyperlinks with a separated target definition are allowed. Do not use explicit hyperlinks consisting entire URLs. For example, links @@ -121,4 +124,30 @@ Use this template to add a hyperlink with a separated definition: The state of `Oregon`_ offers a wide range of recreational activities. - .. _Oregon: http://traveloregon.com/ +The include directive +********************* + +Clear Linux documentation also uses the ``.. include::`` +directive to include a portion of another reST file. + +Use the ``.. include::`` directive to show a select portion of a file. + +.. code-block:: console + + .. include:: rest.rst + :start-after: incl-restructured-text-overview: + :end-before: incl-restructured-text-overview-end: + +In this example, note that you must: + +* Create a `target` that appears directly above a header (ease of inclusion) +* Ensure that the target is unique, as explained in :ref:`target ` +* Use a `:` at the end of the value of `start-after` and `end-before`. + +Use of the ``.. inclusion::`` for :ref:`rest` is shown below. + +.. include:: rest.rst + :start-after: incl-restructured-text-overview: + :end-before: incl-restructured-text-overview-end: + +.. _Oregon: http://traveloregon.com/ diff --git a/source/clear-linux/reference/collaboration/documentation/documentation.rst b/source/clear-linux/reference/collaboration/documentation/documentation.rst index b16ce41b..c9040c58 100644 --- a/source/clear-linux/reference/collaboration/documentation/documentation.rst +++ b/source/clear-linux/reference/collaboration/documentation/documentation.rst @@ -36,7 +36,7 @@ The |CL| technical content is written in simple American English and our This guide includes the following sections: .. toctree:: - :maxdepth: 2 + :maxdepth: 3 basic structures @@ -73,8 +73,7 @@ This style guide applies to the following technical content: We are always grateful to receive content contributions and are happy to help via our mailing list or our IRC channel, #clearlinux. If you have found a -problem with one of our documents, please file a bug report. Use our -:ref:`bug-report` to submit the bug. +problem with one of our documents, please file a `bug report`_. Tone and audience ***************** @@ -86,12 +85,12 @@ Remain professional in your writing and carry an undertone of cordiality, respect, and cooperation. Assume your audience has about the same level of technical understanding and -expertise as you did when you first started collaborating. Do not talk down to -our readers but do not assume they know everything about the subject. +expertise as you did when you first started collaborating. Do not talk down +to our readers but do not assume they know everything about the subject. Offer brief explanations or summaries of "common knowledge" if a significant portion of readers might benefit. -All contributions must follow our :ref:`code-of-conduct`. +All contributions must follow our `code of conduct`_. Methodology *********** @@ -126,7 +125,7 @@ decisions are explained in the respective section. This guide takes precedence over all other style guides in all cases. In cases where the guide does not address the issue at hand, please report the -issue to the `mailing list`_ using our :ref:`bug-report`. +issue to the `mailing list`_ using our `bug report`_. Use the Merriam-Webster's Collegiate Dictionary to determine correct spelling, hyphenation, and usage. @@ -136,3 +135,5 @@ spelling, hyphenation, and usage. .. _documentation section: https://clearlinux.org/documentation .. _Clear Linux documentation repository: https://github.com/clearlinux/clear-linux-documentation +.. _bug report: https://github.com/clearlinux/distribution/issues +.. _code of conduct: https://clearlinux.org/community/code-of-conduct diff --git a/source/clear-linux/reference/collaboration/documentation/inline.rst b/source/clear-linux/reference/collaboration/documentation/inline.rst index 2fb08380..2e516960 100644 --- a/source/clear-linux/reference/collaboration/documentation/inline.rst +++ b/source/clear-linux/reference/collaboration/documentation/inline.rst @@ -6,112 +6,157 @@ Inline Markup Sphinx supports a large number of inline markup elements called roles. The |CLOSIA| documentation encourages the use of as many roles as possible. Thus, you can use any additional roles supported by Sphinx -even if not listed here. Please refer to the `Sphinx Inline Markup`_ +not listed here. Please refer to the `Sphinx reStructuredText Markup`_ documentation for the full list of supported roles. The following markup is required in every instance unless otherwise -specified. Each item provides examples and a template for the correct use of -the roles. +specified. Each item provides a syntax example followed by the rendered +result. -* Use the `:abbr:` abbreviation role to define an acronym or an initialism. +Abbreviations + Use the `:abbr:` abbreviation role to define an acronym or an initialism. Add the abbreviation markup only once per file. After the abbreviation, the acronym can be used without further definition or markup. Do not use abbreviation markup on headings. - :abbr:`API (Application Program Interface)` + :: - Template: + :abbr:`API (Application Program Interface)` - ``:abbr:`TIA (This Is an Abbreviation)``` + .. parsed-literal:: -* Use the `:command:` role when the name of a specific command is used in a + :abbr:`API (Application Program Interface)` + +OS Commands + Use the `:command:` role when the name of a specific command is used in a paragraph for emphasis. Use the ``.. code-block::`` directive for fully actionable commands in a series of steps. - :command:`make` + :: - Template: + :command:`make` - ``:command:`command``` + .. parsed-literal:: -* Use the `:option:` role to emphasize the name of a command option - with or without its value. This markup is usually employed in - combination with the `:command:` role. For example: + :command:`make` - :option:`-f` - :option:`--all` - :option:`-o output.xsl` - The :command:`pandoc` command can be used without the :option:`-o` - option, creating an output file with the same name as the source - but a different extension. +Commandline Options + In most cases, use asterisks "*" to emphasize the name of a command + option. - Template: + :: - ``:option:`Option``` + Use the *-p* option to print the file. -* Use the `:file:` role to emphasize a filename or directory. Do not use the + .. parsed-literal:: + + Use the *-p* option to print the file. + + However, if you have defined an ``.. option::`` directive, you may + use the `:option:` role. Note that the result links back to the + option definition. + + .. code-block:: rest + + .. option: -o + + Description of the -o option + + The :command:`pandoc` command can be used without :option:`-o` + + .. option:: -o + + Description of the -o option + + .. parsed-literal:: + + The :command:`pandoc` command can be used without :option:`-o` + +Files + Use the `:file:` role to emphasize a filename or directory. Do not use the role inside a code-block but use it inside all notices that contain files or directories. Place variable parts of the path or filename in brackets `{}`. - :file:`collaboration.rst` :file:`doc/{user}/collaboration/figures` + .. code-block:: rest - Template: + :file:`collaboration.rst` - ``:file:`filename.ext` :file:`path/or/directory``` + :file:`doc/{user}/collaboration/figures` -* Use the `:guilabel:` role to emphasize elements of a graphic + .. parsed-literal:: + + :file:`collaboration.rst` + + :file:`doc/{user}/collaboration/figures` + +GUI Objects + Use the `:guilabel:` role to emphasize elements of a graphic user interface within a description. It replaces the use of quotes when referring to windows' names, button labels, options, or single menu elements. Always follow the marked element with the appropriate noun. For example: - In the :guilabel:`Tools` menu. - Press the :guilabel:`OK` button. - In the :guilabel:`Settings` window you find the :guilabel:`Hide - Content` option. + :: - Template: + In the :guilabel:`Tools` menu, click :guilabel:`settings`. - ``:guilabel:`UI-Label``` + .. parsed-literal:: -* Use the `:menuselection:` role to indicate the navigation through a menu + In the :guilabel:`Tools` menu, click :guilabel:`settings`. + +Menu Navigation + Use the `:menuselection:` role to indicate the navigation through a menu ending with a selection. Every `:menuselection:` element can have up to two menu steps before the selected item. If more than two steps are required, it can be combined with a `:guilabel:` or with another `:menuselection:` element. For example: - :menuselection:`File --> Save As --> PDF` - Go to :guilabel:`File` and select :menuselection:`Import --> Data - Base --> MySQL`. - Go to :menuselection:`Window --> View` and select :menuselection:` - Perspective --> Other --> C++` + :: - Template: + Go to :guilabel:`File` and select :menuselection:`Import --> Data Base --> MySQL`. - ``:menuselection:`1stMenu --> 2ndMenu --> Selection``` + Go to :menuselection:`Window --> View` and select :menuselection:`Perspective --> Other --> C++` -* Use the `:makevar:` role to emphasize the name of a Makefile variable. + .. parsed-literal:: + + Go to :guilabel:`File` and select :menuselection:`Import --> Data Base --> MySQL`. + + Go to :menuselection:`Window --> View` and select :menuselection:`Perspective --> Other --> C++` + +Makefile Variables + Use the `:makevar:` role to emphasize the name of a Makefile variable. The role can include only the name of the variable or the variable plus its value. - :makevar:`PLATFORM_CONFIG` - :makevar:`PLATFORM_CONFIG=basic_atom` + :: - Template: + :makevar:`PLATFORM_CONFIG` - ``:makevar:`VARIABLE``` + :makevar:`PLATFORM_CONFIG=basic_atom` -* Use the `:envvar:` role to emphasize the name of environment + .. parsed-literal:: + + :makevar:`PLATFORM_CONFIG` + + :makevar:`PLATFORM_CONFIG=basic_atom` + +Environment Variables + Use the `:envvar:` role to emphasize the name of environment variables. Just as with `:makevar:`, the markup can include only the name of the variable or the variable plus its value. - :envvar:`ZEPHYR_BASE` - :envvar:`QEMU_BIN_PATH=/usr/local/bin` + :: - Template: + :envvar:`ZEPHYR_BASE` + + :envvar:`QEMU_BIN_PATH=/usr/local/bin` - ``:envvar:`ENVIRONMENT_VARIABLE``` + .. parsed-literal:: -.. _Sphinx Inline Markup: - http://sphinx-doc.org/markup/inline.html#inline-markup + :envvar:`ZEPHYR_BASE` + + :envvar:`QEMU_BIN_PATH=/usr/local/bin` + +.. _Sphinx reStructuredText Markup: + http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html \ No newline at end of file diff --git a/source/clear-linux/reference/collaboration/documentation/rest.rst b/source/clear-linux/reference/collaboration/documentation/rest.rst index a11522bc..43c71097 100644 --- a/source/clear-linux/reference/collaboration/documentation/rest.rst +++ b/source/clear-linux/reference/collaboration/documentation/rest.rst @@ -3,12 +3,19 @@ RestructuredText guide ###################### +.. incl-restructured-text-overview: + +Overview +******** + The |CLOSIA| uses Sphinx and RestructuredText as authoring tools for its documentation. This section contains the preferred methods for using the :abbr:`ReST (RestructuredText)` markup on your documents. Please refer to the `Sphinx documentation`_ for the complete list of available markup and use as much markup as possible. +.. _Sphinx documentation: http://www.sphinx-doc.org/en/stable/markup/index.html + Remember: **Changing incorrect markup is easier than adding markup from scratch.** @@ -27,6 +34,8 @@ templates, or the code-block directive, for multi-lined templates. Every use case is explained, examples provided and, lastly, templates supplied. +.. incl-restructured-text-overview-end: + .. toctree:: :maxdepth: 3 @@ -36,7 +45,5 @@ templates supplied. cross images code + contents - -.. _Sphinx documentation: - http://sphinx-doc.org/contents.html diff --git a/source/clear-linux/reference/collaboration/documentation/tables.rst b/source/clear-linux/reference/collaboration/documentation/tables.rst index 3ad59298..29d0b2c2 100644 --- a/source/clear-linux/reference/collaboration/documentation/tables.rst +++ b/source/clear-linux/reference/collaboration/documentation/tables.rst @@ -18,7 +18,7 @@ Follow these general guidelines: it is not rendered. * Only create a table if the body of the table contains six or more cells, -which is a minimum table size of at least 2x3 or 3x2. + which is a minimum table size of at least 2x3 or 3x2. ReST supports several types of tables. |CL| uses grid and :abbr:`CSV-tables (Comma Separated Values tables)`. Grid tables are only diff --git a/source/clear-linux/reference/compatible-hardware.rst b/source/clear-linux/reference/compatible-hardware.rst new file mode 100644 index 00000000..98bceeef --- /dev/null +++ b/source/clear-linux/reference/compatible-hardware.rst @@ -0,0 +1,113 @@ +.. _compatible-hardware: + +Compatible Hardware +################### + +This document describes hardware that has been tested and confirmed as +compatible with |CLOSIA|. This list is not comprehensive and will continue to +grow. + +.. list-table:: **Table 1. Clear Linux Compatible Hardware** + :widths: 20, 20 + :header-rows: 1 + + * - Processor SKU + - Platform + + * - Intel® Core™ i5-6260U + - + + * - Intel® Core™ i5-6560U + - Dell XPS\* 13 9350 + + * - Intel® Celeron® J3455 + - NUC6CAYS + + * - Intel® Core™ i5-4250U + - + + * - Intel® Core™ i7-5557U + - + + * - Intel® Core™ i9-7900X + - Gigabyte\* X299 + + * - Intel® Core™ i3-4130 + - Lenovo Thinkserver\* TS140 + + * - Intel® Core™ i7-7567U + - NUC7i7BNH + + * - Intel® Core™ i7-8809G + - NUC8i7HVK + + * - Intel® Core™ i5-7260U + - NUC7i5BNH + + * - Intel® Core™ i7-8650U + - NUC7i7DNKE + + * - Intel® Core™ i5-7300U + - NUC7i5DNHE + + * - Intel® Xeon® Gold 6138 + - + + * - Intel® Xeon® E5-2699A v4 + - Dell PowerEdge\* R630 + + * - Intel® Xeon® E5-2620 v3 + - + + * - Intel® Core™ i5-6600 + - Gigabyte\* Z170X-UD5 + + * - Intel® Core™ i5-4250U + - D54250WYK + + * - Intel® Xeon® E5-2699 v3 + - S2600WT2 + + * - Intel® Atom™ J3455 + - NUC6CAYB + + * - Intel® Xeon® Bronze 3104 + - 0W23H8 + + * - Intel® Atom™ C2750 + - SuperMicro\* A1SAi + + * - Intel® Atom™ E3825 + - CircuitCo MinnowBoard MAX\* + + * - Intel® Core™ i7-8700 + - Gigabyte\* H370 WIFI + + * - Intel® Core™ i7-3667U + - Lenovo ThinkPad\* X1 Carbon laptop + + * - Intel® Core™ i5-4210U + - Dell XPS\* 13 laptop + + * - Intel® Celeron® J3455 + - NUC6CAYB + + * - Intel® Core™ i7-4790 + - Gigabyte\* desktop + + * - Intel® Core™ i5-6260U + - NUC6I6SYH + + * - Intel® Core™ i7-5557U + - NUC5I7RYH + + * - Intel® Core™ i7-4700MQ + - Lenovo ThinkPad\* T540p + + * - Intel® Core™ i7-5557U + - NUC5I7RYB + + * - Intel® Core™ i5-6260U + - NUC6I5SYH + +\* Other names and brands may be claimed as the property of others. \ No newline at end of file diff --git a/source/clear-linux/reference/compatible-kernels.rst b/source/clear-linux/reference/compatible-kernels.rst index caa5fe28..72c6072e 100644 --- a/source/clear-linux/reference/compatible-kernels.rst +++ b/source/clear-linux/reference/compatible-kernels.rst @@ -1,7 +1,7 @@ .. _compatible-kernels: -Compatible Clear Linux kernels -############################## +Compatible Clear Linux\* kernels +################################ The |CLOSIA| provides the following Linux kernels with a respective bundle. This document describes the specific use cases these `bundles`_ serve @@ -19,7 +19,7 @@ Kernel Container ================ The *kernel-container* bundle contains the kernel used by the -`Intel® Clear Containers`_ project. This kernel is optimized for +Intel® Clear Containers project. This kernel is optimized for fast booting and performs best on |CC| running on the Intel® architectures described on the :ref:`supported hardware list`. The optimization patches are found in our `Linux-Container`_ GitHub repo. @@ -44,11 +44,10 @@ The *kernel-kvm* bundle focuses on the Linux :abbr:`KVM (Kernel-based Virtual Machine)`. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. -Use this kernel when running |CL| as the guest OS -on top of *qemu/kvm*. Use this kernel with **cloud orchestrators** using -*qemu/kvm* internally as their **hypervisor**. -This kernel can be used as a stand alone Linux VM, see our -:ref:`instructions on using KVM` for more information. The +Use this kernel when running |CL| as the guest OS on top of *qemu/kvm*. Use +this kernel with **cloud orchestrators** using *qemu/kvm* internally as +their **hypervisor**. This kernel can be used as a standalone |CL| VM, see +our :ref:`instructions on using KVM` for more information. The optimization patches are found in our `Linux-KVM`_ GitHub repo. Kernel Hyper-V\* @@ -59,8 +58,8 @@ Hyper-V. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in -projects such as Microsoft `Azure`_\*. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +projects such as Microsoft `Azure`_\*. This kernel can be used in a +standalone |CL| VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV`_ GitHub repo. @@ -73,8 +72,8 @@ is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in -projects such as Microsoft `Azure`_. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +projects such as Microsoft `Azure`_. This kernel can be used in a standalone +|CL| VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV-LTS`_ GitHub repo. @@ -89,5 +88,4 @@ more information. The optimization patches are found in our .. _CIAO: https://github.com/01org/ciao .. _Azure: https://azuremarketplace.microsoft.com/en-us/marketplace/apps/clear-linux-project.clear-linux-os -.. _Intel® Clear Containers: - https://clearlinux.org/features/intel®-clear-containers + diff --git a/source/clear-linux/reference/image-types.rst b/source/clear-linux/reference/image-types.rst new file mode 100644 index 00000000..50f81dfc --- /dev/null +++ b/source/clear-linux/reference/image-types.rst @@ -0,0 +1,81 @@ +.. _image-types: + +Clear Linux\* image types +######################### + +.. _image-types-content: + +|CLOSIA| offers many types of `images`_ for different platforms and environments. + +.. _incl-image-filename: + +The naming convention of a |CL| image filename is: + +.. code-block:: console + + clear-[version number]-[image type].[compression type] + +* The *[version number]* field specifies the version number. + +* The *[image type]* field specifies the type of image and its corresponding + file format. + +* The *[compression type]* field specifies the compression type. Two types of + compressions are used: GNU\* zip (*.gz*) and XZ (*.xz*). + +.. _incl-image-filename-end: + +Table 1 lists the currently available images that are platform independent. +Table 2 lists the currently available images that are platform specific. + +.. list-table:: Table 1: Types of platform-independent Clear Linux images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - installer.img + - Preferred image of Clear Linux with interactive installer. + + * - installer.iso + - ISO of Clear Linux with interactive installer. Only for special cases where ISO image format is required (not for use with a USB key) + + * - live.img + - image for live booting into memory, without requiring installaton. + +.. list-table:: Table 2: Types of platform-specific Clear Linux images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - azure.vhd + - Virtual Hard Disk for use on Microsoft\* Azure\* cloud platform + + * - azure-docker.vhd + - Virtual Hard Disk for use on Microsoft Azure cloud platform with Docker\* pre-installed + + * - azure-machine-learning.vhd + - Virtual Hard Disk for use on Microsoft Azure cloud platform with the `machine-learning-basic` bundle installed + + * - cloud.img + - Image for use by cloud deployments such as OpenStack\* + + * - containers.img + - Image for use by Clear Containers runtime. Includes `optimized kernel`_ for Clear Containers. + + * - hyperv.vhdx + - Virtual Hard Disk for use with Microsoft Hyper-V\* hypervisor. Includes `optimized kernel`_ for Hyper-V. + + * - kvm.img + - Image for booting in a simple VM with start_qemu.sh. Includes + `optimized kernel`_ for KVM. + + * - vmware.vmdk + - Virtual Machine Disk for VMware\* platforms inclduing Player, Workstation, and ESXi. + +.. _images: https://download.clearlinux.org/image +.. _`optimized kernel`: https://clearlinux.org/documentation/clear-linux/reference/compatible-kernels + diff --git a/source/clear-linux/reference/reference.rst b/source/clear-linux/reference/reference.rst index b7ec6e29..1ad206a7 100644 --- a/source/clear-linux/reference/reference.rst +++ b/source/clear-linux/reference/reference.rst @@ -9,9 +9,11 @@ features. .. toctree:: :maxdepth: 2 + compatible-hardware bundle-commands bundles/available-bundles how-to-clear-overview collaboration/collaboration compatible-kernels - system-requirements \ No newline at end of file + system-requirements + image-types diff --git a/source/clear-linux/tutorials/aws-web/aws-web.rst b/source/clear-linux/tutorials/aws-web/aws-web.rst index 8c433212..25b9b1af 100644 --- a/source/clear-linux/tutorials/aws-web/aws-web.rst +++ b/source/clear-linux/tutorials/aws-web/aws-web.rst @@ -18,9 +18,11 @@ Prerequisites ************* This tutorial assumes the following statements are true: + * You are using a linux-based system to access AWS and can run :command:`SSH` to access the remote |CL| AWS image. -* Your browser puts downloaded files in the :file:`$HOME/Downloads` directory. +* Your browser puts downloaded files in the :file:`$HOME/Downloads` + directory. * You have already set up an AWS user account and logged into the AWS console. diff --git a/source/clear-linux/tutorials/azure/azure.rst b/source/clear-linux/tutorials/azure.rst similarity index 99% rename from source/clear-linux/tutorials/azure/azure.rst rename to source/clear-linux/tutorials/azure.rst index be19af9c..397b7032 100644 --- a/source/clear-linux/tutorials/azure/azure.rst +++ b/source/clear-linux/tutorials/azure.rst @@ -269,7 +269,7 @@ Log into your Microsoft Azure account #. Open your browser and enter the page `https://aka.ms/devicelogin` as shown in figure 1: - .. figure:: figures/azure-1.png + .. figure:: azure/figures/azure-1.png :scale: 50 % :alt: Microsoft Device Login @@ -280,7 +280,7 @@ Log into your Microsoft Azure account session login and will be different each time you log into MS Azure using the :command:`az login` command. - .. figure:: figures/azure-2.png + .. figure:: azure/figures/azure-2.png :scale: 50 % :alt: Microsoft Device Login - Azure CLI @@ -294,7 +294,7 @@ Log into your Microsoft Azure account have signed into the Microsoft Cross-platform Command Line Interface application on your device and you can close the window. - .. figure:: figures/azure-3.png + .. figure:: azure/figures/azure-3.png :scale: 50 % :alt: Microsoft Azure Cross-platform CLI @@ -390,7 +390,7 @@ For this tutorial, we are using the |CL| Basic SKU for our VM. If you have already defined your public/private SSH key pair and they are stored in your :file:`$HOME/.ssh` directory, you do not need to - include the :option:`--generate-ssh-keys` option. + include the *--generate-ssh-keys* option. Your output from this command will look similar to this output, where [user] is your user name: diff --git a/source/clear-linux/tutorials/docker/docker.rst b/source/clear-linux/tutorials/docker/docker.rst index 75c00e35..91d1fa1d 100644 --- a/source/clear-linux/tutorials/docker/docker.rst +++ b/source/clear-linux/tutorials/docker/docker.rst @@ -4,8 +4,8 @@ Run Docker\* on Clear Linux\* ############################# |CLOSIA| supports multiple containerization platforms, including a Docker\* -solution.|CL| has many `unique features`_ including a minimal default -installation which makes it compelling to use as a host for container +solution. |CL| has many `unique features`_ including a minimal default +installation, which makes it compelling to use as a host for container workloads, management, and orchestration. This tutorial covers: @@ -68,48 +68,17 @@ packages to run Docker images as containers. Congratulations! At this point, you have a working installation of Docker on |CL|. You are ready to start using container images on your system. -Integration with Clear Containers (optional) +Integration with Kata Containers (optional) ******************************************** -`Clear Containers`_, is an open source project aiming to increase security -of containers by using a hardware-backed virtual machine container runtime, -instead of software namespace containers that are provided by the standard +`Kata Containers`_, is an open source project aiming to increase security +of containers by using a hardware-backed virtual machine container runtime +rather than software namespace containers that are provided by the standard Docker *runc* runtime. -The Docker package from |CL| will automatically use the *cc-runtime* -required for Clear Containers if it is available on your Clear Linux system. +Clear Linux provides easy integration of the *kata-runtime* with Docker. +More information on installing and using the *kata-runtime* may be found at :ref:`kata`. -#. To take advantage of Clear Containers in |CL|, install the - *containers-virt* bundle by running this command: - - .. code-block:: bash - - sudo swupd bundle-add containers-virt - -#. Restart the Docker daemon through systemd manager by running this command: - - .. code-block:: bash - - sudo systemctl restart docker - -#. After restarting, the Docker daemon will seamlessly use Clear Containers - to launch containers. You can see the runtime has changed to - :command:`cc-runtime` by running this command: - - .. code-block:: bash - - sudo docker info | grep Runtime - -#. You should see the following output indicating the *cc-runtime* is the - Default Runtime: - - .. code-block:: console - - Runtimes: cc-runtime runc - Default Runtime: cc-runtime - -Congratulations! At this point, you have successfully replaced the default -container runtime with Clear Containers. .. note:: @@ -168,16 +137,16 @@ an the official Docker image for nginx, an open source reverse proxy server. .. note:: - Below is an explaination of switches used in the command above. For + Below is an explanation of switches used in the command above. For detailed :command:`docker run` switches and syntax, refer to the `Docker Documentation`_ . - * The :option:`--name` switch lets you provide a friendly name to + * The *--name* switch lets you provide a friendly name to target the container for future operations - * The :option:`-d` switch launches the container in the background + * The *-d* switch launches the container in the background - * The :option:`-p` switch allows the container's HTTP port (80) to be + * The *-p* switch allows the container's HTTP port (80) to be accessible from the Clear Linux host on port 8080 #. You can access the Welcome to Nginx! splash page running in the container @@ -197,12 +166,12 @@ an the official Docker image for nginx, an open source reverse proxy server. sudo docker rm test-nginx Congratulations! At this point, you have successfully pulled a nginx -container image from `Docker Hub`_ and ran an example container. +container image from `Docker Hub`_ and have run an example container. Creating a Docker swarm cluster ******************************* -Clusters of Docker hosts are referred to as swarms. +Clusters of Docker hosts are referred to as *swarms*. The process in this tutorial can be repeated to install Docker on multiple Clear Linux hosts with the intent to form a Docker swarm cluster. @@ -216,18 +185,30 @@ Also see: * `Docker Home`_ * `Docker Documentation`_ * `Docker Hub`_ -* `Clear Containers`_ +* `Kata Containers`_ -.. _`unique features`: https://clearlinux.org/features -.. _`Clear Linux image on Docker Hub`: https://hub.docker.com/_/clearlinux/ -.. _`building a custom Clear Linux docker image`: https://clearlinux.org/documentation/clear-linux/guides/network/custom-clear-container -.. _`Docker proxy instructions`: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy -.. _`bundles`: https://clearlinux.org/documentation/clear-linux/concepts/bundles-about#related-concepts -.. _`stateless system`: https://clearlinux.org/features/stateless -.. _`Docker documentation on daemon configuration`: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file -.. _`Clear Containers`: https://github.com/clearcontainers -.. _`Docker Home`: https://www.docker.com/ -.. _`Docker Documentation`: https://docs.docker.com/ -.. _`Docker Hub`: https://hub.docker.com/ -.. _`Docker documentation on swarm key concepts`: https://docs.docker.com/engine/swarm/key-concepts/ -.. _`Docker documentation on creating a swarm`: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ +.. _unique features: https://clearlinux.org/features + +.. _Clear Linux image on Docker Hub: https://hub.docker.com/_/clearlinux/ + +.. _building a custom Clear Linux docker image: https://clearlinux.org/documentation/clear-linux/guides/network/custom-clear-container + +.. _Docker proxy instructions: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy + +.. _bundles: https://clearlinux.org/documentation/clear-linux/concepts/bundles-about#related-concepts + +.. _stateless system: https://clearlinux.org/features/stateless + +.. _Docker documentation on daemon configuration: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file + +.. _Kata Containers: https://katacontainers.io/ + +.. _Docker Home: https://www.docker.com/ + +.. _Docker Documentation: https://docs.docker.com/ + +.. _Docker Hub: https://hub.docker.com/ + +.. _Docker documentation on swarm key concepts: https://docs.docker.com/engine/swarm/key-concepts/ + +.. _Docker documentation on creating a swarm: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ diff --git a/source/clear-linux/tutorials/fmv.rst b/source/clear-linux/tutorials/fmv.rst index bd05bd3e..ea6699ed 100644 --- a/source/clear-linux/tutorials/fmv.rst +++ b/source/clear-linux/tutorials/fmv.rst @@ -184,7 +184,7 @@ You can see the multiple clones of the `foo` function: The cloned functions use AVX2 registers and vectorized instructions. To verify, enter the following commands: -.. code-block:: assembly +:: vpaddd (%r8,%rax,1),%ymm0,%ymm0 vmovdqu %ymm0,(%rcx,%rax,1) @@ -213,53 +213,54 @@ To follow the same approach with a package like FFT, we must use the For example, the :file:`fftw-3.3.6-pl2/tools/fftw-wisdom.c.patch` file generates the following patches: -.. code-block:: git +.. code-block:: diff + :linenos: - 1 --- fftw-3.3.6-pl2/libbench2/verify-lib.c 2017-01-27 21:08:13.000000000 +0000 - 2 +++ fftw-3.3.6-pl2/libbench2/verify-lib.c~ 2017-09-27 17:49:21.913802006 +0000 - 3 @@ -33,6 +33,7 @@ - 4 - 5 double dmax(double x, double y) { return (x > y) ? x : y; } - 6 - 7 +__attribute__((target_clones("avx2","arch=atom","default"))) - 8 static double aerror(C *a, C *b, int n) - 9 { - 10 if (n > 0) { - 11 @@ -111,6 +112,7 @@ - 12 } - 13 - 14 /* make array hermitian */ - 15 +__attribute__((target_clones("avx2","arch=atom","default"))) - 16 void mkhermitian(C *A, int rank, const bench_iodim *dim, int stride) - 17 { - 18 if (rank == 0) - 19 @@ -148,6 +150,7 @@ - 20 } - 21 - 22 /* C = A + B */ - 23 +__attribute__((target_clones("avx2","arch=atom","default"))) - 24 void aadd(C *c, C *a, C *b, int n) - 25 { - 26 int i; - 27 @@ -159,6 +162,7 @@ - 28 } - 29 - 30 /* C = A - B */ - 31 +__attribute__((target_clones("avx2","arch=atom","default"))) - 32 void asub(C *c, C *a, C *b, int n) - 33 { - 34 int i; - 35 @@ -170,6 +174,7 @@ - 36 } - 37 - 38 /* B = rotate left A (complex) */ - 39 +__attribute__((target_clones("avx2","arch=atom","default"))) - 40 void arol(C *b, C *a, int n, int nb, int na) - 41 { - 42 int i, ib, ia; - 43 @@ -192,6 +197,7 @@ - 44 } - 45 } + --- fftw-3.3.6-pl2/libbench2/verify-lib.c 2017-01-27 21:08:13.000000000 +0000 + +++ fftw-3.3.6-pl2/libbench2/verify-lib.c~ 2017-09-27 17:49:21.913802006 +0000 + @@ -33,6 +33,7 @@ + + double dmax(double x, double y) { return (x > y) ? x : y; } + + +__attribute__((target_clones("avx2","arch=atom","default"))) + static double aerror(C *a, C *b, int n) + { + if (n > 0) { + @@ -111,6 +112,7 @@ + } + + /* make array hermitian */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void mkhermitian(C *A, int rank, const bench_iodim *dim, int stride) + { + if (rank == 0) + @@ -148,6 +150,7 @@ + } + + /* C = A + B */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void aadd(C *c, C *a, C *b, int n) + { + int i; + @@ -159,6 +162,7 @@ + } + + /* C = A - B */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void asub(C *c, C *a, C *b, int n) + { + int i; + @@ -170,6 +174,7 @@ + } + + /* B = rotate left A (complex) */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void arol(C *b, C *a, int n, int nb, int na) + { + int i, ib, ia; + @@ -192,6 +197,7 @@ + } + } With these patches, we can select where to apply the FMV technology making bringing architecture-based optimizations to application code even easier. diff --git a/source/clear-linux/tutorials/kata.rst b/source/clear-linux/tutorials/kata.rst new file mode 100644 index 00000000..aee38778 --- /dev/null +++ b/source/clear-linux/tutorials/kata.rst @@ -0,0 +1,106 @@ + .. _kata: + +Install Kata Containers\* +######################### + +This tutorial describes how to install, configure, and run `Kata Containers`_ +on |CLOSIA|. Kata Containers is an open source project developing a +lightweight implementation of :abbr:`VMs (Virtual Machines)` that offer the +speed of containers and the security of VMs. + +Prerequisites +************* + +This tutorial assumes you have installed |CL| on your host system. +For detailed instructions on installing |CL| on a bare metal system, follow +the :ref:`bare metal installation tutorial`. + +If you have Clear Containers installed on your |CL| system, then follow the +:ref:`migrate Clear Containers to Kata Containers tutorial`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Install Kata Containers +*********************** + +Kata Containers is included in the :file:`containers-virt` bundle. To install the +framework, enter the following command: + +.. code-block:: bash + + sudo swupd bundle-add containers-virt + +Configure Docker\* to use Kata Containers by default. + +.. code-block:: bash + + sudo mkdir -p /etc/systemd/system/docker.service.d/ + cat <` +* :ref:`telemetry-enable` +* https://github.com/clearlinux/telemetrics-client +* https://github.com/clearlinux/telemetrics-backend .. _`Clear Linux telemetry backend server overview`: https://github.com/clearlinux/telemetrics-backend .. _`Intel privacy policies`: https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index 34f0d9a0..c312c8ad 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -3,6 +3,13 @@ Tutorials ######### +Our Tutorials: + +* Demonstrate how to use |CL| features for cloud, client, distributed + processing, and virtual environments +* Involve specific application of |CL| that often use third-party tools +* Extend the development possibilities of |CL| with specific use cases + This section provides detailed instructions to guide you through completing specific |CLOSIA| use cases. @@ -13,7 +20,7 @@ specific |CLOSIA| use cases. flatpak/flatpak machine-learning/machine-learning docker/docker - azure/azure + azure multi-boot/multi-boot hadoop fmv @@ -21,3 +28,6 @@ specific |CLOSIA| use cases. telemetry-backend/telemetry-backend smb/smb spark + kata + kata_migration + diff --git a/source/clear-linux/tutorials/web-server-install/web-server-install.rst b/source/clear-linux/tutorials/web-server-install/web-server-install.rst index 3a2e814f..3c526468 100644 --- a/source/clear-linux/tutorials/web-server-install/web-server-install.rst +++ b/source/clear-linux/tutorials/web-server-install/web-server-install.rst @@ -1,7 +1,7 @@ .. _web-server-install: Create a Clear Linux\* based web server -##################################### +####################################### This tutorial shows you how to create a LAMP server using |CLOSIA| and how to use phpMyAdmin\* to manage an associated database. diff --git a/source/clear-linux/tutorials/wp-install/wp-install.rst b/source/clear-linux/tutorials/wp-install/wp-install.rst index a30d81f5..b5eda738 100644 --- a/source/clear-linux/tutorials/wp-install/wp-install.rst +++ b/source/clear-linux/tutorials/wp-install/wp-install.rst @@ -153,7 +153,7 @@ Once the installation is complete, you can name your blog and create a Wordpress `Figure 6: The WordPress login screen.` #. Enter your WordPress username and password. -#. Check :guilabel:`Remember me`to save your credentials. +#. Check :guilabel:`Remember me` to save your credentials. #. Click :guilabel:`Log in`. Figure 7 shows the WordPress dashboard after a successful login: diff --git a/source/conf.py b/source/conf.py index b9c7947b..448e260b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -295,3 +295,6 @@ texinfo_documents = [ # If true, generates permalinks on the HTML output. html_add_permalinks = "" + +#suppresses warnings for options that aren't referenced +#suppress_warnings = ["ref.option"]