diff --git a/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md b/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md new file mode 100644 index 00000000..fb6e16a1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/identify--guides--type-by-role----for-guides-only-.md @@ -0,0 +1,20 @@ +--- +name: Identify 'Guides' Type by Role -- For Guides only. +about: For new content in 'Guides', select 1 of 3 categories to which it belongs. + +--- + +New pages will appear here: https://clearlinux.org/documentation/clear-linux/guides + +1. Enter an 'x' in the category for the new "guide": +* [ ] Basics +* [ ] Developer +* [ ] Administrator + +Complete the field below, following the colon, that matches option selected above: + +I am a Clear Linux Beginner (Basics). I want to learn how to: + +I am a Clear Linux Developer. I want to learn how to: + +I am a Clear Linux Administrator. I want to learn how to: diff --git a/guides_feedback b/guides_feedback new file mode 100644 index 00000000..08ef7692 --- /dev/null +++ b/guides_feedback @@ -0,0 +1,12 @@ +1. Review existing 'guides' page: https://clearlinux.org/documentation/clear-linux/guides + +2. If you wish to add new content, use one of the following categories. Fill in the phrase after the colon, +based on the role to which the task applies. + +I am a Clear Linux Beginner, and I want to learn how to: + +I am a Clear Linux Developer, and I want to learn how to: + +I am a Clear Linux Administrator, and I want to learn how to: + + diff --git a/source/clear-linux/concepts/autospec-about.rst b/source/clear-linux/concepts/autospec-about.rst new file mode 100644 index 00000000..a314514c --- /dev/null +++ b/source/clear-linux/concepts/autospec-about.rst @@ -0,0 +1,112 @@ +.. _autospec-about: + +Autospec +######## + +.. _incl-autospec-overview: + +Overview +-------- + +Whereas a standard RPM build process using ``rpmbuild`` requires a tarball +and ``spec`` file to start, ``autospec`` only requires a tarball and package +name. ``autospec`` analyzes the source code and :file:`Makefile` information +in order to generate a ``spec`` file for you. Although not required, you can +influence ``autospec`` by providing control files. + +.. code-block:: console + + buildreq_add + buildreq__ban + pkgconfig_add + pkgconfig_ban + requires_add + requires_ban + options.conf + build_pattern + +These files should be located in same directory as the resulting ``spec`` +file. + +.. note:: + + For a comprehensive list of control files, view the `autospec readme`_. + +.. _incl-autospec-overview-end + +Control files are explained in Table 1. + +.. list-table:: **Table 1. Control Files** + :widths: 20 80 + :header-rows: 1 + + * - Filename + - Description + * - buildreq_add + - Each line in the file provides the name of a package to add as a + build dependency to the ``spec``. + * - buildreq_ban + - Each line in the file is a build dependency that under no + circumstance should be automatically added to the build dependencies. + This is useful to block automatic configuration routines adding + undesired functionality, or to omit any automatically discovered + dependencies during tarball scanning. + * - pkgconfig_add + - Each line in the file is assumed to be a pkgconfig() build + dependency. Add the pkg-config names here, as ``autospec`` will + automatically transform the names into their ``pkgconfig($name)`` + style when generating the ``spec``. + * - pkgconfig_ban + - Each line in this file is a pkgconfig() build dependency that should + not be added automatically to the build, much the same as + `` buildreq_ban``. As with ``pkgconfig_add``, these names are + automatically transformed by ``autospec`` into their correct + ``pkgconfig($name))`` style. + * - requires_add + - Each line in the file provides the name of a package to add as a + runtime dependency to the ``spec``. + * - requires_ban + - Each line in the file is a runtime dependency that under no + circumstance should be automatically added to the runtime + dependencies. This is useful to block automatic configuration + routines adding undesired functionality, or to omit any automatically + discovered dependencies during tarball scanning. + * - build_pattern + - In certain situations, the automatically detected build pattern may + not work for the given package. This one line file allows you to + override the build pattern that ``autospec`` will use. + * - options.conf + - Further control of the build can be achieved through the use of the + ``options.conf`` file. If this file does not exist it is created by + autospec with default values. If certain deprecated configuration + files exists autospec will use the value indicated by those files and + remove them. + +How autospec works +****************** + +Autospec attempts to infer the requirements of the ``spec`` file. If +autospec infers correctly, the control files (Table 1) will automatically +correct the build requirements. The control files are used to influence +the ``spec`` file generation. + +#. The :command:`make autospec` command generates a ``spec`` file from the + control files. + +#. ``autospec`` creates a ``build root`` with ``mock`` config. + +#. ``autospec`` attempts to build an RPM from the generated ``spec`` file. + +#. ``autospec`` detects any missed declarations in the ``spec`` file. + +.. note:: + + * If there are missed declarations, ``autospec`` creates another ``mock`` + ``chroot`` and starts building again at Step 1. + * If a build error occurs, ``autospec`` stops for user inspection. + * If no build errors occur, RPM packages are successfully built. + +``autospec`` continues to rebuild the package, based on new information +discovered from build failures until it has a valid ``spec`` file. + +.. _autospec readme: https://github.com/clearlinux/autospec diff --git a/source/clear-linux/concepts/concepts.rst b/source/clear-linux/concepts/concepts.rst index 476b5c8f..496e31e5 100644 --- a/source/clear-linux/concepts/concepts.rst +++ b/source/clear-linux/concepts/concepts.rst @@ -13,4 +13,6 @@ details relevant to the |CL| features. swupd-about mixer-about bundles-about + autospec-about restart + telemetry-about diff --git a/source/clear-linux/concepts/figures/telemetry-about/telemetry-about-1.png b/source/clear-linux/concepts/figures/telemetry-about/telemetry-about-1.png new file mode 100644 index 00000000..37463254 Binary files /dev/null and b/source/clear-linux/concepts/figures/telemetry-about/telemetry-about-1.png differ diff --git a/source/clear-linux/concepts/swupd-about.rst b/source/clear-linux/concepts/swupd-about.rst index 3f4b8452..c3162de0 100644 --- a/source/clear-linux/concepts/swupd-about.rst +++ b/source/clear-linux/concepts/swupd-about.rst @@ -123,7 +123,7 @@ and core update take merely seconds. While we realize our definition of bundles makes sense to us, data center operators may have special needs and ideas. Therefore, we provide a -:ref:`mixer`. This tool allows users to customize and add bundles +:ref:`mixer tool `. This tool allows users to customize and add bundles or even add their own software, while keeping the operating system and its updates as the basis. Using this tool, system administrators can focus on the custom pieces their deployments require while staying on diff --git a/source/clear-linux/concepts/telemetry-about.rst b/source/clear-linux/concepts/telemetry-about.rst new file mode 100644 index 00000000..b37e6d90 --- /dev/null +++ b/source/clear-linux/concepts/telemetry-about.rst @@ -0,0 +1,82 @@ +.. _telemetry-about: + +Telemetrics +########### + +One of the key features of |CLOSIA| is telemetry, which is used to +monitor system health. Telemetry enables developers to observe and proactively +address issues before end users are impacted. + +*Telemetrics* is a combination word made from: + +* *Telemetry* which is sensing and reporting data. +* *Analytics* which is using visualization and statistical inferencing to make + sense of the reported data. + +|CL| telemetry reports system-level debug/crash information using specialized probes. The +probes monitor system tasks such as :abbr:`swupd (software updater)`, kernel +oops, machine error checks, and BIOS error report table for unhandled hardware +failures. Telemetry enables real-time issue reporting to allow system +developers to quickly focus on an issue and monitor corrective actions. + +|CL| telemetry is fully customizable and can be used during software development +for debugging purposes. You can use **libtelemetry** in your code to create custom +telemetry records. You can also use **telem-record-gen** in script files or call +it from another program. + +Architecture +************ + +|CL| telemetry has two fundamental components, which are shown in figure 1: + +* Client: generates and delivers records to the backend server via the network. +* Backend: captures records sent from the client and displays the cumulative + content through a specialized interface. + + .. note:: + + If you want to capture your own records for analysis, you must set up + your own backend server. + +.. figure:: figures/telemetry-about-1.png + :scale: 75% + :alt: Clear Linux Telemetry Architecture. + + Figure 1: Clear Linux Telemetry Architecture. + +The telemetry client provides the front end of a complete telemetrics solution +and includes the following components: + +* **telemd**, a daemon that prepares the records to send to a telemetrics server or + spools the records on disk in case it cannot successfully deliver them. +* Probes that collect specific types of data from the operating system. +* **libtelemetry**, that telemetrics probes use to create telemetrics records and + send them to the telemd daemon for further processing. + +The telemetry backend provides the server-side component of a complete telemetrics solution and +consists of: + +* Nginx web server. +* Two Flask apps: + + * Collector, an ingestion web app for records received from telemetrics-client probes. + * TelemetryUI, a web app that exposes several views to visualize the telemetry data + and also provides a REST API to perform queries. + +* PostgreSQL as the underlying database server. + +The default telemetry backend server reports back to the |CL| development team +and is not viewable outside the Intel firewall. If you want to collect your +own records, then you must set up your own telemetry backend server. + +Next steps +********** + +To put this concept into practice, see the following resources: + +* :ref:`telemetry-enable` +* :ref:`telemetry-backend` +* `Telemetry feature description`_ + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst index 46a492f6..30e0967d 100644 --- a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst @@ -44,7 +44,7 @@ Follow these steps to install |CL| on the target system: #. Power on the system. -#. Open the system BIOS setup menu, normally by pressing the :kbd:`F2` key. +#. Open the system BIOS setup menu by pressing the :kbd:`F2` key. Your BIOS setup menu entry point may vary. #. In the setup menu, enable the UEFI boot and set the USB drive as the first diff --git a/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst index f646defa..0c5eb624 100755 --- a/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst @@ -1,7 +1,7 @@ .. _cgdisk-manual-install: Create partitions for Clear Linux\* using CGDISK -############################################### +################################################ As part of the |CL| manual installation processThese instructions guide you through the initial setup of your hard drive partitions using the :command:`cgdisk` utility . If you do not wish to continue creating your own @@ -291,4 +291,4 @@ partitions and select to format them. https://clearlinux.org/features/software-update .. _`Linux partitioning scheme`: - https://wiki.archlinux.org/index.php/partitioning#Partition_scheme \ No newline at end of file + https://wiki.archlinux.org/index.php/partitioning#Partition_scheme diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst index 76a551c5..4767c26a 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-linux.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-mac` * :ref:`bootable-usb-windows` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-linux.rst :Start-after: verify-linux: diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst index 29ebe375..2aab17b2 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-mac.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-linux` * :ref:`bootable-usb-windows` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-mac.rst :start-after: verify-mac: diff --git a/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst b/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst index 424c7ebb..d4dbd75e 100644 --- a/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst +++ b/source/clear-linux/get-started/bootable-usb/bootable-usb-windows.rst @@ -13,9 +13,9 @@ Instructions are also available for other operating systems: * :ref:`bootable-usb-mac` * :ref:`bootable-usb-linux` -.. include:: ../../guides/maintenance/image-types.rst - :start-after: for different platforms and environments. - :end-before: Table 1 lists the currently available images. +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename: + :end-before: incl-image-filename-end: .. include:: ../../guides/maintenance/download-verify-uncompress-windows.rst :Start-after: verify-windows: diff --git a/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png b/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png new file mode 100644 index 00000000..0b0ee7f6 Binary files /dev/null and b/source/clear-linux/get-started/virtual-machine-install/figures/download-verify-uncompress-windows-fig-1.png differ diff --git a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst index 96819b32..d090222b 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst @@ -64,7 +64,7 @@ For additional help, see the `VMware Workstation Player guide`_. Clear Linux image types *********************** -.. include:: ../../guides/maintenance/image-types.rst +.. include:: ../../reference/image-types.rst :Start-after: image-types-content: Download the latest Clear Linux VMware image diff --git a/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst b/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst index 5bd5afcd..06dc38ee 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmw-player.rst @@ -74,7 +74,7 @@ For additional help, see the `VMware Workstation Player guide`_. Clear Linux image types *********************** -.. include:: ../../guides/maintenance/image-types.rst +.. include:: ../../reference/image-types.rst :Start-after: image-types-content: diff --git a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst index 92dcf185..e9f35d57 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst @@ -1,36 +1,33 @@ .. _vmware-esxi-preconfigured-cl-image: -Run preconfigured Clear Linux image as a VMware\* ESXi guest OS -############################################################### +Run preconfigured Clear Linux\* image as a VMware\* ESXi guest OS +################################################################# `VMware ESXi`_ is a type 1 bare-metal hypervisor which runs directly on top of server hardware. With VMware ESXi, you can create, configure, manage, -and run |CLOSIA| virtual machines in the cloud. +and run |CLOSIA| virtual machines at scale. This section shows you how to deploy a preconfigured |CL| VMware image on -VMware ESXi 6.5 Update 1 using these steps: +VMware ESXi 6.5. -#. Download the latest Clear Linux VMware image. -#. Verify the integrity of the |CL| image. -#. Uncompress the |CL| image. -#. Upload the Clear Linux image to the VMware server. -#. Convert the Clear Linux image to an ESXi-supported format. -#. Create and configure a new VM. -#. Power on the VM and boot |CL|. - -If you would prefer to perform a fresh installation of |CL| into a new VMware -ESXi :abbr:`VM (Virtual Machine)` instead, see :ref:`vmware-esxi-install-cl`. +If you would prefer to perform a manual installation of |CL| into a new +VMware ESXi :abbr:`VM (Virtual Machine)` instead, see +:ref:`vmware-esxi-install-cl`. Visit :ref:`image-types` to learn more about +the available images. .. note:: - VMware also offers a type 2 hypervisor called `VMware Workstation Player`_ which is designed for the desktop environment. + VMware also offers a type 2 hypervisor called `VMware Workstation Player` + which is designed for the desktop environment. + See :ref:`vmw-player-preconf` or see :ref:`vmw-player`. -Visit :ref:`image-types` to learn more about the available images. +.. contents:: :local: + :depth: 2 Download the latest Clear Linux VMware image ******************************************** -Get the latest |CL| VMware image from the `image`_ repository. +Get the latest |CL| VMware prebuilt image from the `image`_ repository. Look for :file:`clear-[version number]-vmware.vmdk.xz`. You can also use this command: @@ -50,59 +47,87 @@ For alternative instructions on other operating systems, see: Upload the Clear Linux image to the VMware server ************************************************* -#. Connect to the VMware server and login to an account with sufficient - permission to create and manage VMs. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Storage`. - See Figure 1. -#. Under the :guilabel:`Datastores` tab, click - the :guilabel:`Datastore browser` button. +Once the |CL| VMware prebuilt image has been downloaded and +uncompressed on your local system, it must be uploaded to a datastore +on the VMware ESXi server. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-1.png +The steps in this section can also be referenced from the `VMware documentation on Using the Datastore File Browser`_ + +#. Connect to the VMware ESXi server and login to an account with sufficient + permission to create and manage VMs. + +#. Under the :guilabel:`Navigator` window on the left side, + select :guilabel:`Storage`. + See Figure 1 + +#. Under the :guilabel:`Datastores` tab, click + the :guilabel:`Datastore browser` button. + + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-1.png :scale: 100 % :alt: VMware ESXi - Navigator > Storage Figure 1: VMware ESXi - Navigator > Storage -#. Click the :guilabel:`Create directory` button and name the directory - `Clear Linux VM`. See Figure 2. +#. Click the :guilabel:`Create directory` button and name the directory + `Clear Linux VM`. See Figure 2. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-2.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-2.png :scale: 100 % :alt: VMware ESXi - Datastore > Create directory Figure 2: VMware ESXi - Datastore > Create directory -#. Select the newly-created directory and click the :guilabel:`Upload` - button. See Figure 3. +#. Select the newly-created directory and click the :guilabel:`Upload` + button. See Figure 3. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-3.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-3.png :scale: 100 % :alt: VMware ESXi - Datastore > Upload VMware image Figure 3: VMware ESXi - Datastore > Upload VMware image -#. Select the uncompressed |CL| VMware image file - :file:`clear-[version number]-vmware.vmdk` and upload it. +#. Select the uncompressed |CL| VMware image file + :file:`clear-[version number]-vmware.vmdk` and upload it. Convert the Clear Linux image to an ESXi-supported format ********************************************************* -#. SSH into the VMware server and login to an account with root privileges. -#. Locate the uploaded image, which is typically found in `/vmfs/volumes/datastore1`. -#. Use the :command:`vmkfstools` command to perform the conversion, as - shown below: +Once the |CL| VMware prebuilt image has been uploaded to the VMware ESXi +datastore, it must be converted to a format for usable with VMware's ESXi +hypervisor. - .. code-block:: console +The steps in this section can also be referenced from the `VMware documentation on Cloning and converting virtual machine disks with vmkfstools`_ - # vmkfstools -i clear-[version number]-vmware.vmdk -d zeroedthick clear-[version number]-esxi.vmdk +#. SSH into the `vSphere Management Assistant`_ appliance that is managing + the ESXi host or connect to the vSphere hosting using the `vSphere CLI`_. + + .. note:: + + If there is no :abbr:`vMA (vSphere Management Assistant)` appliance or :abbr:`vCLI (vSphere CLI)` configured and available, + you can temporarily enable SSH directly on the ESXi host by referencing + the `VMware documentation on Enable the Secure Shell (SSH)`_ . - Two files should result from this: + As a security best practice, remember to disable SSH access after following the steps in this section. - * :file:`clear-[version number]-esxi-flat.vmdk` - * :file:`clear-[version number]-esxi.vmdk` - The :file:`clear-[version number]-esxi.vmdk` file will be used in the - next section when you create a new VM. +#. Locate the uploaded image, which is typically found in + :file:`/vmfs/volumes/datastore1`. + +#. Use the :command:`vmkfstools` command to perform the conversion, as + shown below: + + .. code-block:: console + + vmkfstools -i clear-[version number]-vmware.vmdk -d zeroedthick clear-[version number]-esxi.vmdk + + Two files should result from this: + + * :file:`clear-[version number]-esxi-flat.vmdk` + * :file:`clear-[version number]-esxi.vmdk` + + The :file:`clear-[version number]-esxi.vmdk` file will be used in the + next section when you create a new VM. Create and configure a new VM ***************************** @@ -111,138 +136,153 @@ In this section, you will create a new VM, configure its basic parameters such as number of CPUs, memory size, and then attach the converted |CL| VMware image. Also, in order to boot |CL|, you must enable UEFI support. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. See Figure 4. +#. Under the :guilabel:`Navigator` window, select + :guilabel:`Virtual Machines`. See Figure 4. -#. In the right window, click the :guilabel:`Create / Register VM` button. +#. In the right window, click the :guilabel:`Create / Register VM` button. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-4.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-4.png :scale: 100 % :alt: VMware ESXi - Navigator > Virtual Machines Figure 4: VMware ESXi - Navigator > Virtual Machines -#. On the :guilabel:`Select creation type` step: +#. On the :guilabel:`Select creation type` step: - #. Select the :guilabel:`Create a new virtual machine` option. See - Figure 5. + #. Select the :guilabel:`Create a new virtual machine` option. See + Figure 5. - #. Click the :guilabel:`Next` button. + #. Click the :guilabel:`Next` button. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-5.png - :scale: 100 % - :alt: VMware ESXi - Create a new virtual machine + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-5.png + :scale: 100 % + :alt: VMware ESXi - Create a new virtual machine - Figure 5: VMware ESXi - Create a new virtual machine + Figure 5: VMware ESXi - Create a new virtual machine -#. On the :guilabel:`Select a name and guest OS` step: +#. On the :guilabel:`Select a name and guest OS` step: - #. Give the new VM a name in the :guilabel:`Name` field. See Figure 6. - #. Set the :guilabel:`Compatability` option to :guilabel:`ESXi 6.5 virtual machine`. - #. Set the :guilabel:`Guest OS family` option to :guilabel:`Linux`. - #. Set the :guilabel:`Guest OS version` option to :guilabel:`Other 3.x or later Linux (64-bit)`. - #. Click the :guilabel:`Next` button. +#. Give the new VM a name in the :guilabel:`Name` field. See Figure 6. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-6.png - :scale: 100 % - :alt: VMware ESXi - Give a name and select guest OS type +#. Set the :guilabel:`Compatability` option to + :guilabel:`ESXi 6.5 virtual machine`. +#. Set the :guilabel:`Guest OS family` option to :guilabel:`Linux`. +#. Set the :guilabel:`Guest OS version` option to + :guilabel:`Other 3.x or later Linux (64-bit)`. +#. Click the :guilabel:`Next` button. - Figure 6: VMware ESXi - Give a name and select guest OS type + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-6.png + :scale: 100 % + :alt: VMware ESXi - Give a name and select guest OS type -#. On the :guilabel:`Select storage` step: + Figure 6: VMware ESXi - Give a name and select guest OS type - #. Accept the default option. - #. Click the :guilabel:`Next` button. +#. On the :guilabel:`Select storage` step: -#. On the :guilabel:`Customize settings` step: + #. Accept the default option. + #. Click the :guilabel:`Next` button. - #. Click the :guilabel:`Virtual Hardware` button. See Figure 7. - #. Expand the :guilabel:`CPU` setting and enable :guilabel:`Hardware virtualization` by - checking :guilabel:`Expose hardware assisted virtualization to the guest OS`. +#. On the :guilabel:`Customize settings` step: - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-7.png - :scale: 100 % - :alt: VMware ESXi - Enable hardware virtualization + #. Click the :guilabel:`Virtual Hardware` button. See Figure 7. + #. Expand the :guilabel:`CPU` setting and enable + :guilabel:`Hardware virtualization` by checking + :guilabel:`Expose hardware assisted virtualization to the guest OS`. - Figure 7: VMware ESXi - Enable hardware virtualization + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-7.png + :scale: 100 % + :alt: VMware ESXi - Enable hardware virtualization - #. Remove the default :guilabel:`Hard drive 1` setting by clicking - the `X` icon on the right side. See Figure 8. + Figure 7: VMware ESXi - Enable hardware virtualization - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-8.png - :scale: 100 % - :alt: VMware ESXi - Remove hard drive + #. Remove the default :guilabel:`Hard drive 1` setting by clicking + the `X` icon on the right side. See Figure 8. - Figure 8: VMware ESXi - Remove hard drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-8.png + :scale: 100 % + :alt: VMware ESXi - Remove hard drive - #. Since a preconfigured image will be used, - the :guilabel:`CD/DVD Drive 1` setting will not be needed. Disable it by unchecking the :guilabel:`Connect` checkbox. See Figure 9. + Figure 8: VMware ESXi - Remove hard drive - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-9.png - :scale: 100 % - :alt: VMware ESXi - Disconnect the CD/DVD drive + #. Since a preconfigured image will be used, + the :guilabel:`CD/DVD Drive 1` setting will not be needed. Disable it + by unchecking the :guilabel:`Connect` checkbox. See Figure 9. - Figure 9: VMware ESXi - Disconnect the CD/DVD drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-9.png + :scale: 100 % + :alt: VMware ESXi - Disconnect the CD/DVD drive - #. Attach the :file:`clear-[version number]-esxi.vmdk` file that was - converted from the preconfigured |CL| VMware image. + Figure 9: VMware ESXi - Disconnect the CD/DVD drive - #. Click the :guilabel:`Add hard disk` button and select the - :guilabel:`Existing hard drive` option. See Figure 10. + #. Attach the :file:`clear-[version number]-esxi.vmdk` file that was + converted from the preconfigured |CL| VMware image. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-10.png - :scale: 100 % - :alt: VMware ESXi - Add an existing hard drive + #. Click the :guilabel:`Add hard disk` button and select the + :guilabel:`Existing hard drive` option. See Figure 10. - Figure 10: VMware ESXi - Add an existing hard drive + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-10.png + :scale: 100 % + :alt: VMware ESXi - Add an existing hard drive - #. Select the converted :file:`clear-[version number]-esxi.vmdk` - file. Do not use the original unconverted :file:`clear-[version number]-vmware.vmdk` file. See Figure 11. + Figure 10: VMware ESXi - Add an existing hard drive - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-11.png - :scale: 100 % - :alt: VMware ESXi - Select the converted `vmdk` file + #. Select the converted :file:`clear-[version number]-esxi.vmdk` + file. Do not use the original unconverted + :file:`clear-[version number]-vmware.vmdk` file. See Figure 11. - Figure 11: VMware ESXi - Select the converted :file:`clear-[version number]-esxi.vmdk` file + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-11.png + :scale: 100 % + :alt: VMware ESXi - Select the converted `vmdk` file -#. |CL| needs UEFI support in order to boot. Enable UEFI boot support. + Figure 11: VMware ESXi - Select the converted + :file:`clear-[version number]-esxi.vmdk` file - #. Click the :guilabel:`VM Options` button. See Figure 12. - #. Expand the :guilabel:`Boot Options` setting. - #. For the :guilabel:`Firmware` setting, click the drop-down list to - the right of it and select the :guilabel:`EFI` option. +#. |CL| needs UEFI support in order to boot. Enable UEFI boot support. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-12.png - :scale: 100 % - :alt: VMware ESXi - Set boot firmware to EFI + #. Click the :guilabel:`VM Options` button. See Figure 12. + #. Expand the :guilabel:`Boot Options` setting. + #. For the :guilabel:`Firmware` setting, click the drop-down list to + the right of it and select the :guilabel:`EFI` option. - Figure 12: VMware ESXi - Set boot firmware to EFI + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-12.png + :scale: 100 % + :alt: VMware ESXi - Set boot firmware to EFI -#. Click the :guilabel:`Save` button. -#. Click the :guilabel:`Next` button. -#. Click the :guilabel:`Finish` button. + Figure 12: VMware ESXi - Set boot firmware to EFI + +#. Click the :guilabel:`Save` button. +#. Click the :guilabel:`Next` button. +#. Click the :guilabel:`Finish` button. Power on the VM and boot Clear Linux ************************************ After configuring the settings above, power on the VM. -#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. See Figure 13. -#. In the right window, select the newly-created VM. -#. Click the :guilabel:`Power on` button. -#. Click on the icon representing the VM to bring it into view and maximize - its window. +#. Under the :guilabel:`Navigator` window, select + :guilabel:`Virtual Machines`. See Figure 13. +#. In the right window, select the newly-created VM. +#. Click the :guilabel:`Power on` button. +#. Click on the icon representing the VM to bring it into view and maximize + its window. - .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-13.png + .. figure:: figures/vmware-esxi/vmware-esxi-preconfigured-cl-image-13.png :scale: 100 % :alt: VMware ESXi - Navigator > Virtual Machines > Power on VM Figure 13: VMware ESXi - Navigator > Virtual Machines > Power on VM Also see: -********* +========= * :ref:`vmware-esxi-install-cl` .. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html +.. _VMware documentation on Using the Datastore File Browser: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.html.hostclient.doc/GUID-7533A767-8396-4844-A3F2-206047D254EA.html +.. _vSphere Management Assistant: https://www.vmware.com/support/developer/vima/ +.. _vSphere CLI: https://www.vmware.com/support/developer/vcli/ +.. _VMware documentation on Cloning and converting virtual machine disks with vmkfstools: https://kb.vmware.com/kb/1028042 +.. _VMware documentation on Enable the Secure Shell (SSH): https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.html.hostclient.doc/GUID-B649CB74-832F-467B-B6A4-8BA67AD5C1F0.html +.. _VMware documentation on General ESXi Security Recommendations: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.security.doc/GUID-B39474AF-6778-499A-B8AB-E973BE6D4899.html .. _VMware Workstation Player: https://www.vmware.com/products/workstation-player.html .. _image: https://download.clearlinux.org/image diff --git a/source/clear-linux/guides/maintenance/autospec.rst b/source/clear-linux/guides/maintenance/autospec.rst new file mode 100644 index 00000000..29ed5da7 --- /dev/null +++ b/source/clear-linux/guides/maintenance/autospec.rst @@ -0,0 +1,318 @@ +.. _autospec: + +Build RPMs with autospec +######################## + +This guide shows you how to create RPMs with :ref:`autospec ` +, a tool that assists in automated creation and maintenance of RPM packaging +on |CLOSIA|. Additionally, you learn how to use these RPMs to create bundles +with :ref:`mixer `. + +Prerequisites +************* +This guide assumes that you have: + +* Created :ref:`a custom mix ` of |CL| and deployed it to a + to a target device + +* |CL| running on a host machine or virtual environment + + .. note:: + + To install |CL|, see: + + * :ref:`bare-metal-install` + * :ref:`virtual-machine-install` + +Install Clear Linux tooling framework +===================================== + +Our GitHub\* repository provides you with the resources you need +to create and maintain packages. + +#. On your host system, install this developer bundle. + + .. code-block:: bash + + sudo swupd bundle-add os-clr-on-clr + +#. Run this command to download the :file:`user-setup.sh` script. + + .. code-block:: bash + + curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh + +#. Make :file:`user-setup.sh` executable. + + .. code-block:: bash + + chmod +x user-setup.sh + +#. Run the script as an unprivileged user. + + .. code-block:: bash + + ./user-setup.sh + +#. After the script completes, log out and log in again to complete + the setup process. + + The `user-setup script`_ creates a folder called :file:`clearlinux`, which + contains the :file:`Makefile`, :file:`packages`, and :file:`projects` + subfolders. + + The :file:`projects` folder contains the main tools, `autospec` + and `common`, used for making packages in |CL|. + +Create a RPM with autospec +************************** + +.. include:: ../../concepts/autospec-about.rst + :Start-after: incl-autospec-overview: + :end-before: incl-autospec-overview-end: + +For a detailed explanation of how ``autospec`` works on |CL|, visit our +:ref:`autospec-about` about page. For a general understanding of how RPMs +work, we recommend visiting the `rpm website`_ or the +`RPM Packaging Guide`_ . + +Building RPMs +============= + +Choose one of the following options to build RPMs and manage source +code: + +* :ref:`build-a-new-rpm` and spec file using ``make autospecnew``. + +* :ref:`build-source-code-with-existing-spec-file` (without changing the + spec file) using ``make build``. + +* :ref:`generate-a-new-spec-file` based on changes in the control files with + ``make autospec``. + +.. _build-a-new-rpm: + +Build a new RPM +=============== + +#. Navigate to the autospec workspace. + + .. code-block:: bash + + cd ~/clearlinux + +#. Enter the command: + + .. code-block:: bash + + make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" + NAME="helloclear" + + .. note:: + + For a local tarball, use for the *URL*: + file:// + +#. If build failures or dependency issues occur, continue below. + Otherwise, skip directly to `copy-rpm-packages-to-mixer`_. + + #. Navigate to the specific package. + + .. code-block:: bash + + cd ~/clearlinux/packages/[package-name] + + #. Respond to the build process output by editing control files to resolve + issues, which may include dependencies or exclusions. + See `autospec readme`_ + + #. Run this command: + + .. code-block:: bash + + make autospec + + Repeat the last two steps above until all errors are resolved and you + complete a successful build. + + Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. + +.. _build-source-code-with-existing-spec-file: + +Build source code with an existing spec file +============================================ + +If you only want to build the RPM using the spec file, use this method. This +method assumes that a spec file already exists. In this example, we run a +``make build`` on the ``dmidecode`` package. + +#. Navigate to the ``dmidecode`` package in clearlinux: + + .. code-block:: bash + + cd ~/clearlinux/packages/dmidecode/ + +#. To download the tarball and build, run the command: + + .. code-block:: bash + + make build + + Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. + +.. _generate-a-new-spec-file: + +Generate a new spec file with a pre-defined package +=================================================== + +In this method, you will modify an existing |CL| package called ``dmidecode`` +to create a custom RPM. You will make a simple change to this package, +change the revision to a new number that is higher than the |CL| OS version, +and rebuild the package. + +#. Navigate to clearlinux: + + .. code-block:: bash + + cd ~/clearlinux + +#. Copy the ``dmidecode`` package. + + .. code-block:: bash + + make clone_dmidecode + +#. Navigate into the *dmidecode* directory: + + .. code-block:: bash + + cd packages/dmidecode + +#. With an editor, open the :file:`excludes` file and add these lines: + + .. code-block:: bash + + /usr/bin/biosdecode + /usr/bin/ownership + /usr/bin/vpddecode + /usr/share/man/man8/biosdecode.8 + /usr/share/man/man8/ownership.8 + /usr/share/man/man8/vpddecode.8 + + .. note:: + + These files aren't needed by dmidecode, so we can remove them without + any issues. + +#. Save the file and exit. + +#. At :file:`~/clearlinux/packages/dmidecode`, build the modified + ``dmidecode`` package: + + .. code-block:: bash + + make autospec + + When the process completes, you will see new RPM packages in the + :file:`results/` folder. + +#. To view the new RPM packages, enter: + + .. code-block:: bash + + ls /clearlinux/packages/dmidecode/results/ + +Add a custom RPM to a mix and deploy to target +********************************************** + +We need a RPM repository to store our custom RPMs. This repository also +includes some metadata that allows programs such as ``yum`` and ``dnf`` to +follow and include any specified dependencies. This architecture enables us +to test custom RPMs before we integrate them in a mix. + +.. note:: + + Assure that you followed the :ref:`mixer` instruction and created + a location for **local RPM packages** using the *--local-rpms* flag + with the command: :command:`mixer init --local-rpms`. If you skipped + this step, return and complete it in :ref:`mixer` before proceeding. + +.. _copy-rpm-packages-to-mixer: + +Copy RPM packages to mixer and build bundle +============================================ + +Transfer the newly generated RPM packages to the ``mixer`` folder so +that it can include them as needed. + +.. note:: + + This guide assumes that you have a web server that hosts ``swupd`` update + content. + +#. Change directory into the mix workspace: + + .. code-block:: bash + + cd ~/mix + +#. Copy the contents from the results folder in the RPM packages to the + :file:`local-rpms` folder in the :file:`mix` folder: + + .. code-block:: bash + + cp ~/clearlinux/packages/dmidecode/results/*x86_64*rpm ~/mix/local-rpms/ + +#. Remove the debuginfo: + + .. code-block:: bash + + rm ~/mix/local-rpms/*debuginfo*x86_64* + +#. Generate the yum repo: + + .. code-block:: bash + + sudo mixer add-rpms + +#. Create a local bundle definition file to include the newly generated RPM + package in your mix. In our example, the ``[bundle-name]`` is + either ``dmidecode`` or ``helloclear``. + + .. code-block:: bash + + mixer bundle edit [bundle-name] + +#. Then add the new bundle to the mix. + + .. code-block:: bash + + mixer bundle add [bundle-name] + +#. Build the bundle and update content. + + .. code-block:: bash + + sudo mixer build all + +#. Log into the target device. + +#. On the target device, update and install the new bundle. + + .. code-block:: bash + + sudo swupd update + + sudo swupd bundle-add [bundle-name] + +**Congratulations!** + +You successfully built a RPM and created a mix with it. + +.. _rpm website: http://rpm.org + +.. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/ + +.. _user-setup script: https://github.com/clearlinux/common/blob/master/user-setup.sh + +.. _autospec readme: https://github.com/clearlinux/autospec diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst index f497148c..e90c0170 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-linux.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-linux: @@ -42,6 +42,9 @@ checksum, a warning is displayed with a message indicating the computed checksum does **not** match. Otherwise, the name of the image is printed on the screen followed by `OK`. +For a more in-depth discussion of image verification including checking the +certificate see :ref:`image-content-validation`. + Uncompress the Clear Linux image ******************************** diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst index d37346ef..d94be7b6 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-mac.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-mac: diff --git a/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst b/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst index 24d7c7fa..03b892fb 100644 --- a/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst +++ b/source/clear-linux/guides/maintenance/download-verify-uncompress-windows.rst @@ -14,7 +14,7 @@ Instructions for other operating systems are available: Image types *********** -.. include:: image-types.rst +.. include:: ../../reference/image-types.rst :start-after: image-types-content: .. _verify-windows: diff --git a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png index db5a2f3f..a4692c9a 100755 Binary files a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png and b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-1.png differ diff --git a/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png new file mode 100644 index 00000000..90b1eb28 Binary files /dev/null and b/source/clear-linux/guides/maintenance/figures/increase-virtual-disk-size-2.png differ diff --git a/source/clear-linux/guides/maintenance/image-types.rst b/source/clear-linux/guides/maintenance/image-types.rst deleted file mode 100644 index e3a119f5..00000000 --- a/source/clear-linux/guides/maintenance/image-types.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. _image-types: - -Clear Linux image types -####################### - -.. _image-types-content: - -|CLOSIA| offers many types of `images`_ for different platforms and environments. - -The naming convention of a |CL| image filename is: - - .. code-block:: console - - clear-[version number]-[image type].[compression type] - -The *[version number]* field specifies the version number. - -The *[image type]* field specifies the type of image and its corresponding -file format. - -The *[compression type]* field specifies the compression type. Two types of -compressions are used: GNU\* zip (*.gz*) and XZ (*.xz*). - -Table 1 lists the currently available images. - -.. list-table:: Table 1: Types of Clear Linux images - :widths: 20, 60 - :header-rows: 1 - - * - Image Type - - Description - * - azure.vhd - - Image for Microsoft\* Azure\* - * - azure-docker.vhd - - Image with Docker\* installed for Microsoft Azure - * - azure-machine-learning.vhd - - Image with the `machine-learning-basic` bundle installed for Microsoft - Azure - * - cloud.img - - Image for cloud deployment such as OpenStack\* - * - containers.img - - Optimized image used by Clear Containers runtime - * - hyperv.vhdx - - Image for Microsoft Hyper-V\* - * - hyperv-mini.vhdx - - Minimal image with fewer bundles for Microsoft Hyper-V - * - installer.img - - Interactive installer image - * - installer.iso - - ISO of the interactive installer image - * - kvm.img - - Image for booting in a simple VM with start_qemu.sh - * - live.img - - Live boot image - * - vmware.vmdk - - Image for VMware\* - -.. _images: https://download.clearlinux.org/image diff --git a/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst b/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst index 7b991faa..a676e615 100644 --- a/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst +++ b/source/clear-linux/guides/maintenance/increase-virtual-disk-size.rst @@ -1,126 +1,169 @@ .. _increase-virtual-disk-size: -Increase Clear Linux image virtual disk size -############################################ +Increase virtual disk size of a Clear Linux\* image +################################################### -Prebuilt |CLOSIA| images come in different sizes, ranging from 300 MB to 20 +|CLOSIA| prebuilt images come in different sizes, ranging from 300 MB to 20 GB. This guide describes how to increase the size of your prebuilt |CL| image if you need more capacity. -Determine the prebuilt image size -********************************* +.. contents:: This guide will cover: -There are two methods to find the virtual disk size of your prebuilt |CL| -image. +Determine the partition order and sizes of the prebuilt image +************************************************************* -The first method is to check the config.JSON file of the image, located in the -`releases`_ repository. For example, to find the size of the Hyper-V\* image -version number 20450, follow these steps: +There are two methods to find the order and sizes of partitions virtual disk +of your prebuilt |CL| image. -#. Go to the `releases`_ repository. -#. Drill down into the `20450 > clear > config > image` directory. -#. Open the :file:`hyperv-config.json` file. -#. Locate the `PartitionLayout` key. - The example shows 512 MB for the EFI partition, 32 MB for the swap - partition, and 8 GB for the root partition. +In both examples, the prebuilt Hyper-V image has a disk size of 8.5 GB with / +dev/sda3 being the partition for the root filesystem (/) - .. code-block:: console +Checking :command:`lsblk` on the VM +----------------------------------- - "PartitionLayout" : [ { "disk" : "hyperv.img", - "partition" : 1, - "size" : "512M", - "type" : "EFI" }, - { "disk" : "hyperv.img", - "partition" : 2, - "size" : "32M", - "type" : "swap" }, - { "disk" : "hyperv.img", - "partition" : 3, - "size" : "8G", - "type" : "linux" } ], - -The second method is to boot up your :abbr:`VM (Virtual Machine)` and +The first method is to boot up your :abbr:`VM (Virtual Machine)` and execute the :command:`lsblk` command as shown below: .. code-block:: bash - sudo lsblk + sudo lsblk -An example output: +An example output of the :command:`lsblk` command: .. code-block:: console - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - sda 8:0 0 8.5G 0 disk - ├─sdd1 8:1 0 512M 0 part - ├─sdd2 8:2 0 32M 0 part [SWAP] - └─sdd3 8:3 0 8G 0 part / + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + sda 8:0 0 8.5G 0 disk + ├─sda1 8:1 0 512M 0 part + ├─sda2 8:2 0 32M 0 part [SWAP] + └─sda3 8:3 0 8G 0 part / -In this example, both methods show the prebuilt Hyper-V image is about 8.5 GB. +An example of this can also be seen in Figure 1. + +Checking :file:`config.json` used to build the image +---------------------------------------------------- + +The second method to determine partition to check the :file:`config.json` +file used to create prebuilt image, located in the `releases`_ repository. +For example, to find the size of the Hyper-V\* image version number 20450, +follow these steps: + +#. Go to the `releases`_ repository. +#. Drill down into the `20450 > clear > config > image` directory. +#. Open the :file:`hyperv-config.json` file. +#. Locate the `PartitionLayout` key. + + The example shows 512 MB for the EFI partition, 32 MB for the swap + partition, and 8 GB for the root partition. + + + .. code-block:: console + + "PartitionLayout" : [ { "disk" : "hyperv.img", + "partition" : 1, + "size" : "512M", + "type" : "EFI" }, + { "disk" : "hyperv.img", + "partition" : 2, + "size" : "32M", + "type" : "swap" }, + { "disk" : "hyperv.img", + "partition" : 3, + "size" : "8G", + "type" : "linux" } ], Increase virtual disk size ************************** +Once you have determined the disk and partition to be increased, you are +ready to perform the actual increase of the disk, partition, and filesystem. -To increase the virtual disk size for a prebuilt image, perform the steps below: +Power off VM and increase virtual disk size: +-------------------------------------------- -#. Shut down your VM if it is running. -#. Use an appropriate hypervisor tool to increase the virtual disk size of - your VM. -#. Power up the VM. -#. Log in to an account with root privileges. -#. Open a terminal emulator. -#. Add the |CL| `storage-utils` bundle to install the `parted` and - `resize2fs` tools. +To increase the virtual disk size for a prebuilt image, perform the steps +below: - .. code-block:: bash +#. Shut down your VM if it is running. +#. Use the process defined by your hypervisor + or cloud provider to increase + the virtual disk size of your |CL| VM. +#. Power up the VM. - sudo swupd bundle-add storage-utils -#. Launch the `parted` tool. +Resize the partition of the virtual disk: +----------------------------------------- - .. code-block:: bash +#. Log in to an account with root privileges. +#. Open a terminal emulator. +#. Add the |CL| `storage-utils` bundle to install the + :command:`parted` and :command:`resize2fs` tools. - sudo parted + .. code-block:: bash -#. In the `parted` tool, perform these steps: + sudo swupd bundle-add storage-utils - #. Press :command:`p` to print the partitions table. - #. If the warning message below is displayed, enter :command:`Fix`. +#. Launch the `parted` tool. - .. code-block:: console + .. code-block:: bash - Warning: Not all of the space available to /dev/sda appears to be - used, you can fix the GPT to use all of the space (an extra ... - blocks) or continue with the current setting? + sudo parted - Fix/Ignore? +#. In the `parted` tool, perform these steps: - #. Enter :command:`resizepart [partition number]` where - *[partition number]* is the partition number to modify. - #. Enter :command:`yes` when prompted. - #. Enter the new `End` size. + #. Press :command:`p` to print the partitions table. + #. If the warning message below is displayed, enter :command:`Fix`. - .. note:: + .. code-block:: console - If you want a partition to take up the remaining disk space, then - enter the total size of the disk. When you print the partitions - table with the :command:`p` command, the total disk size is shown - after the `Disk` label. + Warning: Not all of the space available to /dev/sda appears to be + used, you can fix the GPT to use all of the space (an extra ... + blocks) or continue with the current setting? - #. Enter :command:`q` to exit `parted` when you are finished resizing the - image. + Fix/Ignore? -#. Enter :command:`sudo resize2fs -p /dev/[modified partition name]` where - *[modified partition name]* is the partition that was changed in `parted`. + #. Enter :command:`resizepart [partition number]` where + *[partition number]* is the partition number of the partition to modify. + #. Enter :command:`yes` when prompted. + #. Enter the new `End` size. -Figure 1 shows how to increase the size of a |CL| Hyper-V image from 8.5 -GB to 20 GB. Before the steps shown in Figure 1, we used the Hyper-V Manager -to increase the VM virtual disk size from 8.5 GB to 20 GB. + .. note:: -.. figure:: figures/increase-virtual-disk-size-1.png - :scale: 100 % - :alt: Increase root partition size example + If you want a partition to take up the remaining disk space, then + enter the total size of the disk. When you print the partitions + table with the :command:`p` command, the total disk size is shown + after the `Disk` label. - Figure 1: Increase root partition size example. + An example of this can be seen in Figure 1. -.. _releases: https://download.clearlinux.org/releases/ + #. Enter :command:`q` to exit `parted` when you are finished resizing the + image. + + Figure 1 depicts the described steps to resize the partition of the virtual disk from 8.5GB to 20GB. + + .. figure:: figures/increase-virtual-disk-size-1.png + :scale: 100 % + :alt: Increase root partition size + + Figure 1: Increase root partition size. + +Resize the filesytem +-------------------- + +#. Enter :command:`sudo resize2fs -p /dev/[modified partition name]` where + *[modified partition name]* is the partition that was changed in `parted`. + +#. Run the :command:`df -h` to verify that the filesystem size has + increased. + + Figure 2 depicts the described steps to resize the partition of the virtual disk from 8.5GB to 20GB. + + .. figure:: figures/increase-virtual-disk-size-2.png + :scale: 100 % + :alt: Increase root filesystem with resize2fs + + Figure 2: Increase root filesystem size after partition has been expanded. + +Congratulations! You have resized the disk, partition, and filesystem. At +this point, the increase in disk capacity is usable. + +.. _releases: https://download.clearlinux.org/releases/ \ No newline at end of file diff --git a/source/clear-linux/guides/maintenance/maintenance.rst b/source/clear-linux/guides/maintenance/maintenance.rst index 8775a909..b65cadbe 100644 --- a/source/clear-linux/guides/maintenance/maintenance.rst +++ b/source/clear-linux/guides/maintenance/maintenance.rst @@ -10,7 +10,7 @@ maintaining |CLOSIA| after :ref:`installation ` is completed. :maxdepth: 2 enable-user-space - update + swupd-guide bulk-provision mixer mixin @@ -21,4 +21,4 @@ maintaining |CLOSIA| after :ref:`installation ` is completed. download-verify-uncompress-linux download-verify-uncompress-mac download-verify-uncompress-windows - image-types + autospec diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 90e54dec..a2eaa0e4 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -16,35 +16,26 @@ To start working with the mixer tools, you need a recent image of |CL| with the `mixer` bundle installed. If the bundle is not yet installed, you can add it with the :command:`swupd bundle-add` command as follows: - .. code-block:: bash +.. code-block:: bash - sudo swupd bundle-add mixer + sudo swupd bundle-add mixer Current mixing workflow *********************** There are two different workflows to create your own mix. +First, if your mix only uses |CL| content, *skip step 5* below. +Second, if your mix includes your own +:abbr:`RPMs (RPM Package Manager files)`, follow all these steps. -The following workflow applies if the mix includes your own -:abbr:`RPMs (RPM Package Manager files)`: - -#. `Create a workspace`_ -#. `Generate the starting point for your mix`_ -#. `Edit builder.conf`_ -#. `Create or locate RPMs for the mix`_ -#. `Import RPMs into workspace`_ -#. `Create a local RPM repo`_ -#. `List, edit, create, add, remove, or validate bundles`_ -#. `Build the bundle chroots`_ -#. `Create an update`_ -#. `Create an image`_ - -Alternatively, the following workflow applies if the mix only uses |CL| -content. - +#. `Create ngninx web server to host mixer updates`_ #. `Create a workspace`_ #. `Generate the starting point for your mix`_ #. `Edit builder.conf`_ +#. `Create custom RPMs`_ + #. `Create or locate RPMs for the mix`_ + #. `Import RPMs into workspace`_ + #. `Create a local RPM repo`_ #. `List, edit, create, add, remove, or validate bundles`_ #. `Build the bundle chroots`_ #. `Create an update`_ @@ -53,6 +44,114 @@ content. The following sections contain detailed information on every step of these workflows. +.. _create-nginx-web-server: + +Create ngninx web server to host mixer updates +********************************************** + +Follow these steps to set up a HTTP service with ``nginx`` web +server, where you can host custom |CL| mixes. + +#. Install ``web-server-basic``: + + .. code-block:: bash + + sudo swupd bundle-add web-server-basic + +#. Make the directory where mixer updates will reside. + + .. code-block:: bash + + sudo mkdir -p /var/www + +#. Create a symbolic link. + + .. code-block:: bash + + sudo ln -sf $HOME/mixer/update/www /var/www/mixer + +#. Set up ``nginx`` configuration. + + .. code-block:: bash + + sudo mkdir -p /etc/nginx/conf.d + +#. Copy the default example configuration file. + + .. code-block:: bash + + sudo cp -f /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf + +#. Open ``mixer.conf`` with an editor. + + .. code-block:: bash + + sudo nano /etc/nginx/conf.d/mixer.conf + +#. Add the server configuration content below to ``mixer.conf``. + + .. code-block:: console + + server { + server_name localhost; + location / { + root /var/www/mixer; + autoindex on; + } + } + +#. Restart the daemon, enable nginx on boot, and start the service. + + .. code-block:: bash + + sudo systemctl daemon-reload + + sudo systemctl enable nginx + + sudo systemctl start nginx + +#. To verify the web server is running, enter in an Internet browser: + + .. code-block:: bash + + http://localhost + +#. Alternatively, verify the server is running by entering: + + .. code-block:: bash + + ps aux | grep nginx + + .. note:: + + If `nginx` is active, a list should appear showing one master process + and a few worker processes. + +Connect the URL to mixer +======================== + +Add the URL of the `nginx` server to builder.conf. Your |CL| clients connect +to this URL to find the update content. + +#. Get the IP address of your nginx server: + + .. code-block:: bash + + networkctl status + +#. In the mixer workspace, edit `builder.conf` to set the value for + CONTENTURL and VERSIONURL to the IP `Address` of your `nginx` server. + + .. code-block:: bash + + nano builder.conf + + .. note:: + + For example: + CONTNENTURL=http://192.168.25.52 + VERSIONURL=http://192.168.25.52 + Create a workspace ****************** @@ -103,9 +202,8 @@ directories in your mix workspace and adds their paths to the generated the paths manually. For more information on using these directories or setting them up manually, see `Create or locate RPMs for the mix`_. -If all upstream |CL| bundles will be part of the mix, you can easily add them all -during initialization with the optional :option:`--all-upstream` flag. For -example: +If all upstream |CL| bundles will be part of the mix, you can easily add +them all during initialization with the optional :option:`--all-upstream` flag. For example: .. code-block:: bash @@ -197,8 +295,8 @@ The following variables require further explanation: the folder at all. * The `YUM_CONF` variable sets the path where mixer automatically generates - the :file:`.yum-mix.conf` yum configuration file. The yum configuration file - points the chroot-builder to the path where the RPMs are stored. + the :file:`.yum-mix.conf` yum configuration file. The yum configuration + file points the chroot-builder to the path where the RPMs are stored. * The `CERT` variable sets the path where mixer stores the :file:`Swupd_Root.pem` certificate file. The chroot-builder needs the @@ -206,9 +304,9 @@ The following variables require further explanation: security for content verification. The value of the `CERT` variable can point to a different certificate. The chroot-builder inserts the certificate specified in this value into the - :file:`/os-core-update/usr/share/clear/update-ca/` path. The software update - client uses this certificate to verify the :file:`Manifest.MoM` file's - signature. For now, we **strongly** recommend that you do not modify + :file:`/os-core-update/usr/share/clear/update-ca/` path. The software + update client uses this certificate to verify the :file:`Manifest.MoM` + file's signature. For now, we **strongly** recommend that you do not modify this line, as the certificate that :abbr:`swupd (Software Updater)` expects needs to have a very specific configuration to sign and verify properly. Mixer automatically generates the certificate, if you do not @@ -216,14 +314,14 @@ The following variables require further explanation: file to provide security for the updated content you create. * The `CONTENTURL` and `VERSIONURL` variables set the domain or IP address - where swupd looks for your update content and the corresponding version. You - must set these variables to the domain or IP-address of the server hosting the - update content. You can use any web server to host your update content. To learn - how to install and configure web server using |CL|, visit - :ref:`web-server-install`. For our example, the web update content within - the `SERVER_STATE_DIR` directory is located here: - :file:`/home/clr/mix/update/www`. If the web server is on the same machine as - this directory, you can create a symlink to the directory in your web + where swupd looks for your update content and the corresponding version. + You must set these variables to the domain or IP-address of the server + hosting the update content. You can use any web server to host your update + content. To learn how to install and configure web server using |CL|, see + :ref:`create-nginx-web-server`. For our example, the web update content + within the `SERVER_STATE_DIR` directory is located here: + :file:`/home/clr/mix/update/www`. If the web server is on the same machine + as this directory, you can create a symlink to the directory in your web server's document root to easily host the content. These URLs are embedded in images created for your mix. The `swupd-client` looks at these URLs to determine if a new version is available and the location @@ -242,8 +340,12 @@ The following variables require further explanation: .. note:: If you are working only with |CL| bundles, then skip to `List, edit, create, add, remove, or validate bundles`_. + +Create custom RPMs +****************** + Create or locate RPMs for the mix -********************************* +================================= If you create RPMs from scratch, you can use `autospec`, `mock`, `rpmbuild`, or similar tools to build them. If the RPMs are not built on |CL|, ensure @@ -252,7 +354,7 @@ there is no guarantee they will be compatible. For more information on building the RPMs properly, refer to our `build RPMs instructions`_. Import RPMs into workspace -************************** +========================== #. Create a :file:`local-rpms` directory in your workspace, for example, :file:`/home/clr/mix/local-rpms`. @@ -269,7 +371,7 @@ Mixer uses this directory to find the RPMs to build a local RPM repo for yum to use. Create a local RPM repo -*********************** +======================= #. Create an empty directory in your workspace named :file:`local-yum`. #. Add the path to your :file:`builder.conf` file: @@ -416,7 +518,8 @@ as part of the bundle. .. note:: The :command:`mixer bundle edit` command accepts multiple bundles at once. - Thus, you can create multiple new bundles in a single command, for example: + Thus, you can create multiple new bundles in a single command, for + example: .. code-block:: bash @@ -486,8 +589,8 @@ Validate the bundles in the mix Mixer performs basic validation on all bundles when used throughout the system. -Mixer checks the validity of the bundle's syntax and name. Optionally, you can -run this validation manually on `bundle1` with the following command: +Mixer checks the validity of the bundle's syntax and name. Optionally, you +can run this validation manually on `bundle1` with the following command: .. code-block:: bash @@ -746,7 +849,8 @@ modifications as needed, for example: .. code-block:: bash - sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + sudo mixer-pack-maker.sh --to --from -S / + home/clr/mix/update .. _mixer-format: diff --git a/source/clear-linux/guides/maintenance/swupd-guide.rst b/source/clear-linux/guides/maintenance/swupd-guide.rst new file mode 100644 index 00000000..db385152 --- /dev/null +++ b/source/clear-linux/guides/maintenance/swupd-guide.rst @@ -0,0 +1,183 @@ +.. _swupd-guide: + +Use swupd +######### + +*swupd* manages the software update capability of |CLOSIA|. It can check for +valid system updates and, if found, download and install them. It can also +perform verification of the system software. + +|CL| uses :ref:`bundles-about` as the base abstraction for +installing functionality on top of the core operating system. Use the `swupd` +tool to install and remove bundles. + +This guide assumes: + +* The device is on a well-connected network. + +* The device is able to connect to the release infrastructure + http://update.clearlinux.org + +.. contents:: + :local: + :depth: 2 + +.. _man_swupd: + +For a full listing of commands and options please see the man page in |CL| + +.. code-block:: bash + + man swupd + +OS update and verification +************************** + +|CL| is designed to promote a regular update cadence. `swupd` helps to +make sure that process is simple and secure. + +View OS info +============ + +Current OS version and update server info: + +.. code-block:: bash + + swupd info + +.. code-block:: console + + Installed version: 23330 + Version URL: https://download.clearlinux.org/update/ + Content URL: https://cdn.download.clearlinux.org/update/ + +Enable or disable automatic updates +=================================== + +|CL| updates are automatic by default but can be set to occur only +on demand. To verify your current auto-update setting: + +.. code-block:: bash + + sudo swupd autoupdate + +Disable automatic updates: + +.. code-block:: bash + + sudo swupd autoupdate --disable + +To re-enable automatic: + +.. code-block:: bash + + sudo swupd autoupdate --enable + +Check for updates +================= + +.. code-block:: bash + + sudo swupd check-update + +Perform a manual update +======================= + +You can update to a specific version or accept the latest as the default with +no arguments. Initiate a manual update: + +.. code-block:: bash + + sudo swupd update -m 23330 + +.. note:: + + After a kernel update, reboot immediately for the enhancements to + take effect. + +System Software Verification +============================ + +`swupd` can determine whether system directories and files have been added +to, overwritten, removed, or modified (e.g., permissions). + +.. code-block:: bash + + sudo swupd verify + +All directories that are watched by `swupd` are verified according to +the manifest data and hash mismatches are flagged as follows: + +.. code-block:: console + + Verifying version 23300 + Verifying files + ...0% + Hash mismatch for file: /usr/bin/chardetect + ... + ... + Hash mismatch for file: /usr/lib/python3.6/site-packages/urllib3/util/wait.py + ...100% + Inspected 237180 files + 423 files did not match + Verify successful + +In this case, python packages that were installed on top of the default +install were flagged as mismatched. `swupd` can be directed to ignore +or fix issues based on :ref:`command line options `. + +Fixing hash mismatches +====================== + +`swupd` can correct any issues it detects. Additional directives can be +added including a white list of directories that will be ignored, if +desired. + +The following command will repair issues, remove unknown items, and +ignore files or directories matching `/usr/lib/python`: + +.. code-block:: bash + + sudo swupd verify --fix --picky --picky-whitelist=/usr/lib/python + +Bundles +******* + +Listing installed bundles +========================= + +You can list all of the bundles currently installed on the system + +.. code-block:: bash + + sudo swupd bundle-list --all + +Finding a bundle containing a binary +==================================== + +Run the following to display a list of bundles that contain a particular +binary. Note that it may be present in multiple bundles: + +.. code-block:: bash + + swupd search -b + +Adding a bundle +=============== + +Start by selecting a bundle from the list of :ref:`available-bundles`. In +this example we're adding dev-utils-dev, which is useful for development. + +.. code-block:: bash + + sudo swupd bundle-add dev-utils-dev + +Removing a bundle +================= + +Dependencies common to other bundles will not be removed + +.. code-block:: bash + + sudo swupd bundle-remove dev-utils-dev + diff --git a/source/clear-linux/guides/maintenance/telemetry-enable.rst b/source/clear-linux/guides/maintenance/telemetry-enable.rst index 3f6ed348..6d401226 100644 --- a/source/clear-linux/guides/maintenance/telemetry-enable.rst +++ b/source/clear-linux/guides/maintenance/telemetry-enable.rst @@ -5,11 +5,11 @@ Enable and disable telemetry in Clear Linux |CLOSIA| includes a telemetry solution as part of the OS that records events of interest and reports them back to the development team via the telemetrics -daemon, :command:`telemd`. This functionality is maintained in the -``telemetrics`` software bundle. +daemon, **telemd**. This functionality is maintained in the +**telemetrics** software bundle. .. note:: - The telemetry functionality adheres to `Intel's privacy policies`_ + The telemetry functionality adheres to `Intel privacy policies`_ regarding the collection and use of :abbr:`PII (Personally Identifiable Information)` and is open source. Specifically, no intentionally identifiable information about the user or system owner is collected. @@ -20,10 +20,10 @@ redirect where the records go if they wish to collect records for themselves. Install the telemetry software bundle ************************************* -During the initial installation of |CL| you are requested to join the +During the initial installation of |CL|, you are requested to join the stability enhancement program and allow |CLOSIA| to collect anonymous reports -to improve system stability. If you chose not to join this program at that -time then the telemetry software bundle is not added to your system. +to improve system stability. If you choose not to join this program, then the +telemetry software bundle is not added to your system. To install the telemetry bundle, enter the following command as either the root user or with :command:`sudo` privileges: @@ -104,9 +104,13 @@ To completely remove telemetrics from your system, use the command Additional resources ******************** -https://clearlinux.org/features/telemetry +* `Telemetry feature description`_ +* :ref:`Telemetry architecture` +* :ref:`telemetry-backend` +* https://github.com/clearlinux/telemetrics-client -https://github.com/clearlinux/telemetrics-client - -.. _`Intel's privacy policies`: +.. _`Intel privacy policies`: https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/guides/maintenance/update.rst b/source/clear-linux/guides/maintenance/update.rst deleted file mode 100644 index 5705fda3..00000000 --- a/source/clear-linux/guides/maintenance/update.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. _update: - -Update Clear Linux -################## - -This section describes how to update |CLOSIA|. - -Prerequisites -============= - -* The system is up and running the Clear Linux 15400 release or later. - -* The device is on a well-connected network. - -* The device is able to connect to the release infrastructure - http://update.clearlinux.org - -To verify the current release running on the system, enter: - -.. code-block:: console - - $ sudo swupd update -s - -How to update the system -======================== - -Starting with version 15400, Clear Linux supports auto-update. By default, it -is turned on. - -To verify the current auto-update setting: - - .. code-block:: console - - $ swupd autoupdate - -To enable auto-update: - - .. code-block:: console - - $ sudo swupd autoupdate --enable - -To disable auto-update: - - .. code-block:: console - - $ sudo swupd autoupdate --disable - -To force a manual update: - - .. code-block:: console - - $ sudo swupd update - -.. note:: - - When the swupd console output indicates a kernel update, reboot - immediately for the enhancements to take effect. - - -Additional information -====================== - -To see the man page listing additional swupd options, install the -`sysadmin-basic` bundle to access the `man` command and later access -the man page: - - .. code-block:: console - - $ sudo swupd bundle-add sysadmin-basic - $ man swupd diff --git a/source/clear-linux/guides/maintenance/validate-signatures.rst b/source/clear-linux/guides/maintenance/validate-signatures.rst index 190e4f59..1001369a 100644 --- a/source/clear-linux/guides/maintenance/validate-signatures.rst +++ b/source/clear-linux/guides/maintenance/validate-signatures.rst @@ -9,9 +9,11 @@ creates a chain of trust. A broken chain of trust, seen as an invalid signature, means the content is not valid. This guide covers how to validate the contents of an image, which is a manual -process, and describes the automatic process ``swupd`` performs internally to +process and is the same process ``swupd`` performs internally to validate an update. +.. _image-content-validation: + Image content validation ======================== diff --git a/source/clear-linux/guides/network/vnc.rst b/source/clear-linux/guides/network/vnc.rst index e2f1e21d..8de6f3ef 100644 --- a/source/clear-linux/guides/network/vnc.rst +++ b/source/clear-linux/guides/network/vnc.rst @@ -674,12 +674,12 @@ For `Method 3`: Figure 7 shows two VNC sessions (5901 and 5905) accepting connections from any host as specified by the `0.0.0.0`'s. This is before the `-localhost` option was used. - + .. figure:: figures/vnc/vnc-7.png :scale: 100 % :alt: VNC session accepting connection from any host - Figure 7: VNC sessions (5901 and 5905) accepting connections from any host + Figure 7: VNC sessions (5901 and 5905) accepting connections from any host Figure 8 shows two VNC sessions (5901 and 5905) only accepting connections from localhost as specified by `127.0.0.1`'s. This is after the `-localhost` option was used. diff --git a/source/clear-linux/reference/collaboration/documentation/contents.rst b/source/clear-linux/reference/collaboration/documentation/contents.rst new file mode 100644 index 00000000..e88c4d7c --- /dev/null +++ b/source/clear-linux/reference/collaboration/documentation/contents.rst @@ -0,0 +1,40 @@ +.. _contents: + + +Contents directive +################## + +For |CL| documentation that has three or more sections, use the `contents::` +directive as shown in the example below. This directive automatically captures the headings (and +subheadings if used) as specified in the value given after `:depth:`. Adding this directive to +longer documents allows users to quickly navigate to the desired section. + +.. contents:: :local: + :depth: 2 + +.. code-block:: bash + + .. contents:: :local: + :depth: 2 + +.. note:: + + Assure that you add `:local:` as the value. For more resources on this directive, + visit the `reStruturedText Directives`_ + +EXAMPLE: + +Clear Linux Guide Example +************************* + +Introduction +============ + +Step-by-Step +============ + +Launch +====== + + +.. _reStruturedText Directives: http://docutils.sourceforge.net/0.4/docs/ref/rst/directives.html#table-of-contents diff --git a/source/clear-linux/reference/collaboration/documentation/rest.rst b/source/clear-linux/reference/collaboration/documentation/rest.rst index a11522bc..3b1a81f6 100644 --- a/source/clear-linux/reference/collaboration/documentation/rest.rst +++ b/source/clear-linux/reference/collaboration/documentation/rest.rst @@ -36,6 +36,7 @@ templates supplied. cross images code + contents .. _Sphinx documentation: diff --git a/source/clear-linux/reference/collaboration/documentation/tables.rst b/source/clear-linux/reference/collaboration/documentation/tables.rst index 3ad59298..29d0b2c2 100644 --- a/source/clear-linux/reference/collaboration/documentation/tables.rst +++ b/source/clear-linux/reference/collaboration/documentation/tables.rst @@ -18,7 +18,7 @@ Follow these general guidelines: it is not rendered. * Only create a table if the body of the table contains six or more cells, -which is a minimum table size of at least 2x3 or 3x2. + which is a minimum table size of at least 2x3 or 3x2. ReST supports several types of tables. |CL| uses grid and :abbr:`CSV-tables (Comma Separated Values tables)`. Grid tables are only diff --git a/source/clear-linux/reference/compatible-kernels.rst b/source/clear-linux/reference/compatible-kernels.rst index caa5fe28..72c6072e 100644 --- a/source/clear-linux/reference/compatible-kernels.rst +++ b/source/clear-linux/reference/compatible-kernels.rst @@ -1,7 +1,7 @@ .. _compatible-kernels: -Compatible Clear Linux kernels -############################## +Compatible Clear Linux\* kernels +################################ The |CLOSIA| provides the following Linux kernels with a respective bundle. This document describes the specific use cases these `bundles`_ serve @@ -19,7 +19,7 @@ Kernel Container ================ The *kernel-container* bundle contains the kernel used by the -`Intel® Clear Containers`_ project. This kernel is optimized for +Intel® Clear Containers project. This kernel is optimized for fast booting and performs best on |CC| running on the Intel® architectures described on the :ref:`supported hardware list`. The optimization patches are found in our `Linux-Container`_ GitHub repo. @@ -44,11 +44,10 @@ The *kernel-kvm* bundle focuses on the Linux :abbr:`KVM (Kernel-based Virtual Machine)`. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. -Use this kernel when running |CL| as the guest OS -on top of *qemu/kvm*. Use this kernel with **cloud orchestrators** using -*qemu/kvm* internally as their **hypervisor**. -This kernel can be used as a stand alone Linux VM, see our -:ref:`instructions on using KVM` for more information. The +Use this kernel when running |CL| as the guest OS on top of *qemu/kvm*. Use +this kernel with **cloud orchestrators** using *qemu/kvm* internally as +their **hypervisor**. This kernel can be used as a standalone |CL| VM, see +our :ref:`instructions on using KVM` for more information. The optimization patches are found in our `Linux-KVM`_ GitHub repo. Kernel Hyper-V\* @@ -59,8 +58,8 @@ Hyper-V. It is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in -projects such as Microsoft `Azure`_\*. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +projects such as Microsoft `Azure`_\*. This kernel can be used in a +standalone |CL| VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV`_ GitHub repo. @@ -73,8 +72,8 @@ is optimized for fast booting and performs best on Virtual Machines running on the Intel® architectures described on the :ref:`supported hardware list`. Use this kernel when running |CL| as the guest OS of **Cloud Instances** in -projects such as Microsoft `Azure`_. This kernel can be used as a stand -alone Linux VM, see our :ref:`instructions on using Hyper-V` for +projects such as Microsoft `Azure`_. This kernel can be used in a standalone +|CL| VM, see our :ref:`instructions on using Hyper-V` for more information. The optimization patches are found in our `Linux-HyperV-LTS`_ GitHub repo. @@ -89,5 +88,4 @@ more information. The optimization patches are found in our .. _CIAO: https://github.com/01org/ciao .. _Azure: https://azuremarketplace.microsoft.com/en-us/marketplace/apps/clear-linux-project.clear-linux-os -.. _Intel® Clear Containers: - https://clearlinux.org/features/intel®-clear-containers + diff --git a/source/clear-linux/reference/image-types.rst b/source/clear-linux/reference/image-types.rst new file mode 100644 index 00000000..50f81dfc --- /dev/null +++ b/source/clear-linux/reference/image-types.rst @@ -0,0 +1,81 @@ +.. _image-types: + +Clear Linux\* image types +######################### + +.. _image-types-content: + +|CLOSIA| offers many types of `images`_ for different platforms and environments. + +.. _incl-image-filename: + +The naming convention of a |CL| image filename is: + +.. code-block:: console + + clear-[version number]-[image type].[compression type] + +* The *[version number]* field specifies the version number. + +* The *[image type]* field specifies the type of image and its corresponding + file format. + +* The *[compression type]* field specifies the compression type. Two types of + compressions are used: GNU\* zip (*.gz*) and XZ (*.xz*). + +.. _incl-image-filename-end: + +Table 1 lists the currently available images that are platform independent. +Table 2 lists the currently available images that are platform specific. + +.. list-table:: Table 1: Types of platform-independent Clear Linux images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - installer.img + - Preferred image of Clear Linux with interactive installer. + + * - installer.iso + - ISO of Clear Linux with interactive installer. Only for special cases where ISO image format is required (not for use with a USB key) + + * - live.img + - image for live booting into memory, without requiring installaton. + +.. list-table:: Table 2: Types of platform-specific Clear Linux images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - azure.vhd + - Virtual Hard Disk for use on Microsoft\* Azure\* cloud platform + + * - azure-docker.vhd + - Virtual Hard Disk for use on Microsoft Azure cloud platform with Docker\* pre-installed + + * - azure-machine-learning.vhd + - Virtual Hard Disk for use on Microsoft Azure cloud platform with the `machine-learning-basic` bundle installed + + * - cloud.img + - Image for use by cloud deployments such as OpenStack\* + + * - containers.img + - Image for use by Clear Containers runtime. Includes `optimized kernel`_ for Clear Containers. + + * - hyperv.vhdx + - Virtual Hard Disk for use with Microsoft Hyper-V\* hypervisor. Includes `optimized kernel`_ for Hyper-V. + + * - kvm.img + - Image for booting in a simple VM with start_qemu.sh. Includes + `optimized kernel`_ for KVM. + + * - vmware.vmdk + - Virtual Machine Disk for VMware\* platforms inclduing Player, Workstation, and ESXi. + +.. _images: https://download.clearlinux.org/image +.. _`optimized kernel`: https://clearlinux.org/documentation/clear-linux/reference/compatible-kernels + diff --git a/source/clear-linux/tutorials/aws-web/aws-web.rst b/source/clear-linux/tutorials/aws-web/aws-web.rst index 8c433212..25b9b1af 100644 --- a/source/clear-linux/tutorials/aws-web/aws-web.rst +++ b/source/clear-linux/tutorials/aws-web/aws-web.rst @@ -18,9 +18,11 @@ Prerequisites ************* This tutorial assumes the following statements are true: + * You are using a linux-based system to access AWS and can run :command:`SSH` to access the remote |CL| AWS image. -* Your browser puts downloaded files in the :file:`$HOME/Downloads` directory. +* Your browser puts downloaded files in the :file:`$HOME/Downloads` + directory. * You have already set up an AWS user account and logged into the AWS console. diff --git a/source/clear-linux/tutorials/azure/azure.rst b/source/clear-linux/tutorials/azure.rst similarity index 99% rename from source/clear-linux/tutorials/azure/azure.rst rename to source/clear-linux/tutorials/azure.rst index be19af9c..168fbcfd 100644 --- a/source/clear-linux/tutorials/azure/azure.rst +++ b/source/clear-linux/tutorials/azure.rst @@ -269,7 +269,7 @@ Log into your Microsoft Azure account #. Open your browser and enter the page `https://aka.ms/devicelogin` as shown in figure 1: - .. figure:: figures/azure-1.png + .. figure:: azure/figures/azure-1.png :scale: 50 % :alt: Microsoft Device Login @@ -280,7 +280,7 @@ Log into your Microsoft Azure account session login and will be different each time you log into MS Azure using the :command:`az login` command. - .. figure:: figures/azure-2.png + .. figure:: azure/figures/azure-2.png :scale: 50 % :alt: Microsoft Device Login - Azure CLI @@ -294,7 +294,7 @@ Log into your Microsoft Azure account have signed into the Microsoft Cross-platform Command Line Interface application on your device and you can close the window. - .. figure:: figures/azure-3.png + .. figure:: azure/figures/azure-3.png :scale: 50 % :alt: Microsoft Azure Cross-platform CLI diff --git a/source/clear-linux/tutorials/docker/docker.rst b/source/clear-linux/tutorials/docker/docker.rst index 75c00e35..6c6cae2a 100644 --- a/source/clear-linux/tutorials/docker/docker.rst +++ b/source/clear-linux/tutorials/docker/docker.rst @@ -4,8 +4,8 @@ Run Docker\* on Clear Linux\* ############################# |CLOSIA| supports multiple containerization platforms, including a Docker\* -solution.|CL| has many `unique features`_ including a minimal default -installation which makes it compelling to use as a host for container +solution. |CL| has many `unique features`_ including a minimal default +installation, which makes it compelling to use as a host for container workloads, management, and orchestration. This tutorial covers: @@ -68,48 +68,17 @@ packages to run Docker images as containers. Congratulations! At this point, you have a working installation of Docker on |CL|. You are ready to start using container images on your system. -Integration with Clear Containers (optional) +Integration with Kata Containers (optional) ******************************************** -`Clear Containers`_, is an open source project aiming to increase security -of containers by using a hardware-backed virtual machine container runtime, -instead of software namespace containers that are provided by the standard +`Kata Containers`_, is an open source project aiming to increase security +of containers by using a hardware-backed virtual machine container runtime +rather than software namespace containers that are provided by the standard Docker *runc* runtime. -The Docker package from |CL| will automatically use the *cc-runtime* -required for Clear Containers if it is available on your Clear Linux system. +Clear Linux provides easy integration of the *kata-runtime* with Docker. +More information on installing and using the *kata-runtime* may be found at :ref:`kata`. -#. To take advantage of Clear Containers in |CL|, install the - *containers-virt* bundle by running this command: - - .. code-block:: bash - - sudo swupd bundle-add containers-virt - -#. Restart the Docker daemon through systemd manager by running this command: - - .. code-block:: bash - - sudo systemctl restart docker - -#. After restarting, the Docker daemon will seamlessly use Clear Containers - to launch containers. You can see the runtime has changed to - :command:`cc-runtime` by running this command: - - .. code-block:: bash - - sudo docker info | grep Runtime - -#. You should see the following output indicating the *cc-runtime* is the - Default Runtime: - - .. code-block:: console - - Runtimes: cc-runtime runc - Default Runtime: cc-runtime - -Congratulations! At this point, you have successfully replaced the default -container runtime with Clear Containers. .. note:: @@ -168,7 +137,7 @@ an the official Docker image for nginx, an open source reverse proxy server. .. note:: - Below is an explaination of switches used in the command above. For + Below is an explanation of switches used in the command above. For detailed :command:`docker run` switches and syntax, refer to the `Docker Documentation`_ . @@ -197,12 +166,12 @@ an the official Docker image for nginx, an open source reverse proxy server. sudo docker rm test-nginx Congratulations! At this point, you have successfully pulled a nginx -container image from `Docker Hub`_ and ran an example container. +container image from `Docker Hub`_ and have run an example container. Creating a Docker swarm cluster ******************************* -Clusters of Docker hosts are referred to as swarms. +Clusters of Docker hosts are referred to as *swarms*. The process in this tutorial can be repeated to install Docker on multiple Clear Linux hosts with the intent to form a Docker swarm cluster. @@ -216,18 +185,30 @@ Also see: * `Docker Home`_ * `Docker Documentation`_ * `Docker Hub`_ -* `Clear Containers`_ +* `Kata Containers`_ -.. _`unique features`: https://clearlinux.org/features -.. _`Clear Linux image on Docker Hub`: https://hub.docker.com/_/clearlinux/ -.. _`building a custom Clear Linux docker image`: https://clearlinux.org/documentation/clear-linux/guides/network/custom-clear-container -.. _`Docker proxy instructions`: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy -.. _`bundles`: https://clearlinux.org/documentation/clear-linux/concepts/bundles-about#related-concepts -.. _`stateless system`: https://clearlinux.org/features/stateless -.. _`Docker documentation on daemon configuration`: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file -.. _`Clear Containers`: https://github.com/clearcontainers -.. _`Docker Home`: https://www.docker.com/ -.. _`Docker Documentation`: https://docs.docker.com/ -.. _`Docker Hub`: https://hub.docker.com/ -.. _`Docker documentation on swarm key concepts`: https://docs.docker.com/engine/swarm/key-concepts/ -.. _`Docker documentation on creating a swarm`: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ +.. _unique features: https://clearlinux.org/features + +.. _Clear Linux image on Docker Hub: https://hub.docker.com/_/clearlinux/ + +.. _building a custom Clear Linux docker image: https://clearlinux.org/documentation/clear-linux/guides/network/custom-clear-container + +.. _Docker proxy instructions: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy + +.. _bundles: https://clearlinux.org/documentation/clear-linux/concepts/bundles-about#related-concepts + +.. _stateless system: https://clearlinux.org/features/stateless + +.. _Docker documentation on daemon configuration: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file + +.. _Kata Containers: https://katacontainers.io/ + +.. _Docker Home: https://www.docker.com/ + +.. _Docker Documentation: https://docs.docker.com/ + +.. _Docker Hub: https://hub.docker.com/ + +.. _Docker documentation on swarm key concepts: https://docs.docker.com/engine/swarm/key-concepts/ + +.. _Docker documentation on creating a swarm: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ diff --git a/source/clear-linux/tutorials/kata.rst b/source/clear-linux/tutorials/kata.rst new file mode 100644 index 00000000..aee38778 --- /dev/null +++ b/source/clear-linux/tutorials/kata.rst @@ -0,0 +1,106 @@ + .. _kata: + +Install Kata Containers\* +######################### + +This tutorial describes how to install, configure, and run `Kata Containers`_ +on |CLOSIA|. Kata Containers is an open source project developing a +lightweight implementation of :abbr:`VMs (Virtual Machines)` that offer the +speed of containers and the security of VMs. + +Prerequisites +************* + +This tutorial assumes you have installed |CL| on your host system. +For detailed instructions on installing |CL| on a bare metal system, follow +the :ref:`bare metal installation tutorial`. + +If you have Clear Containers installed on your |CL| system, then follow the +:ref:`migrate Clear Containers to Kata Containers tutorial`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Install Kata Containers +*********************** + +Kata Containers is included in the :file:`containers-virt` bundle. To install the +framework, enter the following command: + +.. code-block:: bash + + sudo swupd bundle-add containers-virt + +Configure Docker\* to use Kata Containers by default. + +.. code-block:: bash + + sudo mkdir -p /etc/systemd/system/docker.service.d/ + cat <` +* :ref:`telemetry-enable` +* https://github.com/clearlinux/telemetrics-client +* https://github.com/clearlinux/telemetrics-backend .. _`Clear Linux telemetry backend server overview`: https://github.com/clearlinux/telemetrics-backend .. _`Intel privacy policies`: https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html + +.. _`Telemetry feature description`: + https://clearlinux.org/features/telemetry diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index 34f0d9a0..9bc120ad 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -13,7 +13,7 @@ specific |CLOSIA| use cases. flatpak/flatpak machine-learning/machine-learning docker/docker - azure/azure + azure multi-boot/multi-boot hadoop fmv @@ -21,3 +21,6 @@ specific |CLOSIA| use cases. telemetry-backend/telemetry-backend smb/smb spark + kata + kata_migration + diff --git a/source/clear-linux/tutorials/web-server-install/web-server-install.rst b/source/clear-linux/tutorials/web-server-install/web-server-install.rst index 3a2e814f..3c526468 100644 --- a/source/clear-linux/tutorials/web-server-install/web-server-install.rst +++ b/source/clear-linux/tutorials/web-server-install/web-server-install.rst @@ -1,7 +1,7 @@ .. _web-server-install: Create a Clear Linux\* based web server -##################################### +####################################### This tutorial shows you how to create a LAMP server using |CLOSIA| and how to use phpMyAdmin\* to manage an associated database. diff --git a/source/clear-linux/tutorials/wp-install/wp-install.rst b/source/clear-linux/tutorials/wp-install/wp-install.rst index a30d81f5..b5eda738 100644 --- a/source/clear-linux/tutorials/wp-install/wp-install.rst +++ b/source/clear-linux/tutorials/wp-install/wp-install.rst @@ -153,7 +153,7 @@ Once the installation is complete, you can name your blog and create a Wordpress `Figure 6: The WordPress login screen.` #. Enter your WordPress username and password. -#. Check :guilabel:`Remember me`to save your credentials. +#. Check :guilabel:`Remember me` to save your credentials. #. Click :guilabel:`Log in`. Figure 7 shows the WordPress dashboard after a successful login: