diff --git a/source/clear-linux/concepts/autospec-about.rst b/source/clear-linux/concepts/autospec-about.rst index 6b1c3bc3..61575fe2 100644 --- a/source/clear-linux/concepts/autospec-about.rst +++ b/source/clear-linux/concepts/autospec-about.rst @@ -1,112 +1,107 @@ -.. _autospec-about: +.. _autospec-about: Autospec ######## -.. _incl-autospec-overview: - -Overview -******** - -Whereas a standard RPM build process using ``rpmbuild`` requires a tarball -and ``spec`` file to start, ``autospec`` only requires a tarball and package -name. ``autospec`` analyzes the source code and :file:`Makefile` information -in order to generate a ``spec`` file for you. Although not required, you can -influence ``autospec`` by providing control files. - -.. code-block:: console - - buildreq_add - buildreq__ban - pkgconfig_add - pkgconfig_ban - requires_add - requires_ban - options.conf - build_pattern - -These files should be located in same directory as the resulting ``spec`` -file. - -.. note:: - - For a comprehensive list of control files, view the `autospec readme`_. - -.. _incl-autospec-overview-end: - -Control files are explained in Table 1. - -.. list-table:: **Table 1. Control Files** - :widths: 20 80 - :header-rows: 1 - - * - Filename - - Description - * - buildreq_add - - Each line in the file provides the name of a package to add as a - build dependency to the ``spec``. - * - buildreq_ban - - Each line in the file is a build dependency that under no - circumstance should be automatically added to the build dependencies. - This is useful to block automatic configuration routines adding - undesired functionality, or to omit any automatically discovered - dependencies during tarball scanning. - * - pkgconfig_add - - Each line in the file is assumed to be a pkgconfig() build - dependency. Add the pkg-config names here, as ``autospec`` will - automatically transform the names into their ``pkgconfig($name)`` - style when generating the ``spec``. - * - pkgconfig_ban - - Each line in this file is a pkgconfig() build dependency that should - not be added automatically to the build, much the same as - `` buildreq_ban``. As with ``pkgconfig_add``, these names are - automatically transformed by ``autospec`` into their correct - ``pkgconfig($name))`` style. - * - requires_add - - Each line in the file provides the name of a package to add as a - runtime dependency to the ``spec``. - * - requires_ban - - Each line in the file is a runtime dependency that under no - circumstance should be automatically added to the runtime - dependencies. This is useful to block automatic configuration - routines adding undesired functionality, or to omit any automatically - discovered dependencies during tarball scanning. - * - build_pattern - - In certain situations, the automatically detected build pattern may - not work for the given package. This one line file allows you to - override the build pattern that ``autospec`` will use. - * - options.conf - - Further control of the build can be achieved through the use of the - ``options.conf`` file. If this file does not exist it is created by - autospec with default values. If certain deprecated configuration - files exists autospec will use the value indicated by those files and - remove them. +``autospec`` is a tool to assist in the automated creation and maintenance of +RPM packaging in |CL-ATTR|. Where a standard RPM build process using ``rpmbuild`` +requires a tarball and .spec file to start, ``autospec`` requires only a tarball +and package name to start. How autospec works ****************** -Autospec attempts to infer the requirements of the ``spec`` file. If -autospec infers correctly, the control files (Table 1) will automatically -correct the build requirements. The control files are used to influence -the ``spec`` file generation. +``autospec`` attempts to infer the requirements of the .spec file by analyzing +the source code and :file:`Makefile` information. It will continuously run +updated builds based on new information discovered from build failures until it +has a complete and valid .spec file. Although not required, you can influence +the behavior of ``autospec`` by providing :ref:`control files `. -#. The :command:`make autospec` command generates a ``spec`` file from the - control files. +The basic process is described in the following steps: -#. ``autospec`` creates a ``build root`` with ``mock`` config. - -#. ``autospec`` attempts to build an RPM from the generated ``spec`` file. - -#. ``autospec`` detects any missed declarations in the ``spec`` file. +#. The :command:`make autospec` command generates a .spec based on + analysis of code and control files, if present. -.. note:: +#. ``autospec`` creates a ``build root`` with ``mock`` config. - * If there are missed declarations, ``autospec`` creates another ``mock`` - ``chroot`` and starts building again at Step 1. - * If a build error occurs, ``autospec`` stops for user inspection. - * If no build errors occur, RPM packages are successfully built. +#. ``autospec`` attempts to build an RPM from the generated .spec. -``autospec`` continues to rebuild the package, based on new information -discovered from build failures until it has a valid ``spec`` file. +#. ``autospec`` detects any missed declarations in the .spec. + +#. If build errors occur, ``autospec`` will scan the build log to try and detect + the root cause. + +#. If ``autospec`` detects the root cause and knows how to continue, it will restart the build + automatically at step 1 with updated build instructions. + +#. Otherwise, ``autospec`` will stop the build for user inspection and editing of control files + to resolve the errors. The user resumes the process at step 1 after errors are resolved. + +Following these steps, ``autospec`` continues to rebuild the package, based on +new information discovered from build failures, until it has a valid .spec. If +no build errors occur, RPM packages are successfully built. + +.. _control-files: + +Control files +************* + +It is possible to influence the behavior of ``autospec`` by providing control +files. These files may be used to alter the default behavior of the configure +routine, to blacklist build dependencies, etc. Control files must be located +in the same directory as the resulting .spec. + +Table 1 shows control files used to control dependencies, for example. + +.. list-table:: **Table 1. Control files to control dependencies** + :widths: 20 80 + :header-rows: 1 + + * - Filename + - Description + * - buildreq_add + - Each line in the file provides the name of a package to add as a + build dependency to the .spec. + * - buildreq_ban + - Each line in the file is a build dependency that under no + circumstance should be automatically added to the build dependencies. + This is useful to block automatic configuration routines adding + undesired functionality, or to omit any automatically discovered + dependencies during tarball scanning. + * - pkgconfig_add + - Each line in the file is assumed to be a pkgconfig() build + dependency. Add the pkg-config names here, as ``autospec`` will + automatically transform the names into their ``pkgconfig($name)`` + style when generating the .spec. + * - pkgconfig_ban + - Each line in this file is a pkgconfig() build dependency that should + not be added automatically to the build, much the same as + `` buildreq_ban``. As with ``pkgconfig_add``, these names are + automatically transformed by ``autospec`` into their correct + ``pkgconfig($name))`` style. + * - requires_add + - Each line in the file provides the name of a package to add as a + runtime dependency to the .spec. + * - requires_ban + - Each line in the file is a runtime dependency that under no + circumstance should be automatically added to the runtime + dependencies. This is useful to block automatic configuration + routines adding undesired functionality, or to omit any automatically + discovered dependencies during tarball scanning. + +Further control of the build can be achieved through the use of the +``options.conf`` file. If this file does not exist, it is created by +``autospec`` with default values. If certain deprecated configuration +files exists ``autospec`` will use the value indicated by those files and +remove them. + +For a comprehensive list of control files, view the `autospec readme`_. + +Related topics +************** + +* :ref:`autospec` +* :ref:`mixer` +* :ref:`mixin` .. _autospec readme: https://github.com/clearlinux/autospec diff --git a/source/clear-linux/concepts/concepts.rst b/source/clear-linux/concepts/concepts.rst index 3f78eda3..1a324c33 100644 --- a/source/clear-linux/concepts/concepts.rst +++ b/source/clear-linux/concepts/concepts.rst @@ -3,9 +3,8 @@ Concepts ######## -The concepts section provides content for a deeper understanding of the -features of |CLOSIA|. These concepts attempt to provide all the technical -details relevant to the |CL| features. +|CL-ATTR| does things differently than other Linux distributions. Use the concepts section to learn in detail about the features that make |CL| +different. .. toctree:: :maxdepth: 2 diff --git a/source/clear-linux/concepts/security.rst b/source/clear-linux/concepts/security.rst index 5b6b250f..27ec5235 100644 --- a/source/clear-linux/concepts/security.rst +++ b/source/clear-linux/concepts/security.rst @@ -1,6 +1,6 @@ .. _security: -|CL-ATTR| OS Security +OS Security ************************* |CL-ATTR| aims to make systemic and layered security-conscious decisions @@ -9,7 +9,7 @@ within the project's codebase and operating culture. .. contents:: :local: - :depth: 2 + :depth: 1 diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst index 47045d37..ca00f24d 100644 --- a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst @@ -20,7 +20,7 @@ Look for the :file:`clear-[version number]-installer.img.xz` file. You can also .. code-block:: bash - curl -O https://download.clearlinux.org/image/clear-$(curl https://download.clearlinux.org/latest)-installer.img.xz + curl -O https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep "installer") Once you have downloaded the image, verify and decompress the file. @@ -322,4 +322,4 @@ instructions. .. _`NUC6i5SYH product page`: http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html -.. _image: https://download.clearlinux.org/image \ No newline at end of file +.. _image: https://download.clearlinux.org/image diff --git a/source/clear-linux/get-started/bare-metal-manual-install/bare-metal-manual-install.rst b/source/clear-linux/get-started/bare-metal-manual-install/bare-metal-manual-install.rst index 512478fd..d5849e12 100644 --- a/source/clear-linux/get-started/bare-metal-manual-install/bare-metal-manual-install.rst +++ b/source/clear-linux/get-started/bare-metal-manual-install/bare-metal-manual-install.rst @@ -89,7 +89,7 @@ to install |CL| onto. Figure 4: :guilabel:`Device installation warning` -.. _Additional_manual_installer_settings: +.. _incl-additional-manual-installer-settings: Additional manual installer settings ==================================== @@ -183,6 +183,8 @@ For a complete description of the content of these additional bundles, go to the :ref:`software bundle list ` and select the name for a specific bundle to show the contents within the bundle. +.. _incl-additional-manual-installer-settings-end: + Target system network configuration =================================== diff --git a/source/clear-linux/get-started/cgdisk-manual-install/cgdisk-manual-install.rst b/source/clear-linux/get-started/cgdisk-manual-install/cgdisk-manual-install.rst index 653f83d0..5f00c804 100644 --- a/source/clear-linux/get-started/cgdisk-manual-install/cgdisk-manual-install.rst +++ b/source/clear-linux/get-started/cgdisk-manual-install/cgdisk-manual-install.rst @@ -265,21 +265,40 @@ partitions and select to format them. Figure 14: :guilabel:`Set mount point of sda3` - Upon completion, the :guilabel:`Set mount points` appear as shown - in figure 15: +#. Optional: Select :guilabel:`Encrypt Root Partition`, if + desired, as shown in figure 15. + .. note:: + `Set mount points` now show as completed. + .. figure:: figures/cgdisk-manual-install-15.png :scale: 50 % - :alt: Set mount point completed + :alt: Encrypt root partition - Figure 15: :guilabel:`Set mount points completed` + Figure 15: :guilabel:`Encrypt root partition` + +#. Type a confirmation passphrase as directed. + + .. note: + + It is recommended to record the passphrase for safekeeping. + + .. figure:: figures/cgdisk-manual-install-16.png + :scale: 50 % + :alt: Type confirmation passphrase + + Figure 16: :guilabel: `Type confirmation passphrase` #. Select the :guilabel:`< Next >` button and press :kbd:`Enter`. You have completed the process of manually partitioning your target - system. Now, :ref:`return to the bare metal manual installation + system. + +#. Now, :ref:`return to the bare metal manual installation ` to complete installation of |CL|. - Continue at the section *Additional manual installer settings*. + + Continue at the section :ref:`Additional manual installer settings + `. .. _`GPT fdisk tutorial`: http://www.rodsbooks.com/gdisk/ diff --git a/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-15.png b/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-15.png index cc885b6d..6b824a1f 100644 Binary files a/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-15.png and b/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-15.png differ diff --git a/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-16.png b/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-16.png index a5897712..77ff7d05 100644 Binary files a/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-16.png and b/source/clear-linux/get-started/cgdisk-manual-install/figures/cgdisk-manual-install-16.png differ diff --git a/source/clear-linux/get-started/compatibility-check.rst b/source/clear-linux/get-started/compatibility-check.rst index 798a05a4..76b795b1 100644 --- a/source/clear-linux/get-started/compatibility-check.rst +++ b/source/clear-linux/get-started/compatibility-check.rst @@ -1,7 +1,7 @@ .. _compatibility-check: -Check processor and EFI firmware compatibility with |CL-ATTR| -############################################################# +Check processor and EFI firmware compatibility +############################################## On a system that is currently running a Linux\* operating system, follow the instructions below to determine if your system's processor and EFI firmware is @@ -48,7 +48,7 @@ below. .. code-block:: console - Checking if host is capable of running Clear Liunx* OS for Intel® Architecture + Checking if host is capable of running Clear Linux* OS SUCCESS: 64-bit CPU (lm) SUCCESS: Supplemental Streaming SIMD Extensions 3 (ssse3) diff --git a/source/clear-linux/get-started/virtual-machine-install/kvm.rst b/source/clear-linux/get-started/virtual-machine-install/kvm.rst index ac192a9d..718bd559 100644 --- a/source/clear-linux/get-started/virtual-machine-install/kvm.rst +++ b/source/clear-linux/get-started/virtual-machine-install/kvm.rst @@ -55,7 +55,7 @@ Download and launch the virtual machine .. code-block:: bash - curl -O https://download.clearlinux.org/image/clear-$(curl https://download.clearlinux.org/latest)-kvm.img.xz + curl -O https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep '[0-9]'-kvm) #. Uncompress the downloaded image: diff --git a/source/clear-linux/get-started/virtual-machine-install/virtualbox.rst b/source/clear-linux/get-started/virtual-machine-install/virtualbox.rst index 3765aa8b..a526f7a8 100644 --- a/source/clear-linux/get-started/virtual-machine-install/virtualbox.rst +++ b/source/clear-linux/get-started/virtual-machine-install/virtualbox.rst @@ -38,7 +38,7 @@ Create a virtual machine in VirtualBox .. code-block:: bash - curl -O https://download.clearlinux.org/image/clear-$(curl https://download.clearlinux.org/latest)-live.img.xz + curl -O https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep live) #. Decompress the downloaded image. Uncompressed image size is ~ **5GB**. @@ -193,4 +193,4 @@ To enable Hyper-V again, you should execute:: .. _VirtualBox hypervisor: https://www.virtualbox.org/ .. _latest: https://download.clearlinux.org/image/ .. _7zip: http://www.7-zip.org/ -.. _Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html \ No newline at end of file +.. _Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html diff --git a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst index 1355506c..0af68bd7 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmw-player-preconf.rst @@ -64,7 +64,7 @@ this command: .. code-block:: bash - curl -O https://download.clearlinux.org/image/clear-$(curl https://download.clearlinux.org/latest)-vmware.vmdk.xz + curl -O https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep vmware) Visit :ref:`image-types` for additional information about all available |CL| images. diff --git a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst index a1d84fa3..eb1ee92d 100644 --- a/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst +++ b/source/clear-linux/get-started/virtual-machine-install/vmware-esxi-preconfigured-cl-image.rst @@ -36,7 +36,7 @@ this command: .. code-block:: bash - curl -O https://download.clearlinux.org/image/clear-$(curl https://download.clearlinux.org/latest)-vmware.vmdk.xz + curl -O https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep vmware) Visit :ref:`image-types` for additional information about all available |CL| images. diff --git a/source/clear-linux/guides/maintenance/autospec.rst b/source/clear-linux/guides/maintenance/autospec.rst index dd4dd2d7..aa25a6cf 100644 --- a/source/clear-linux/guides/maintenance/autospec.rst +++ b/source/clear-linux/guides/maintenance/autospec.rst @@ -3,45 +3,41 @@ Build RPMs with autospec ######################## -This guide shows you how to create RPMs with :ref:`autospec ` -, a tool that assists in automated creation and maintenance of RPM packaging -on |CLOSIA|. Additionally, you learn how to use these RPMs to create bundles -with :ref:`mixer `. +This guide shows you how to create RPMs with :ref:`autospec `, +a tool that assists in automated creation and maintenance of RPM packaging +on |CL-ATTR|. + +See our :ref:`autospec concept page ` for a detailed explaination +of how ``autospec`` works on |CL|. For a general understanding of how RPMs work, +we recommend visiting the `rpm website`_ or the `RPM Packaging Guide`_ . Prerequisites ************* -This guide assumes that you have: -* Created :ref:`a custom mix ` of |CL| and deployed it to a - to a target device +This guide requires that you: -* |CL| running on a host machine or virtual environment - - .. note:: +* Have installed |CL| on a host machine or virtual environment. For detailed + instructions on installing |CL|, visit the :ref:`get-started` section. - To install |CL|, see: +* :ref:`install-tooling` - * :ref:`bare-metal-install` - * :ref:`virtual-machine-install` - -Install Clear Linux tooling framework -===================================== +.. _install-tooling: -Our GitHub\* repository provides you with the resources you need -to create and maintain packages. +Install the |CL| tooling framework +================================== + +#. Install the `os-clr-on-clr` developer bundle on your host system. -#. On your host system, install this developer bundle. - .. code-block:: bash sudo swupd bundle-add os-clr-on-clr -#. Run this command to download the :file:`user-setup.sh` script. +#. Download the :file:`user-setup.sh` script. .. code-block:: bash curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh - + #. Make :file:`user-setup.sh` executable. .. code-block:: bash @@ -49,7 +45,7 @@ to create and maintain packages. chmod +x user-setup.sh #. Run the script as an unprivileged user. - + .. code-block:: bash ./user-setup.sh @@ -59,41 +55,31 @@ to create and maintain packages. The `user-setup script`_ creates a folder called :file:`clearlinux`, which contains the :file:`Makefile`, :file:`packages`, and :file:`projects` - subfolders. + subfolders. - The :file:`projects` folder contains the main tools, `autospec` - and `common`, used for making packages in |CL|. + The :file:`projects` folder contains the main tools, `autospec` + and `common`, used for making packages in |CL|. Create a RPM with autospec ************************** -.. include:: ../../concepts/autospec-about.rst - :start-after: incl-autospec-overview: - :end-before: incl-autospec-overview-end: - -For a detailed explanation of how ``autospec`` works on |CL|, visit our -:ref:`autospec-about` about page. For a general understanding of how RPMs -work, we recommend visiting the `rpm website`_ or the -`RPM Packaging Guide`_ . - -Building RPMs -============= - Choose one of the following options to build RPMs and manage source -code: +code: -* :ref:`build-a-new-rpm` and spec file using ``make autospecnew``. +* :ref:`build-a-new-rpm` and spec file using ``make autospecnew``. -* :ref:`build-source-code-with-existing-spec-file` (without changing the - spec file) using ``make build``. +* :ref:`build-source-code-with-existing-spec-file` using ``make build``, without changing the + spec file. -* :ref:`generate-a-new-spec-file` based on changes in the control files with - ``make autospec``. +* :ref:`generate-a-new-spec-file` using ``make autospec``, based on changes in the control files. .. _build-a-new-rpm: -Build a new RPM -=============== +Option 1: Build a new RPM +========================= + +Use this method to build a new RPM with no spec file. In this example, +we build a new helloclear RPM. #. Navigate to the autospec workspace. @@ -101,21 +87,21 @@ Build a new RPM cd ~/clearlinux -#. Enter the command: +#. Enter the command: .. code-block:: bash - make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" + make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" NAME="helloclear" - .. note:: + .. note:: - For a local tarball, use for the *URL*: + For a local tarball, use for the *URL*: file:// - + #. If build failures or dependency issues occur, continue below. - Otherwise, skip directly to `copy-rpm-packages-to-mixer`_. - + Otherwise, skip directly to `Next steps`_. + #. Navigate to the specific package. .. code-block:: bash @@ -123,52 +109,60 @@ Build a new RPM cd ~/clearlinux/packages/[package-name] #. Respond to the build process output by editing control files to resolve - issues, which may include dependencies or exclusions. + issues, which may include dependencies or exclusions. See `autospec readme`_ - #. Run this command: + #. Run this command: .. code-block:: bash make autospec - - Repeat the last two steps above until all errors are resolved and you - complete a successful build. - Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. + Repeat the last two steps above until all errors are resolved and you + complete a successful build. + +**Congratulations!** + +You've successfully created a RPM. + +Skip to `Next steps`_. .. _build-source-code-with-existing-spec-file: -Build source code with an existing spec file -============================================ +Option 2: Build source code with an existing spec file +====================================================== -If you only want to build the RPM using the spec file, use this method. This -method assumes that a spec file already exists. In this example, we run a -``make build`` on the ``dmidecode`` package. +Use this method if you only want to build the RPM using the spec file. This +method assumes that a spec file already exists. In this example, we run a +``make build`` on the ``dmidecode`` package. -#. Navigate to the ``dmidecode`` package in clearlinux: +#. Navigate to the ``dmidecode`` package in clearlinux: .. code-block:: bash - cd ~/clearlinux/packages/dmidecode/ + cd ~/clearlinux/packages/dmidecode/ -#. To download the tarball and build, run the command: +#. To download the tarball and build, run the command: - .. code-block:: bash + .. code-block:: bash make build - Skip to `copy-rpm-packages-to-mixer`_ to add the new RPM to your mix. +**Congratulations!** + +You've successfully created a RPM. + +Skip to `Next steps`_. .. _generate-a-new-spec-file: -Generate a new spec file with a pre-defined package -=================================================== +Option 3: Generate a new spec file with a pre-defined package +============================================================= -In this method, you will modify an existing |CL| package called ``dmidecode`` -to create a custom RPM. You will make a simple change to this package, -change the revision to a new number that is higher than the |CL| OS version, -and rebuild the package. +Use this method to modify an existing package. In this example, you will +modify an existing |CL| package called ``dmidecode`` to create a custom +RPM. You will make a simple change to this package, change the revision to +a new number that is higher than the |CL| OS version, and rebuild the package. #. Navigate to clearlinux: @@ -182,7 +176,7 @@ and rebuild the package. make clone_dmidecode -#. Navigate into the *dmidecode* directory: +#. Navigate into the *dmidecode* directory: .. code-block:: bash @@ -199,15 +193,15 @@ and rebuild the package. /usr/share/man/man8/ownership.8 /usr/share/man/man8/vpddecode.8 - .. note:: + .. note:: These files aren't needed by dmidecode, so we can remove them without any issues. -#. Save the file and exit. +#. Save the file and exit. -#. At :file:`~/clearlinux/packages/dmidecode`, build the modified - ``dmidecode`` package: +#. At :file:`~/clearlinux/packages/dmidecode`, build the modified + ``dmidecode`` package: .. code-block:: bash @@ -216,98 +210,32 @@ and rebuild the package. When the process completes, you will see new RPM packages in the :file:`results/` folder. -#. To view the new RPM packages, enter: +#. To view the new RPM packages, enter: .. code-block:: bash - ls /clearlinux/packages/dmidecode/results/ + ls /clearlinux/packages/dmidecode/results/ -Add a custom RPM to a mix and deploy to target -********************************************** - -We need a RPM repository to store our custom RPMs. This repository also -includes some metadata that allows programs such as ``yum`` and ``dnf`` to -follow and include any specified dependencies. This architecture enables us -to test custom RPMs before we integrate them in a mix. - -.. note:: - - Assure that you followed the :ref:`mixer` instruction and created - a location for **local RPM packages** using the *--local-rpms* flag - with the command: :command:`mixer init --local-rpms`. If you skipped - this step, return and complete it in :ref:`mixer` before proceeding. - -.. _copy-rpm-packages-to-mixer: - -Copy RPM packages to mixer and build bundle -============================================ - -Transfer the newly generated RPM packages to the ``mixer`` folder so -that it can include them as needed. - -.. note:: - - This guide assumes that you have a web server that hosts ``swupd`` update - content. - -#. Change directory into the mix workspace: - - .. code-block:: bash - - cd ~/mix - -#. Copy the contents from the results folder in the RPM packages to the - :file:`local-rpms` folder in the :file:`mix` folder: - - .. code-block:: bash - - cp ~/clearlinux/packages/dmidecode/results/*x86_64*rpm ~/mix/local-rpms/ - -#. Remove the debuginfo: - - .. code-block:: bash - - rm ~/mix/local-rpms/*debuginfo*x86_64* - -#. Generate the yum repo: - - .. code-block:: bash - - sudo mixer add-rpms - -#. Create a local bundle definition file to include the newly generated RPM - package in your mix. In our example, the ``[bundle-name]`` is - either ``dmidecode`` or ``helloclear``. - - .. code-block:: bash - - mixer bundle edit [bundle-name] - -#. Then add the new bundle to the mix. - - .. code-block:: bash - - mixer bundle add [bundle-name] - -#. Build the bundle and update content. - - .. code-block:: bash - - sudo mixer build all - -#. Log into the target device. - -#. On the target device, update and install the new bundle. - - .. code-block:: bash - - sudo swupd update - - sudo swupd bundle-add [bundle-name] - **Congratulations!** -You successfully built a RPM and created a mix with it. +You've successfully created a RPM. + +Next steps +********** + +Now you can create a custom bundle with your new RPM and use it with |CL|: + +* Use the :ref:`Mixer tool ` to add a new bundle to your derivative of |CL|. +* Use the :ref:`Mixin tool ` to customize your upstream |CL| installation with a new bundle. + +Related topics +************** + +* :ref:`Mixer tool ` +* :ref:`Mixin tool ` +* :ref:`autospec ` +* :ref:`Bundles ` + .. _rpm website: http://rpm.org diff --git a/source/clear-linux/guides/maintenance/bulk-provision.rst b/source/clear-linux/guides/maintenance/bulk-provision.rst index 71e6e1d1..6b52e760 100644 --- a/source/clear-linux/guides/maintenance/bulk-provision.rst +++ b/source/clear-linux/guides/maintenance/bulk-provision.rst @@ -143,13 +143,13 @@ Configuration # MAC address,role default,ciao -#. Verify the following URLs are accessible: +#. Verify the following URLs are accessible on your local network: - * http://192.168.1.1:60000/icis/static/ister/ister.conf - * http://192.168.1.1:60000/icis/static/ister/ister.json - * http://192.168.1.1:60000/icis/get_config/ - * http://192.168.1.1:60000/icis/get_role/ - * http://192.168.1.1:60000/ipxe/ipxe_boot_script.txt + * ``http://192.168.1.1:60000/icis/static/ister/ister.conf`` + * ``http://192.168.1.1:60000/icis/static/ister/ister.json`` + * ``http://192.168.1.1:60000/icis/get_config/`` + * ``http://192.168.1.1:60000/icis/get_role/`` + * ``http://192.168.1.1:60000/ipxe/ipxe_boot_script.txt`` #. Power on the PXE client and watch it boot and install |CL|. diff --git a/source/clear-linux/guides/maintenance/enable-user-space.rst b/source/clear-linux/guides/maintenance/enable-user-space.rst index 5dab93b7..e48e272b 100644 --- a/source/clear-linux/guides/maintenance/enable-user-space.rst +++ b/source/clear-linux/guides/maintenance/enable-user-space.rst @@ -41,19 +41,16 @@ and to perform system updates called software update utility or :command:`swupd`. Software applications are installed as bundles using the sub-command :command:`bundle-add`. -The `os-clr-on-clr` bundle installs the vast majority of -applications useful to a system administrator or a developer. The bundle -contains other bundles such as `sysadmin-basic`, `editors`, `c-basic`, -`dev-utils-dev`, and other useful packages. - -Install the `os-clr-on-clr` bundle: +The `sysadmin-basic` bundle installs the vast majority of +applications useful to a system administrator. +Install the `sysadmin-basic` bundle: .. code-block:: bash - swupd bundle-add os-clr-on-clr + swupd bundle-add sysadmin-basic We provide the full list of bundles and packages installed with the -`os-clr-on-clr`_ bundle. Additionally, we have listed +`sysadmin-basic`_ bundle. Additionally, we have listed `all Clear Linux bundles`_, active or deprecated. Click any bundle on the list to view the manifest of the bundle. @@ -81,8 +78,8 @@ To be able to execute all applications with root privileges, add the #. Enter the new `` and the password created earlier. You will now be in the home directory of ``. The bundle - `os-clr-on-clr`_ contains the majority of applications that a developer or - system administrator would want, but it does not include a graphical user + `sysadmin-basic`_ contains the majority of applications that a system + administrator would want, but it does not include a graphical user interface. The `desktop` bundle includes the GNOME\* Display Manager and additional supporting applications. @@ -136,8 +133,8 @@ With your system now running |CL|, many opportunities exist. Visit the :ref:`tutorials ` page for examples on using your |CL| system. -.. _`os-clr-on-clr`: - https://github.com/clearlinux/clr-bundles/blob/master/bundles/os-clr-on-clr +.. _`sysadmin-basic`: + https://github.com/clearlinux/clr-bundles/blob/master/bundles/sysadmin-basic .. _`all Clear Linux bundles`: https://github.com/clearlinux/clr-bundles/tree/master/bundles diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 9c6e9f66..f826ab4f 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -40,7 +40,7 @@ Second, if your mix includes your own .. contents:: :local: - :depth: 2 + :depth: 1 :backlinks: top The following sections contain detailed information on every step of diff --git a/source/clear-linux/reference/bundles/bundles.rst b/source/clear-linux/reference/bundles/bundles.rst index e063c3c5..26c3acb9 100644 --- a/source/clear-linux/reference/bundles/bundles.rst +++ b/source/clear-linux/reference/bundles/bundles.rst @@ -6,16 +6,6 @@ Available bundles This document provides a current list of `available bundles`_ as of ``12 September 2018``. -See in depth descriptions of the following bundles: - -.. toctree:: - :maxdepth: 1 - - containers-basic - openssh-server - os-core - kvm-host - Bundle list =========== diff --git a/source/clear-linux/reference/bundles/containers-basic.rst b/source/clear-linux/reference/bundles/containers-basic.rst deleted file mode 100644 index e9e0bc1d..00000000 --- a/source/clear-linux/reference/bundles/containers-basic.rst +++ /dev/null @@ -1,41 +0,0 @@ -.. _containers-basic: - -containers-basic -################ - -The `containers-basic` bundle adds the necessary tools to enable running -containers using Docker*. The bundle includes Intel® |CC| as an -additional Docker runtime. - -|CC| enable a hardware backed :abbr:`VM (Virtual Machine)` based -container runtime, compared with the normal software namespace containers -provided by standard Docker `runc` runtime. - -Default runtime -=============== -If your system has `VT-x` enabled, then, under |CL|, |CC| -will be used as the default Docker runtime, otherwise, the standard Docker -`runc` runtime will be used. - -To identify which runtimes are available and which is being used as -the default on your installed system, run the following command. - -.. code-block:: console - - sudo docker info | grep Runtime - -The |CC| runtime will be listed as `cor`. - -For more information on |CC| please see the -`Clear Containers runtime GitHub`_. - -Working with a proxy -==================== - -If you are behind an HTTP proxy server, in a corporate -setting for example, please follow the `Docker proxy instructions`_. - -.. _Clear Containers runtime GitHub: https://github.com/01org/cc-oci-runtime - -.. _Docker proxy instructions: - https://docs.docker.com/engine/admin/systemd/#http-proxy diff --git a/source/clear-linux/reference/bundles/kvm-host.rst b/source/clear-linux/reference/bundles/kvm-host.rst deleted file mode 100644 index 1c580b83..00000000 --- a/source/clear-linux/reference/bundles/kvm-host.rst +++ /dev/null @@ -1,42 +0,0 @@ -.. _bdl-kvm-host: - -kvm-host -######## - -This bundle provides the software needed to run virtual machines. - -QEMU -==== - -|CL| provides :abbr:`QEMU (Quick Emulator)` to perform hardware virtualization. -When possible, qemu would be used together with -:abbr:`KVM (Kernel-based Virtual Machine)` in order to run virtual machines at -near-native speed. - -For more information about how to use qemu, please refer to `qemu.org`_ - -virsh -===== - -|CL| provides the virsh program to manage `libvirt`_ guests and the hypervisor. -libvirtd management daemon is not enabled nor started by default. To enable -the hypervisor, perform the following commands as root. - -#. Enable the service to start every time the machine boots. - This step is optional - - .. code-block:: console - - # systemctl enable libvirtd.service - -#. Start the libvirtd management daemon service - - .. code-block:: console - - # systemctl start libvirtd.service - -Congratulations! The libvirtd management daemon is running and virsh is ready -to be used. - -.. _qemu.org: http://www.qemu.org/ -.. _libvirt: http://libvirt.org/ diff --git a/source/clear-linux/reference/bundles/openssh-server.rst b/source/clear-linux/reference/bundles/openssh-server.rst deleted file mode 100644 index 99e6d80d..00000000 --- a/source/clear-linux/reference/bundles/openssh-server.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. _bdl-openssh-server: - -openssh-server -############## - -This bundle provides the OpenSSH\* package needed to enable a SSH service. -Remote users require a SSH service to be able to use an encrypted login -shell. The first time OpenSSH starts, it generates the server SSH keys needed -for the service. - -SFTP -==== - -|CL| *disables* the :abbr:`SFTP (SSH File Transfer Protocol)` subsystem by -default due to security considerations. To enable the SFTP subsystem, perform -the following configuration of the :abbr:`SSHD (SSH Daemon)` service file: - -#. Create a systemd drop-in directory for the SSHD service: - - .. code-block:: console - - # mkdir -p /etc/systemd/system/sshd@.service.d - -#. Create the following file: - :file:`/etc/systemd/system/sshd@.service.d/sftp.conf` - -#. Add the OPTIONS environment variable - - .. code-block:: console - - [Service] - Environment="OPTIONS=-o Subsystem=\"sftp /usr/libexec/sftp-server\"" - -#. Reload systemd configuration: - - .. code-block:: console - - # systemctl daemon-reload - -Congratulations! The SFTP subsystem is enabled. - -Root login -========== - -To enable root login via ssh, perform the following steps: - -#. Create a *ssh* directory in :file:`/etc`, only if it does not exist) - - .. code-block:: console - - # mkdir /etc/ssh - -#. Set the configuration variable. - - .. code-block:: console - - # echo "PermitRootLogin yes" >> /etc/ssh/sshd_config diff --git a/source/clear-linux/reference/bundles/os-core.rst b/source/clear-linux/reference/bundles/os-core.rst deleted file mode 100644 index d27808e1..00000000 --- a/source/clear-linux/reference/bundles/os-core.rst +++ /dev/null @@ -1,106 +0,0 @@ -.. _os-core: - -os-core -####### - -This bundle contains the basic core components of the operating system. -|CLOSIA| relies on `systemd` to provide the basic OS components. The -`systemd` package contains solutions for: - -* Service management -* Basic network management -* Hostname management -* Time synchronization management -* Boot control management -* Journal management - -Additionally, `os-core` includes the `util-linux` and `coreutils` packages to -provide a set of basic Linux\* command line tools such as `ls`, `cp`, `rm`, -etc. - - -Static IP -========= - -To configure a static IP, follow these steps: - -#. Create the file: :file:`/etc/systemd/network/50-static.network` If the - path does not exist, create it. - -#. Add, at the very least, the following lines to the - :file:`50-static.network` file: - - .. code-block:: console - - [Match] - Name= - - [Network] - Address= - Gateway= - - The `` is your network device name, for example: enp1s0 or - enp0s25. - -#. To add more options you can consult the `systemd network configuration`_ - manual. - -#. Restart the *systemd-networkd* service with the following command: - - - .. code-block:: console - - # systemctl restart systemd-networkd - - Alternatively, restart |CL|. - -#. Check the new IP with the following command: - - .. code-block:: console - - # ip addr - - -Setting time -============ - -Clear Linux uses the `systemd-timesyncd.service` service to synchronize the -system's time. Default :abbr:`NTP (Network Time Protocol)` servers are -configured, for example: `time1.google.com`, `time2.google.com`, -`time3.google.com`, and `time4.google.com`. Manually setting the time via -`timedatectl` or using RTC mode is not possible. If those servers cannot be -reached and the system time is incorrect, follow these steps: - -#. Set the timezone, for example, Pacific time: - - .. code-block:: console - - timedatectl set-timezone America/Los_Angeles - - To see a list of timezones, use the following command: - - .. code-block:: console - - timedatectl list-timezones - -#. Go to the :file:`/etc/systemd/` directory, if it does not exist, create - it. - -#. Create the :file:`/etc/systemd/timesyncd.conf` file and enter the - following lines: - - .. code-block:: console - - [Time] - NTP= - FallbackNTP= - -#. Restart the timesync daemon with the following command: - - .. code-block:: console - - # systemctl restart systemd-timesyncd - -.. _systemd network configuration: - https://www.freedesktop.org/software/systemd/man/systemd.network.html diff --git a/source/clear-linux/reference/collaboration/documentation/rest.rst b/source/clear-linux/reference/collaboration/documentation/rest.rst index 43c71097..aaf7016f 100644 --- a/source/clear-linux/reference/collaboration/documentation/rest.rst +++ b/source/clear-linux/reference/collaboration/documentation/rest.rst @@ -14,7 +14,7 @@ documentation. This section contains the preferred methods for using the `Sphinx documentation`_ for the complete list of available markup and use as much markup as possible. -.. _Sphinx documentation: http://www.sphinx-doc.org/en/stable/markup/index.html +.. _Sphinx documentation: http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html Remember: **Changing incorrect markup is easier than adding markup from scratch.** diff --git a/source/clear-linux/tutorials/docker/docker.rst b/source/clear-linux/tutorials/docker/docker.rst index 91d1fa1d..e9643777 100644 --- a/source/clear-linux/tutorials/docker/docker.rst +++ b/source/clear-linux/tutorials/docker/docker.rst @@ -99,13 +99,20 @@ exist by default. .. code-block:: bash - touch /etc/docker/daemon.json + sudo touch /etc/docker/daemon.json .. note:: Refer to the `Docker documentation on daemon configuration`_ for the full list of available configuration options and examples. + A minimal configuration would be. + .. code-block:: json + + { + "storage-driver": "devicemapper" + } + #. Once you've made any required changes, be sure to restart the Docker daemon through systemd manager by running this command: diff --git a/source/clear-linux/tutorials/greengrass.rst b/source/clear-linux/tutorials/greengrass.rst new file mode 100644 index 00000000..315f0804 --- /dev/null +++ b/source/clear-linux/tutorials/greengrass.rst @@ -0,0 +1,406 @@ +.. _greengrass: + +Enable AWS Greengrass* and OpenVINO™ on |CL-ATTR| +################################################# + +Hardware accelerated Function-as-a-Service (FaaS) enables cloud developers +to deploy inference functionalities [1] on Intel® IoT edge devices with +accelerators (Integrated GPU, Intel® FPGA, and Intel® Movidius™). These +functions provide a great developer experience and seamless migration of +visual analytics from cloud to edge in a secure manner using a containerized +environment. Hardware-accelerated FaaS provides the best-in-class +performance by accessing optimized deep learning libraries on Intel® IoT +edge devices with accelerators. + +This tutorial shows how to: + +* Set up the Intel® edge device with |CL-ATTR| +* Install the OpenVINO™ and AWS Greengrass* software stacks +* Use AWS Greengrass and lambdas to deploy the FaaS samples from the cloud + +Supported Platforms +******************* + +* Operating System: |CL-ATTR| latest release +* Hardware: Intel® core platforms (Tutorial supports inference on CPU only) + +Description of Samples +********************** + +The AWS Greengrass samples are located at the `Edge-Analytics-FaaS`_. + +We provide the following AWS Greengrass samples: + +* :file:`greengrass_classification_sample.py` + + This AWS Greengrass sample classifies a video stream using classification + networks such as AlexNet and GoogLeNet and publishes top-10 results on AWS* + IoT Cloud every second. + +* :file:`greengrass_object_detection_sample_ssd.py` + + This AWS Greengrass sample detects objects in a video stream and + classifies them using single-shot multi-box detection (SSD) networks such + as SSD Squeezenet, SSD Mobilenet, and SSD300. This sample publishes + detection outputs such as class label, class confidence, and bounding box + coordinates on AWS IoT Cloud every second. + +Converting Deep Learning Models +******************************* + +Sample Models +============= + +For classification, `download the BVLC Alexnet model`_ as an example. +Any custom pre-trained classification models can be used with the +classification sample. + +For object detection, the sample models optimized for Intel® edge platforms +are present at :file:`/usr/share/openvino/models`. These models are provided +as an example; however, any custom pre-trained SSD models can be used with +the object detection sample. + +Running Model Optimizer +======================= + +Follow these instructions for `converting deep learning models to Intermediate Representation using Model Optimizer`_. For example, use the +following commands. + +For classification using BVLC Alexnet model: + +.. code-block:: bash + + python3 mo.py --framework caffe --input_model < + model_location>/bvlc_alexnet.caffemodel --input_proto < + model_location>/deploy.prototxt --data_type --output_dir < + output_dir> --input_shape [1,3,227,227] + +For object detection using SqueezeNetSSD-5Class model: + +.. code-block:: bash + + python3 mo.py --framework caffe --input_model + SqueezeNetSSD-5Class.caffemodel --input_proto + SqueezeNetSSD-5Class.prototxt + --data_type --output_dir + +In these examples: + +* ```` is :file:`/usr/share/openvino/models` + +* ```` is FP32 or FP16, depending on target device. + +* ```` is the directory where the user wants to store the + Intermediate Representation (IR). IR contains .xml format corresponding + to the network structure and .bin format corresponding to weights. This + .xml file should be passed to . + +* In the BVLC Alexnet model, the prototxt defines the input shape with + batch size 10 by default. In order to use any other batch size, the + entire input shape needs to be provided as an argument to the model + optimizer. For example, to use batch size 1, you can provide + “--input_shape [1,3,227,227]”. + +Installing |CL| on the edge device +********************************** + +Start with a clean installation of |CL| on a new system, using the +:ref:`bare-metal-install`, found in :ref:`get-started`. + +Create user accounts +==================== + +After |CL| is installed, create two user accounts. Create an administrative +user in |CL|. You will also create a user account for the Greengrass +services to use (see Greengrass user below). + +#. Create a new user and set a password for that user. Enter the following + commands as ``root``: + + .. code-block:: bash + + useradd + passwd + +#. Next, enable the :command:`sudo` command for your new ````. Add + ```` to the ``wheel`` group: + + .. code-block:: bash + + usermod -G wheel -a + +#. Create the user and group account for the Greengrass daemon: + + .. code-block:: console + + useradd ggc_user + groupadd ggc_group + +#. Create a :file:`/etc/fstab` file. + + .. code-block:: bash + + touch /etc/fstab + + .. note:: + + By default |CL| does not create an :file:`/etc/fstab` file. + The Greengrass service needs to have the file created before + it will run. + +Add required bundles +==================== + +Use the ``swupd`` software updater utility to add the following bundles to +enable the OpenVINO software stack: + +.. code-block:: bash + + swupd bundle-add os-clr-on-clear desktop-autostart computer-vision-basic + +.. note:: + + Learn more about how to :ref:`swupd-guide`. + +The ``computer-vision-basic`` bundle will install the OpenVINO software, +along with the edge device models needed. + +Configuring an AWS Greengrass group +=================================== + +For each Intel® edge platform, we need to create a new AWS Greengrass group +and install AWS Greengrass core software to establish the connection between +cloud and edge. + +#. To create an AWS Greengrass group, follow the + `AWS Greengrass developer guide`_ + +#. To install and configure AWS Greengrass core on edge platform, follow + the instructions at `Start AWS Greengrass on the Core Device`_. + + .. note:: + + You will not need to run the ``cgroupfs-mount.sh`` script in step #6 + of Module 1 of the `AWS Greengrass developer guide`_ because this is + enabled already in |CL|. + +Creating and Packaging Lambda Functions +======================================= + +#. Complete the tutorial at `Configure AWS Greengrass on AWS IoT`_ . + + .. note:: + + This creates the tarball needed to create the AWS Greengrass + environment on the edge device. + +#. Assure to download both the security resources and the AWS Greengrass + core software. + + .. note:: + + Security certificates are linked to your AWS* account. + +#. Replace greengrassHelloWorld.py with Greengrass samples: + + * greengrass_classification_sample.py + + * greengrass_object_detection_sample_ssd.py + +#. Zip these files with extracted Greengrass SDK folders from the previous + step into :file:`greengrass_sample_python_lambda.zip`. + + The zip should contain: + + * greengrasssdk + + * greengrass sample + + For the sample, choose one of these: + + - greengrass_classification_sample.py + + - greengrass_object_detection_sample_ssd.py + + For example: + + .. code-block:: bash + + zip -r greengrass_lambda.zip greengrasssdk + greengrass_object_detection_sample_ssd.py + +#. Follow steps 6-11 to `complete creating lambdas`_. + + .. note:: + + In the AWS documentation, step 9(a), while uploading the zip file, + make sure to name the handler as below depending on the AWS Greengrass + sample you are using: + + * greengrass_object_detection_sample_ssd.function_handler (or) + * greengrass_classification_sample.function_handler + +Deploying Lambdas +================= + +Configuring the Lambda function +------------------------------- + +After creating the Greengrass group and the lambda function, start +configuring the lambda function for AWS Greengrass. + +#. Follow steps 1-8 in `Configure the Lambda Function`_ of the AWS + documentation. + +#. In addition to the details mentioned in step 8, change the Memory limit + to 2048MB to accommodate large input video streams. + +#. Add the following environment variables as key-value pairs when editing + the lambda configuration and click on update: + + .. list-table:: **Table 1. Environment Variables: Lambda Configuration** + :widths: 20 80 + :header-rows: 1 + + * - Key + - Value + * - PARAM_MODEL_XML + - /, where is user specified and + contains IR.xml, the Intermediate Representation file from Intel® Model Optimizer + * - PARAM_INPUT_SOURCE + - /input.webm to be specified by user. Holds both input and + output data. For webcam, set PARAM_INPUT_SOURCE to ‘/dev/video0’ + * - PARAM_DEVICE + - For CPU, specify "CPU" + * - PARAM_CPU_EXTENSION_PATH + - /usr/lib64/libcpu_extension.so + * - PARAM_OUTPUT_DIRECTORY + - to be specified by user. Holds both input and output + data + * - PARAM_NUM_TOP_RESULTS + - User specified for classification sample. + (e.g. 1 for top-1 result, 5 for top-5 results) + +#. Add subscription to subscribe, or publish messages from AWS Greengrass + lambda function by following the steps 10-14 in `Configure the Lambda Function`_ + + .. note:: + + The “Optional topic filter” field should be the topic + mentioned inside the lambda function. + + For example, openvino/ssd or openvino/classification + +Local Resources +--------------- +#. Select `this link to add local resources and access privileges`_. + + Following are the local resources needed for the CPU: + + .. list-table:: **Local Resources** + :widths: 20, 20, 20, 20 + :header-rows: 1 + + * - Name + - Resource type + - Local path + - Access + + * - ModelDir + - Volume + - to be specified by user + - Read-Only + + * - Webcam + - Device + - /dev/video0 + - Read-Only + + * - DataDir + - Volume + - to be specified by user. Holds both input and output + data. + - Read and Write + +Deploy +------ + +To `deploy the lambda function to AWS Greengrass core device`_, select +“Deployments” on group page and follow the instructions. + +Output Consumption +------------------ + +There are four options available for output consumption. These options are +used to report, stream, upload, or store inference output at an interval +defined by the variable ``reporting_interval`` in the AWS Greengrass samples. + +a. IoT Cloud Output: + This option is enabled by default in the AWS Greengrass samples using a + variable ``enable_iot_cloud_output``. We can use it to verify the lambda + running on the edge device. It enables publishing messages to IoT cloud + using the subscription topic specified in the lambda (For example, + ‘openvino/classification’ for classification and ‘openvino/ssd’ for + object detection samples). For classification, top-1 result with class + label are published to IoT cloud. For SSD object detection, detection + results such as bounding box co-ordinates of objects, class label, and + class confidence are published. + + Follow the instructions here to `view the output on IoT cloud`_ + +b. Kinesis Streaming: + + This option enables inference output to be streamed from the edge device + to cloud using Kinesis [3] streams when ‘enable_kinesis_output’ is set + to True. The edge devices act as data producers and continually push + processed data to the cloud. The users need to set up and specify + Kinesis stream name, Kinesis shard, and AWS region in the AWS Greengrass + samples. + +c. Cloud Storage using AWS S3 Bucket: + + When the ‘enable_s3_jpeg_output’ variable is set to True, it enables uploading and storing processed frames (in JPEG format) in an AWS S3 bucket. The users need to set up and specify the S3 bucket name in the + AWS Greengrass samples to store the JPEG images. The images are named using the timestamp and uploaded to S3. + +d. Local Storage: + + When the ‘enable_s3_jpeg_output’ variable is set to True, it enables storing processed frames (in JPEG format) on the edge device. The + images are named using the timestamp and stored in a directory specified + by ‘PARAM_OUTPUT_DIRECTORY’. + +References +----------- + +1. AWS Greengrass: https://aws.amazon.com/greengrass/ +2. AWS Lambda: https://aws.amazon.com/lambda/ +3. AWS Kinesis: https://aws.amazon.com/kinesis/ + +.. _Edge-Analytics-FaaS: https://github.com/intel/Edge-Analytics-FaaS/tree/master/AWS%20Greengrass + +.. _download the BVLC Alexnet model: https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet + +.. _converting deep learning models to Intermediate Representation using Model Optimizer: https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer + +.. _AWS Greengrass developer guide: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html + +.. _Start AWS Greengrass on the Core Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-device-start.html + +.. _AWS Greengrass Core SDK: https://docs.aws.amazon.com/greengrass/latest/developerguide/create-lambda.html + +.. _complete creating lambdas: https://docs.aws.amazon.com/greengrass/latest/developerguide/create-lambda.html + +.. _Configure the Lambda Function: https://docs.aws.amazon.com/greengrass/latest/developerguide/config-lambda.html + +.. _Add local resources and access privileges: https://docs.aws.amazon.com/greengrass/latest/developerguide/access-local-resources.html + +.. _deploy the lambda function to AWS Greengrass core device: https://docs.aws.amazon.com/greengrass/latest/developerguide/configs-core.html + +.. _Edge-optmized models repository: https://github.com/intel/Edge-optimized-models + +.. _view the output on IoT cloud: https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-check.html + +.. _this link to add local resources and access privileges: https://docs.aws.amazon.com/greengrass/latest/developerguide/access-local-resources.html + +.. _Configure AWS Greengrass on AWS IoT: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html + + diff --git a/source/clear-linux/tutorials/kubernetes.rst b/source/clear-linux/tutorials/kubernetes.rst new file mode 100644 index 00000000..b5761750 --- /dev/null +++ b/source/clear-linux/tutorials/kubernetes.rst @@ -0,0 +1,414 @@ +.. _kubernetes: + +Run Kubernetes\* on |CL-ATTR| +############################# + +This tutorial describes how to install, configure, and run the +`Kubernetes container orchestration system`_ on |CL-ATTR| using different +container engines and runtimes. + +Kubernetes\* is an open source system for automating deployment, scaling, and +management of containerized applications. It groups containers that make up +an application into logical units for easy management and discovery. + +Runc and Kata Containers\* kata-runtime adhere to :abbr:`OCI (Open Container Initiative*)` +guidelines and work seamlessly with Kubernetes. `Kata Containers`_ provide +strong isolation for untrusted workloads or multi-tenant scenarios. Runc and +Kata Containers can be allocated on a per-pod basis so you can mix and match +both on the same host to suit your needs. + +This tutorial describes the following combinations: + +* Kubernetes with Docker and runc +* Kubernetes with CRI-O and kata-runtime + +Prerequisites +************* + +This tutorial assumes you have installed |CL| and updated to the latest +release on your host system. You can learn about the benefits of having an +up-to-date system for cloud orchestration on the :ref:`swupd-about` +page. For detailed instructions on installing |CL| on a bare metal system, +follow the :ref:`bare metal installation tutorial`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Install Kubernetes and CRI runtimes +*********************************** + +Kubernetes and a set of supported :abbr:`CRI (Container Runtime Interface)` +runtimes are included in the `cloud-native-basic`_ bundle. To install the +framework, enter the following command: + +.. code-block:: bash + + sudo swupd bundle-add cloud-native-basic + +Configure Kubernetes +******************** + +This tutorial uses the basic default Kubernetes configuration for simplicity. +You must define your Kubernetes configuration according to your specific +deployment and your security needs. + +#. Enable IP forwarding to avoid kubeadm `preflight check`_ errors: + + Create (or edit if it exists) the file :file:`/etc/sysctl.d/60-k8s.conf` + and include the following line: + + .. code-block:: bash + + net.ipv4.ip_forward = 1 + + Apply the change: + + .. code-block:: bash + + sudo systemctl restart systemd-sysctl + +#. Enable the kubelet service: + + .. code-block:: bash + + sudo systemctl enable kubelet.service + +#. Disable swap using one of the following methods, either: + + a) Temporarily: + + .. code-block:: bash + + sudo swapoff -a + + .. note:: + + Swap will be enabled at next reboot, causing failures in + your cluster. + + or: + + b) Permanently: + + Mask the swap partition: + + .. code-block:: bash + + sudo systemctl mask $(sed -n -e 's#^/dev/\([0-9a-z]*\).*#dev-\1.swap#p' /proc/swaps) 2>/dev/null + sudo swapoff -a + + .. note:: + + On systems with limited resources, some performance degradation may + be observed while swap is disabled. + +#. Switch to root to modify `hostname`: + + .. code-block:: bash + + sudo -s + +#. Create (or edit if it exists) the hosts file that Kubernetes will read to + locate the master's host: + + .. code-block:: bash + + echo "127.0.0.1 localhost `hostname`" >> /etc/hosts + +#. Exit root: + + .. code-block:: bash + + exit + +Configure and run Kubernetes +**************************** + +This section describes how to configure and run Kubernetes with: + +* Docker and runc +* CRI-O and kata-runtime + +Configure and run Docker + runc +=============================== + +#. Enable the Docker service: + + .. code-block:: bash + + sudo systemctl enable docker.service + +#. Create (or edit if it exists) the file + :file:`/etc/systemd/system/docker.service.d/51-runtime.conf` and include the following lines: + + .. code-block:: bash + + [Service] + Environment="DOCKER_DEFAULT_RUNTIME=--default-runtime runc" + +#. Create (or edit if it exists) the file :file:`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and include the following lines: + + .. code-block:: bash + + [Service] + Environment="KUBELET_EXTRA_ARGS=" + +#. Enter the commands: + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl restart docker + sudo systemctl restart kubelet + +#. Initialize the master control plane with the command: + + .. code-block:: bash + + sudo kubeadm init --ignore-preflight-errors=SystemVerification + + +Configure and run CRI-O + kata-runtime +====================================== + +#. Enable the CRI-O service: + + .. code-block:: bash + + sudo systemctl enable crio.service + +#. Enter the commands: + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl restart crio + sudo systemctl restart kubelet + +#. Initialize the master control plane with the command: + + .. code-block:: bash + + sudo kubeadm init --cri-socket=/run/crio/crio.sock + + +Install pod network add-on +************************** + +You must choose and install a `pod network add-on`_ to allow your pods to +communicate. Check whether or not your add-on requires special flags when you +initialize the master control plane. + +**Notes about flannel add-on** + +If you choose the `flannel` add-on, then you must add the following to the +`kubeadm init` command: + +.. code-block:: bash + + --pod-network-cidr 10.244.0.0/16 + +If you are using CRI-O and `flannel` and you want to use Kata Containers, edit +the :file:`/etc/crio/crio.conf` file to add: + +.. code-block:: bash + + [crio.runtime] + manage_network_ns_lifecycle = true + +**Notes about Weave Net add-on** + +If you choose the `Weave Net` add-on, then you must make the following changes +because it installs itself in the :file:`/opt/cni/bin` directory. + +If you are using Docker and `Weave Net`, edit the :file:`kubeadm.conf` file to +add: + +.. code-block:: bash + + Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin" + +If you are using CRI-O and `Weave Net`, you must complete the following steps. + +#. Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from: + + .. code-block:: bash + + plugin_dir = "/usr/libexec/cni/" + + to: + + .. code-block:: bash + + plugin_dir = "/opt/cni/bin" + +#. Add the `loopback` CNI plugin to the plugin path with the command: + + .. code-block:: bash + + sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback + + +Use your cluster +**************** + +Once your master control plane is successfully initialized, instructions on +how to use your cluster and its *IP*, *token*, and *hash* values are +displayed. It is important that you record the cluster values because they are +needed when joining worker nodes to the cluster. Some values have a valid +period. The values are presented in a format similar to: + +.. code-block:: bash + + kubeadm join : --token --discovery-token-ca-cert-hash + + +**Congratulations!** + +You've successfully installed and set up Kubernetes in |CL| using Docker and +runc or CRI-O and kata-runtime. You are now ready to follow on-screen +instructions to deploy a pod network to the cluster and join worker nodes +with the displayed token and IP information. + +Related topics +************** + +Read the Kubernetes documentation to learn more about: + +* `Understanding basic Kubernetes architecture`_ + +* `Deploying an application to your cluster`_ + +* Installing a `pod network add-on`_ + +* `Joining your nodes`_ + +Package configuration customization in |CL| (optional) +****************************************************** + +|CL| is a stateless system that looks for user-defined package configuration +files in the :file:`/etc/` directory to be used as default. If +user-defined files are not found, |CL| uses the distribution-provided +configuration files for each package. + +If you customize any of the default package configuration files, you **must** +store the customized files in the :file:`/etc/` directory. If you edit any of +the distribution-provided default files, your changes will be lost in the +next system update. + +For example, to customize CRI-O configuration in your system, run the +following commands: + +.. code-block:: bash + + sudo mkdir /etc/crio + sudo cp /usr/share/defaults/crio/crio.conf /etc/crio/ + sudo $EDITOR /etc/crio/crio.conf + +Learn more about `Stateless in Clear Linux`_ and view the `Clear Linux documentation`_. + +Proxy configuration (optional) +****************************** + +If you use a proxy server, you must set your proxy environment variables and +create an appropriate proxy configuration file for both CRI-O and Docker +services. Consult your IT department if you are behind a corporate proxy for +the appropriate values. Ensure that your local IP is **explicitly included** +in the environment variable *NO_PROXY*. (Setting *localhost* is not enough.) + +If you have already set your proxy environment variables, run the following +commands as a shell script to configure all of these services in one step: + +.. code-block:: bash + + services=('crio' 'docker') + for s in "${services[@]}"; do + sudo mkdir -p "/etc/systemd/system/${s}.service.d/" + cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf" + [Service] + Environment="HTTP_PROXY=${http_proxy}" + Environment="HTTPS_PROXY=${https_proxy}" + Environment="SOCKS_PROXY=${socks_proxy}" + Environment="NO_PROXY=${no_proxy}" + EOF + done + +Troubleshooting +*************** + +* not found in message. + + Your DNS server may not be appropriately configured. Try adding an + entry to the :file:`/etc/hosts` file with your host's IP and Name. + + For example: 100.200.50.20 myhost + + Use the commands :command:`hostname` and :command:`hostname -I` to retrieve them. + +* Images cannot be pulled. + + You may be behind a proxy server. Try configuring your proxy settings, + using the environment variables *HTTP_PROXY*, *HTTPS_PROXY*, and *NO_PROXY* + as required in your environment. + +* Connection refused error. + + If you are behind a proxy server, you may need to add the master's IP to + the environment variable *NO_PROXY*. + +* Connection timed-out or Access Refused errors. + + You must ensure that the appropriate proxy settings are available from the + same terminal where you will initialize the control plane. To verify the + proxy settings that Kubernetes will actually use, run the commands: + + .. code-block:: bash + + echo $HTTP_PROXY + echo $HTTPS_PROXY + echo $NO_PROXY + + If the displayed proxy values are different from your assigned values, the + cluster initialization will fail. Contact your IT support team to learn how + to set the proxy variables permanently, and how to make them available for + all the types of access that you will use, such as remote SSH access. + +* Missing environment variables. + + If you are behind a proxy server, pass environment variables by adding *-E* + to the command that initializes the master control plane. + + .. code-block:: bash + + /* Kubernetes with Docker + runc */ + sudo -E kubeadm init --ignore-preflight-errors=SystemVerification + + /* Kubernetes with CRI-O + kata-runtime */ + sudo -E kubeadm init --cri-socket=/run/crio/crio.sock + + +.. _Kubernetes container orchestration system: https://kubernetes.io/ + +.. _Kata Containers: https://katacontainers.io/ + +.. _Software Update documentation: https://clearlinux.org/documentation/clear-linux/concepts/swupd-about#updating + +.. _cloud-native-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/cloud-native-basic + +.. _preflight check: https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/#preflight-checks + +.. _Understanding basic Kubernetes architecture: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3 + +.. _Deploying an application to your cluster: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2 + +.. _pod network add-on: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network + +.. _Joining your nodes: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes + +.. _Stateless in Clear Linux: https://clearlinux.org/features/stateless + +.. _Clear Linux documentation: https://clearlinux.org/documentation/clear-linux + diff --git a/source/clear-linux/tutorials/machine-learning/machine-learning.rst b/source/clear-linux/tutorials/machine-learning/machine-learning.rst index a1aff29e..e16a25f0 100644 --- a/source/clear-linux/tutorials/machine-learning/machine-learning.rst +++ b/source/clear-linux/tutorials/machine-learning/machine-learning.rst @@ -1,22 +1,29 @@ .. _machine-learning: -Machine learning tutorial -######################### +TensorFlow\* machine learning on |CL-ATTR| +########################################## -This tutorial guides you through installing and using a Jupyter\* notebook to -set up and execute a TensorFlow\* machine learning example using the MNIST -data for handwriting recognition using the |CLOSIA|. The initial steps will -have you set up a Jupyter kernel and run a notebook on a bare-metal |CL| -system. +This tutorial will demonstrate the installation and execusion of a TensorFlow\* +machine learning example on |CL-ATTR|. It uses a Jupyter\* Notebook and MNIST +data for handwriting recognition.  -Install and configure a Clear Linux Host OS on bare metal -========================================================= +The initial steps will have you set up a Jupyter kernel and run a Notebook +on a bare-metal |CL| system. -First, follow our instructions to install -:ref:`Clear Linux on bare metal`. +Prerequisites +************* -Once the bare metal installation and initial configuration are complete, add -the following two bundles to your system: +This tutorial assumes you have installed |CL| on your host system. For detailed +instructions on installing |CL| on a bare metal system, follow the +:ref:`bare metal installation tutorial`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Once your system is updated, add the following bundles to your system: * `machine-learning-web-ui`: This bundle contains the Jupyter application. @@ -26,14 +33,14 @@ the following two bundles to your system: To install the bundles, run the following commands in your :file:`$HOME` directory: -.. code-block:: console +.. code-block:: bash sudo swupd bundle-add machine-learning-web-ui sudo swupd bundle-add machine-learning-basic Set up a Jupyter notebook -========================= +************************* With all required packages and libraries installed, set up the file structure for the Jupyter Notebook. @@ -41,19 +48,19 @@ for the Jupyter Notebook. #. In the :file:`$HOME` directory, create a directory for the Jupyter Notebooks named :file:`Notebooks`. - .. code-block:: console + .. code-block:: bash mkdir Notebooks #. Within :file:`Notebooks`, create a directory named :file:`Handwriting`. - .. code-block:: console + .. code-block:: bash mkdir Notebooks/Handwriting #. Change to the new directory. - .. code-block:: console + .. code-block:: bash cd Notebooks/Handwriting @@ -82,7 +89,7 @@ The files needed are: * `t10k-labels-idx1-ubyte.gz`_: Test set labels (4542 bytes) Run the Jupyter machine learning example code -============================================= +********************************************* With |CL|, Jupyter, and TensorFlow installed and configured, we can run the example code. @@ -90,7 +97,7 @@ run the example code. #. Go to the :file:`($HOME)/Notebooks` directory and start Jupyter with the following commands: - .. code-block:: console + .. code-block:: bash cd ~/Notebooks @@ -184,18 +191,18 @@ run the example code. Figure 8: The system's accuracy for the entire data set. -For more in-depth and detailed information on the model used and the -mathematics it entails, visit the TensorFlow tutorials +For more in-depth information on the model used and the mathematics it entails, +visit the TensorFlow tutorials `TensorFlow MNIST beginners demo`_ and `TensorFlow MNIST pros demo`_. **Congratulations!** -You have successfully installed a Jupyter kernel on |CL|. Furthermore, you +You have successfully installed a Jupyter kernel on |CL|. In addition, you trained a neural network to successfully predict the values contained in a data set of hand-written number images. -Additional resources -==================== +Related topics +************** * `MNIST Database website`_ * `TensorFlow MNIST beginners demo`_ diff --git a/source/clear-linux/tutorials/telemetry-backend/telemetry-backend.rst b/source/clear-linux/tutorials/telemetry-backend/telemetry-backend.rst index 4b026f4e..131471c3 100644 --- a/source/clear-linux/tutorials/telemetry-backend/telemetry-backend.rst +++ b/source/clear-linux/tutorials/telemetry-backend/telemetry-backend.rst @@ -5,8 +5,8 @@ Create a telemetry backend server in Clear Linux |CLOSIA| includes a telemetry and analytics solution, also known as telemetrics, as part of the OS that records events of interest and reports -them back to the development team using the :command:`telemd` telemetrics -daemon. +them back to the development team using the telemetrics daemons that are +running on the |CL| client system. End users can enable or disable the telemetry client component of |CL| and also redirect where records go if they wish to collect records for themselves diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index c312c8ad..e5d51254 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -3,20 +3,19 @@ Tutorials ######### -Our Tutorials: +This section offers a range of tutorials that demonstrate: -* Demonstrate how to use |CL| features for cloud, client, distributed - processing, and virtual environments -* Involve specific application of |CL| that often use third-party tools -* Extend the development possibilities of |CL| with specific use cases +* |CL-ATTR| features for cloud, client, distributed processing, + and virtual environments +* Use of |CL| with third-party tools +* Development possibilities of |CL| with specific use cases -This section provides detailed instructions to guide you through completing -specific |CLOSIA| use cases. +Explore our tutorials to discover what you can do with |CL|! .. toctree:: - :maxdepth: 2 + :maxdepth: 1 - wordpress + wordpress/wordpress flatpak/flatpak machine-learning/machine-learning docker/docker @@ -30,4 +29,5 @@ specific |CLOSIA| use cases. spark kata kata_migration - + kubernetes + greengrass diff --git a/source/clear-linux/tutorials/wordpress.rst b/source/clear-linux/tutorials/wordpress.rst deleted file mode 100644 index f66bb1f6..00000000 --- a/source/clear-linux/tutorials/wordpress.rst +++ /dev/null @@ -1,27 +0,0 @@ -.. _wordpress: - -WordPress\* use case tutorial -############################# - -Welcome to our WordPress use case tutorial. The objective of the tutorial is -to guide you to a fully realized WordPress website using |CLOSIA|. - -In this tutorial we assume your system is already running |CL|. For -instructions on how to install |CL| on bare metal, see the -:ref:`bare metal install` page from our -:ref:`get-started` guide. For your convenience, we have -split this tutorial in two different sections. If you already have a |CL| -based :abbr:`LAMP (Linux, Apache, MySQL, PHP)` web server you can skip ahead -to the WordPress tutorial. - -.. toctree:: - :maxdepth: 1 - - web-server-install/web-server-install.rst - wp-install/wp-install.rst - -.. note:: - - This tutorial is for development and testing purposes only. Additional - steps are required to secure production systems. - Those steps are beyond the scope of this tutorial. diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-1.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-1.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-1.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-1.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-2.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-2.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-2.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-2.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-3.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-3.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-3.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-3.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-4.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-4.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-4.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-4.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-5.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-5.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-5.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-5.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-6.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-6.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-6.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-6.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-7.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-7.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-7.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-7.png diff --git a/source/clear-linux/tutorials/web-server-install/figures/web-server-install-8.png b/source/clear-linux/tutorials/wordpress/figures/web-server-install-8.png similarity index 100% rename from source/clear-linux/tutorials/web-server-install/figures/web-server-install-8.png rename to source/clear-linux/tutorials/wordpress/figures/web-server-install-8.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-1.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-1.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-1.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-1.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-2.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-2.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-2.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-2.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-3.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-3.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-3.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-3.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-4.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-4.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-4.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-4.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-5.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-5.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-5.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-5.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-6.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-6.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-6.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-6.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-7.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-7.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-7.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-7.png diff --git a/source/clear-linux/tutorials/wp-install/figures/wp-install-8.png b/source/clear-linux/tutorials/wordpress/figures/wp-install-8.png similarity index 100% rename from source/clear-linux/tutorials/wp-install/figures/wp-install-8.png rename to source/clear-linux/tutorials/wordpress/figures/wp-install-8.png diff --git a/source/clear-linux/tutorials/web-server-install/web-server-install.rst b/source/clear-linux/tutorials/wordpress/web-server-install.rst similarity index 75% rename from source/clear-linux/tutorials/web-server-install/web-server-install.rst rename to source/clear-linux/tutorials/wordpress/web-server-install.rst index 24793b97..b4c5fb93 100644 --- a/source/clear-linux/tutorials/web-server-install/web-server-install.rst +++ b/source/clear-linux/tutorials/wordpress/web-server-install.rst @@ -1,35 +1,24 @@ .. _web-server-install: -Create a Clear Linux\* based web server -####################################### +Set up a LAMP web server on |CL-ATTR| +##################################### -This tutorial shows you how to create a LAMP server using |CL-ATTR| and how to use phpMyAdmin\* to manage an associated database. +This tutorial provides instructions on how to set up a +:abbr:`LAMP (Linux, Apache, MySQL, PHP)` web server on |CL-ATTR| and how +to use phpMyAdmin\* to manage an associated database. Note that this +tutorial installs MariaDB\*, which is a drop-in replacement for MySQL\*. In order to create a web server using |CL| as the host OS, your host system must be running |CL|. This tutorial assumes you have successfully installed :ref:`Clear Linux on bare metal`. -Before you install new packages, update the |CL| OS with the following console command: +This tutorial covers: -.. code-block:: bash - - sudo swupd update - -Create a LAMP Server -******************** - -A LAMP server uses Linux\*, Apache\*, MySQL\*, and PHP\* to set up a fully functional web server and host a website. Note that this tutorial installs MariaDB, which is a drop-in replacement for MySQL. - -This tutorial follows these steps: - -* Install Apache. -* Change the default configuration and data directory. -* Install PHP. -* Install MariaDB. -* Install phpMyAdmin and create a database. +.. contents:: :local: + :depth: 1 Install Apache -***************** +************** Apache is an open source HTTP web server application that can run on several operating systems, including |CL|. Go to the `Apache HTTP Server Project`_ @@ -38,9 +27,18 @@ for more information. Install the web-server-basic bundle =================================== -The web-server-basic bundle contains the packages needed to install the +The **web-server-basic** bundle contains the packages needed to install the Apache software bundle on |CL|. +.. note:: + + Before you install new packages, update the |CL| with the following + console command: + + .. code-block:: bash + + sudo swupd update + #. To install the bundle, enter the following command: .. code-block:: bash @@ -55,11 +53,11 @@ Apache software bundle on |CL|. sudo systemctl enable httpd.service sudo systemctl start httpd.service - #. To verify that the Apache server application is running, open a web browser and navigate to: http://localhost. - If the service is running, a confirmation message appears, as shown in figure 1. + If the service is running, a confirmation message will appear, similar to the + message shown in figure 1. .. figure:: figures/web-server-install-1.png :alt: This web server is operational from host. @@ -69,20 +67,22 @@ Apache software bundle on |CL|. .. note:: - The :file:`index.html` file is located in the :file:`/var/www/html` directory of your host system. You will copy this file into a new location after you modify the configuration in the next step. + The :file:`index.html` file is located in the :file:`/var/www/html` directory + of your host system. You will copy this file into a new location after you + modify the configuration in the next step. Change the default configuration and data directory *************************************************** -|CL| is designed to be a stateless operating system which means that you -must create an optional configuration file to make changes over the default -values. The default location of the Apache configuration file, -:file:`httpd.conf`, is located in the :file:`/usr/share/defaults/httpd` -directory. |CL| can overwrite this directory as part of the stateless -paradigm. This default :file:`.conf` file includes the following directives -that allow for additional locations of configuration definitions: +|CL| is designed to be a `stateless`_ operating system which means that you +must create an optional configuration file to override the default values. +The default location of the Apache configuration file, :file:`httpd.conf`, +is located in the :file:`/usr/share/defaults/httpd` directory. |CL| can +override this directory as part of the stateless paradigm. This default +:file:`.conf` file includes the following directives that allow for additional +locations of configuration definitions: -.. code-block:: bash +.. code-block:: console # Virtual hosts IncludeOptional /usr/share/defaults/httpd/conf.d/*.conf @@ -90,21 +90,27 @@ that allow for additional locations of configuration definitions: IncludeOptional /etc/httpd/conf.d/*.conf IncludeOptional /etc/httpd/conf.modules.d/*.conf -This tutorial follows these steps: +In this section you will define your own httpd.conf file to override the +default values, and define a custom DocumentRoot for your web server. -* Create the directory structure for :file:`/etc/httpd/conf.d`. - -* Create the :file:`httpd.conf` file in directory :file:`/etc/httpd/conf.d`. - -* Add the ``DocumentRoot`` variable to :file:`httpd.conf`. - -Open a text editor and perform the following: - -#. Copy the content listed below into the new file - :file:`/etc/httpd/conf.d/httpd.conf`. +#. Create the directory structure for :file:`/etc/httpd/conf.d`. .. code-block:: bash + sudo mkdir -p /etc/httpd/conf.d + +#. Create and open the :file:`httpd.conf` file in your new :file:`/etc/httpd/conf.d` + directory. + + .. code-block:: bash + + sudo nano /etc/httpd/conf.d/httpd.conf + +#. Add the ``DocumentRoot`` variable to :file:`httpd.conf`. Copy the content + listed below into the new :file:`/etc/httpd/conf.d/httpd.conf` file. + + .. code-block:: console + # # Set a new location for DocumentRoot # @@ -118,7 +124,6 @@ Open a text editor and perform the following: Require all granted - #. Create a new ``DocumentRoot`` directory structure and copy the :file:`index.html` file from :file:`/var/www/html` directory to :file:`/var/www/tutorial`. @@ -129,16 +134,21 @@ Open a text editor and perform the following: cd /var/www/tutorial sudo cp /var/www/html/index.html . +#. To ensure a successful setup, edit the new :file:`index.html` file with an + obvious change. -#. To ensure a successful setup, edit the new :file:`index.html` file. - Change the original text from + .. code-block:: bash - "This web server is operational from host." + sudo nano index.html - to + For example, we changed the default message - "This web server is operational from its new location." + "It works!" + to + + "It works from its new location!" + #. Stop and then restart ``httpd.service``. .. code-block:: bash @@ -146,12 +156,17 @@ Open a text editor and perform the following: sudo systemctl stop httpd.service sudo systemctl start httpd.service -#. Go to http://localhost to view the new screen. +#. Go to http://localhost to view the new screen. You should see your updated + default message from step 5. #. Change the configuration back to the default :file:`/var/www/html` location. To do this, edit the :file:`/etc/httpd/conf.d/httpd.conf` file again and replace any instance of /var/www/tutorial with /var/www/html. + .. code-block:: bash + + sudo nano /etc/httpd/conf.d/httpd.conf + #. Stop and then restart ``httpd.service``. .. code-block:: bash @@ -162,7 +177,7 @@ Open a text editor and perform the following: #. Go to http://localhost and verify that you can see the default screen again. - Optionally, remove the /var/www/tutorial directory you previously created. +#. Optionally, remove the /var/www/tutorial directory you previously created. .. code-block:: bash @@ -192,8 +207,12 @@ functionality to your web server, install PHP on your system. After restarting the Apache service, test your PHP installation. -#. Create a file named :file:`phpinfo.php` in the - :file:`/var/www/html/` directory using a text editor. +#. Create and open a file named :file:`phpinfo.php` in the :file:`/var/www/html/` + directory using a text editor. + + .. code-block:: bash + + sudo nano /var/www/html/phpinfo.php #. Add the following line to the file: @@ -203,7 +222,7 @@ functionality to your web server, install PHP on your system. #. Go to http://localhost/phpinfo.php. -#. Verify that the PHP information screen appears, as shown in figure 2: +#. Verify that the PHP information screen appears, similar to figure 2: .. figure:: figures/web-server-install-2.png :alt: PHP information screen @@ -216,7 +235,7 @@ the PHP components and are now ready to add your database application to complete your LAMP server implementation. Install MariaDB -****************** +*************** Install MariaDB to store content. MariaDB is a drop-in replacement for MySQL and is available in the database-basic |CL| bundle. @@ -240,7 +259,9 @@ and is available in the database-basic |CL| bundle. sudo systemctl status mariadb -Security Hardening + Press :kbd:`Ctrl` + :kbd:`c` or :kbd:`q` to exit. + +Security hardening ================== With the MariaDB service running, we can perform some basic security @@ -250,7 +271,7 @@ hardening. .. code-block:: bash - mysql_secure_installation + sudo mysql_secure_installation #. Respond to the questions that appear in the script below. @@ -307,14 +328,18 @@ hardening. ... Success! Disallow root login remotely? [Y/n] - Normally, root should only be allowed to connect from the 'localhost'. This ensures that someone cannot guess the root password from the network. To block any remote root login, type 'y'. + Normally, root should only be allowed to connect from the 'localhost'. This + ensures that someone cannot guess the root password from the network. To + block any remote root login, type 'y'. .. code-block:: bash ... Success! Remove test database and access to it? [Y/n] - By default, MariaDB includes a database named 'test' which anyone can access. This database is also intended only for testing and should be removed. To remove the test database, type 'y'. + By default, MariaDB includes a database named 'test' which anyone can access. + This database is also intended only for testing and should be removed. To + remove the test database, type 'y'. .. code-block:: bash @@ -349,8 +374,7 @@ MariaDB databases. Visit the `phpMyAdmin`_ website for the complete discussion regarding phpMyAdmin, its documentation, the latest downloads, and other useful information. -This tutorial uses the latest English version of phpMyAdmin to install it on -our |CL| host system. +In this tutorial, we use the latest English version of phpMyAdmin. #. Download the :file:`phpMyAdmin--english.tar.gz` file to your :file:`~/Downloads` directory. Here, refers to the current @@ -377,10 +401,11 @@ our |CL| host system. sudo mv phpMyAdmin-4.6.4-english phpMyAdmin -Use phpMyAdmin to Manage Databases -==================================== +Use phpMyAdmin to manage a database +*********************************** -You can use the phpMyAdmin web-based tool to manage your databases. Follow the steps below for setting up a database called Wordpress. +You can use the phpMyAdmin web-based tool to manage your databases. Follow the +steps below for setting up a database called "WordPress". #. Verify that a successful installation of all LAMP server components by going to http://localhost/phpMyAdmin. See figure 3. @@ -395,7 +420,6 @@ You can use the phpMyAdmin web-based tool to manage your databases. Follow the s `Figure 3: The phpMyAdmin login page.` - #. Verify a successful login by confirming that the main phpMyAdmin page displays, as shown in figure 4: @@ -405,7 +429,6 @@ You can use the phpMyAdmin web-based tool to manage your databases. Follow the s `Figure 4: The phpMyAdmin dashboard.` - #. Set up a database by selecting the :guilabel:`Databases` tab, as shown in figure 5. @@ -423,7 +446,6 @@ You can use the phpMyAdmin web-based tool to manage your databases. Follow the s `Figure 5: The Databases tab.` - #. Set up user permissions by selecting the :guilabel:`WordPress` database located in the left panel. See figure 6. @@ -470,10 +492,13 @@ If successful, you should see the screen shown in figure 8: **Congratulations!** You have now created a fully functional LAMP server along with a -WordPress-ready database using |CL|. +WordPress\*-ready database using |CL|. -As a next step, you could :ref:`create a WordPress server ` and -present it to the world. +Next steps +********** + +Next, add the WordPress components needed to host a WordPress website with :ref:`wp-install`. .. _Apache HTTP Server Project: https://httpd.apache.org/ -.. _phpMyAdmin: https://www.phpmyadmin.net/ \ No newline at end of file +.. _phpMyAdmin: https://www.phpmyadmin.net/ +.. _stateless: https://clearlinux.org/features/stateless \ No newline at end of file diff --git a/source/clear-linux/tutorials/wordpress/wordpress.rst b/source/clear-linux/tutorials/wordpress/wordpress.rst new file mode 100644 index 00000000..31060894 --- /dev/null +++ b/source/clear-linux/tutorials/wordpress/wordpress.rst @@ -0,0 +1,27 @@ +.. _wordpress: + +WordPress\* on |CL-ATTR| +######################## + +This tutorial describes how to setup a web server to host a WordPress\* site on a +|CL-ATTR| system. The tutorial is split into two sections: + +#. :ref:`web-server-install` + +#. :ref:`wp-install` + +If you already have a |CL| based :abbr:`LAMP (Linux, Apache, MySQL, PHP)` web server, +you can skip ahead to the second section of this tutorial. + +.. note:: + + This tutorial is for development and testing purposes only. Additional steps + are required to secure production systems. Those steps are beyond the scope + of this tutorial. + +.. toctree:: + :maxdepth: 1 + :hidden: + + web-server-install.rst + wp-install.rst diff --git a/source/clear-linux/tutorials/wp-install/wp-install.rst b/source/clear-linux/tutorials/wordpress/wp-install.rst similarity index 52% rename from source/clear-linux/tutorials/wp-install/wp-install.rst rename to source/clear-linux/tutorials/wordpress/wp-install.rst index b5eda738..aa062e3e 100644 --- a/source/clear-linux/tutorials/wp-install/wp-install.rst +++ b/source/clear-linux/tutorials/wordpress/wp-install.rst @@ -1,31 +1,37 @@ .. _wp-install: -Create a Clear Linux WordPress server +Set up WordPress\* on a LAMP web server ####################################### -This tutorial shows you how to install the WordPress components on your |CL| LAMP server. At the end of :ref:`web-server-install`, you created a WordPress\*-ready database using phpMyAdmin\* and MariaDB\*. Now that the LAMP server is up and running, you can add the WordPress components needed to host a WordPress website on your system. +This tutorial shows you how to install the WordPress\* components on your |CL| +:abbr:`LAMP (Linux, Apache, MySQL, PHP)` server. At the end of +:ref:`web-server-install`, you created a WordPress-ready database using +phpMyAdmin\* and MariaDB\*. Now that the LAMP server is up and running, you +can add the WordPress components needed to host a WordPress website on your system. Before you begin -================= - -This tutorial assumes you have successfully installed :ref:`Clear Linux on bare metal` and that you have :ref:`set up a LAMP web server `. - - -Before you install new packages, update the |CL| OS with the following command: - -.. code-block:: bash - - sudo swupd update +**************** +This tutorial assumes you have successfully completed :ref:`bare-metal-install` +and that you have :ref:`web-server-install`. Create a WordPress server -=========================== +************************* -WordPress can be installed in a variety of ways. These instructions are written for users who have followed our instructions for installing phpMyAdmin when they :ref:`set up a LAMP web server `. Note that all steps in this tutorial have been tested using a NUC6i5SYH Intel® NUC. Visit the `NUC6i5SYH product page`_ for detailed information. +WordPress can be installed in a variety of ways. These instructions are +written for users who have followed our instructions for installing phpMyAdmin +when they :ref:`set up a LAMP web server `. Note that +all steps in this tutorial have been tested using a NUC6i5SYH Intel® NUC. +Visit the `NUC6i5SYH product page`_ for detailed information. -Numerous online articles are available to help you name your website and acquire the necessary certificates. Those tasks are beyond the scope of this tutorial. +Numerous online articles are available to help you name your website and +acquire the necessary certificates. Those tasks are beyond the scope of this tutorial. -You can take several actions to harden your website from attacks. The security of your website and the data it contains are complex and ever-evolving tasks. Prioritize security if you plan to expose your website to the outside world. This tutorial does not address security measures that you can take to harden your site but we strongly encourage you to take action. +You can take several actions to harden your website from attacks. The security +of your website and the data it contains are complex and ever-evolving tasks. +Prioritize security if you plan to expose your website to the outside world. +This tutorial does not address security measures that you can take to harden +your site but we strongly encourage you to take action. .. note:: @@ -33,11 +39,13 @@ You can take several actions to harden your website from attacks. The security o Download WordPress and manage directories ------------------------------------------- +========================================= -For this tutorial, you will create a WordPress blog that can be accessed at: http:///blog. +For this tutorial, you will create a WordPress blog that can be accessed at: +http:///blog. -To accomplish this setup, you must add WordPress components to the :file:`/var/www/html/blog` directory. +To accomplish this setup, you must add WordPress components to the :file:`/var/www/html/blog` +directory. Follow these steps: @@ -72,74 +80,76 @@ Follow these steps: sudo rm latest.tar.gz -Set up WordPress with Web-based GUI ------------------------------------ +Set up WordPress with web-based GUI +=================================== -Recall that you created a database and user when you installed phpMyAdmin during :ref:`Creating a Clear Linux based web server `. Next, you must connect WordPress to the database and install WordPress. +Recall that you created a database and user when you installed phpMyAdmin during +:ref:`Creating a Clear Linux based web server `. Next, you must +connect WordPress to the database and install WordPress. -To continue with the setup, go to: http:///blog/wp-admin/install.php. The WordPress language option screen appears, as shown in figure 1. +To continue with the setup, go to: http:///blog/wp-admin/install.php. +The WordPress language option screen appears, as shown in figure 1. #. Select :guilabel:`English` and click :guilabel:`Continue`. .. figure:: figures/wp-install-1.png - :alt: WordPress language selection - :width: 600 + :alt: WordPress language selection + :width: 600 - `Figure 1: WordPress language selection screen.` + `Figure 1: WordPress language selection screen.` - The WordPress installation continues until the Welcome screen appears, as shown in figure 2: + The WordPress installation continues until the Welcome screen appears, as shown in + figure 2: .. figure:: figures/wp-install-2.png - :alt: WordPress welcome screen - :width: 600 + :alt: WordPress welcome screen + :width: 600 - `Figure 2: WordPress Welcome screen.` + `Figure 2: WordPress Welcome screen.` #. Click :guilabel:`Let’s go!`. -#. Enter database connection specifics in the screen that appears as shown in Figure 3 below. +#. Enter database connection specifics in the screen that appears, as shown in figure 3 + below. + * Database name: WordPress - * Database username: wordpressuser - * Database password: wp-example (asterisks will not appear in the text box) - * Database host: localhost - * Table prefix: wp\_ - Figure 3 shows the completed fields: - .. figure:: figures/wp-install-3.png - :alt: Database connection details - :width: 600 + :alt: Database connection details + :width: 600 - `Figure 3: Information necessary for WordPress to connect to the database.` + `Figure 3: Information necessary for WordPress to connect to the database.` #. Click :guilabel:`Submit` to complete the setup. - Figure 4 shows the confirmation screen that verifies a successful setup. WordPress is connected to the MariaDB database. + Figure 4 shows the confirmation screen that verifies a successful setup. WordPress + is connected to the MariaDB database. .. figure:: figures/wp-install-4.png - :alt: Successful database connection. - :width: 600 + :alt: Successful database connection. + :width: 600 - `Figure 4: Successful WordPress connection.` + `Figure 4: Successful WordPress connection.` #. Click :guilabel:`Run the install`. The installer runs until WordPress is fully installed on your system. Complete successful login ---------------------------- +========================= -Once the installation is complete, you can name your blog and create a Wordpress username and password. See figure 5. +Once the installation is complete, you can name your blog and create a WordPress username +and password. See figure 5. .. figure:: figures/wp-install-5.png - :alt: WordPress user creation - :width: 600 + :alt: WordPress user creation + :width: 600 - `Figure 5: WordPress site information screen.` + `Figure 5: WordPress site information screen.` #. Enter all required information. @@ -147,10 +157,10 @@ Once the installation is complete, you can name your blog and create a Wordpress #. Verify that the initial login screen appears once the installation is complete. See figure 6: .. figure:: figures/wp-install-6.png - :alt: WordPress login - :width: 600 + :alt: WordPress login + :width: 600 - `Figure 6: The WordPress login screen.` + `Figure 6: The WordPress login screen.` #. Enter your WordPress username and password. #. Check :guilabel:`Remember me` to save your credentials. @@ -159,10 +169,10 @@ Once the installation is complete, you can name your blog and create a Wordpress Figure 7 shows the WordPress dashboard after a successful login: .. figure:: figures/wp-install-7.png - :alt: WordPress Dashboard - :width: 600 + :alt: WordPress Dashboard + :width: 600 - `Figure 7: The WordPress dashboard.` + `Figure 7: The WordPress dashboard.` You are ready to go! @@ -170,16 +180,16 @@ To check out your blog as it is seen by the outside world, enter: http:///blog on your browser. Figure 8 shows the result: .. figure:: figures/wp-install-8.png - :alt: WordPress blog - :width: 600 + :alt: WordPress blog + :width: 600 - `Figure 8: Your WordPress blog.` + `Figure 8: Your WordPress blog.` **Congratulations, your WordPress blog is up and running!** You have successfully installed WordPress on a host system. -Add new entries to your blog and share them with the world using |CLOSIA|! +Add new entries to your blog and share them with the world using |CL|! .. _`step-by-step guide`: https://codex.wordpress.org/Installing_WordPress#Famous_5-Minute_Install