Remove escaping from installer config password hashes

Password hashes in the installer configuration yaml are written as-is to `/etc/shadow`. The backslashes preceding `$` are unnecessary and produce an invalid hash string that will prevent login.

.github/workflows/check.yml

@@ -13,11 +13,12 @@ jobs:
     - name: Install dependencies
       run: |
         export PATH="$HOME/.local/bin:$PATH"
-        sudo apt-get install -y python3-setuptools
+        sudo apt-get install -y python3-setuptools pandoc
         pip3 install --user -r requirements.txt
     - name: Build the docs
       run: |
         export PATH="$HOME/.local/bin:$PATH"
         make py
+        make man
         make htmlall
         

.github/workflows/linkcheck.yml

@@ -0,0 +1,22 @@
+name: Linkcheck
+
+on:
+  schedule:
+    - cron: '0 0 * * Mon,Wed,Fri'
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Install dependencies
+      run: |
+        export PATH="$HOME/.local/bin:$PATH"
+        sudo apt-get install -y python3-setuptools
+        pip3 install --user -r requirements.txt
+    - name: CheckLinks
+      run: |
+        export PATH="$HOME/.local/bin:$PATH"
+        make linkcheck

.github/workflows/main.yml

@@ -15,12 +15,13 @@ jobs:
     - name: Install dependencies
       run: |
         export PATH="$HOME/.local/bin:$PATH"
-        sudo apt-get install -y python3-setuptools
+        sudo apt-get install -y python3-setuptools pandoc
         pip3 install --user -r requirements.txt
     - name: Build the docs
       run: |
         export PATH="$HOME/.local/bin:$PATH"
         make py
+        make man
         make htmlall
     - name: Pre-deploy
       run: |

.github/workflows/publish.yml

@@ -15,12 +15,13 @@ jobs:
     - name: Install dependencies
       run: |
         export PATH="$HOME/.local/bin:$PATH"
-        sudo apt-get install -y python3-setuptools
+        sudo apt-get install -y python3-setuptools pandoc
         pip3 install --user -r requirements.txt
     - name: Build the docs
       run: |
         export PATH="$HOME/.local/bin:$PATH"
         make py
+        make man
         make htmlall
     - name: Pre-deploy
       run: |

.gitignore

@@ -17,3 +17,13 @@ bundles.html.txt
 
 # ignore the venv, used for running make py
 venv
+
+#ignore for reDocs GUI
+.tox
+MANIFEST
+
+# ignore artifacts of man page generation
+source/_scripts/_python/manpages/*.rst
+source/reference/manpages
+source/_scripts/_python/manpages/*/
+source/reference/man-pages.rst

Makefile

@@ -25,6 +25,12 @@ linkcheck:
 py:
 	make -C source py
 
+man:
+	make -C source man
+
+clean-man:
+	make -C source clean-man
+
 help:
 	@echo "Please use \`make <target>' where <target> is one of"
 	@echo "  html       to make standalone HTML files"

README.rst

@@ -63,8 +63,13 @@ running ``make html``:
 Open one of the HTML pages found in ``source/_build/html`` in a web browser
 to view the rendered documentation.
 
-If you want to build the documentation exactly as seen on the website, use
-``make py`` followed by ``make htmlall``. This builds some
+This build will generate several warnings as there are two other optional make commands required to build the full documentation.
+
+1. ``make py`` to generate the bundle reference material.
+2. ``make man`` to generate man page reference material.
+
+To build the documentation exactly as seen on the website, use
+``make man``, ``make py``, and ``make htmlall``. This builds both
 external dependencies and all supported languages.
 
 Use virtualenv 
@@ -163,6 +168,9 @@ build before building again by running ``make clean``:
 This will completely remove the previous build output, including artifacts 
 from the `make venv` target when done outside an active venv.
 
+Before running ``make man``, please run ``make clean-man`` to clear out any
+previous attempts.
+
 Convenience script
 ==================
 

locale/zh_CN/LC_MESSAGES/guides/stacks/dars.po

@@ -1,192 +0,0 @@
-# SOME DESCRIPTIVE TITLE.
-# Copyright (C) 2019, many
-# This file is distributed under the same license as the Clear Linux*
-# Project Docs package.
-# FIRST AUTHOR <EMAIL@ADDRESS>, 2019.
-#
-msgid ""
-msgstr "Project-Id-Version: Clear Linux* Project Docs latest\n"
-"Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2019-08-09 14:33-0700\n"
-"PO-Revision-Date: 2019-09-04 16:21-0008\n"
-"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
-"Language: zh-Hans\n"
-"Language-Team: zh-Hans\n"
-"Plural-Forms: nplurals=2; plural=(n != 1)\n"
-"MIME-Version: 1.0\n"
-"Content-Type: text/plain; charset=utf-8\n"
-"Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Intel® International Developer Studio Version 4.1.273.0\n"
-
-#: ../../guides/stacks/dars.rst:4
-msgid "Data Analytics Reference Stack"
-msgstr "数据分析参考堆栈"
-
-#: ../../guides/stacks/dars.rst:6
-msgid ""
-"This guide explains how to use the :abbr:`DARS (Data Analytics Reference "
-"Stack)`, and to optionally build your own DARS container image."
-msgstr "本指南说明了如何使用 :abbr:`DARS (Data Analytics Reference Stack)`，以及如何选择性地构建您自己的 DARS 容器映像。"
-
-#: ../../guides/stacks/dars.rst:9
-msgid ""
-"Any system that supports Docker\\* containers can be used with DARS. This"
-" steps in this guide use |CL-ATTR| as the host system."
-msgstr "任何支持 Docker\\* 容器的系统都可与 DARS 一起使用。本指南中的这些步骤使用 |CL-ATTR| 作为主机系统。"
-
-#: ../../guides/stacks/dars.rst:17
-msgid "The Data Analytics Reference Stack release"
-msgstr "数据分析参考堆栈版本"
-
-#: ../../guides/stacks/dars.rst:19
-msgid ""
-"The Data Analytics Reference Stack (DARS) provides developers and "
-"enterprises a straightforward, highly optimized software stack for "
-"storing and processing large amounts of data.  More detail is available "
-"on the `DARS architecture and performance benchmarks`_."
-msgstr "数据分析参考堆栈 (DARS) 为开发人员和企业提供了一个简单、高度优化的软件堆栈来存储和处理大量数据。更多详细信息请参阅 `DARS architecture and performance benchmarks`_。"
-
-
-#: ../../guides/stacks/dars.rst:23
-msgid ""
-"The Data Analytics Reference Stack provides two pre-built Docker images, "
-"available on `Docker Hub`_:"
-msgstr "数据分析参考堆栈提供了两个预构建的 Docker 映像，可在 `Docker Hub`_ 获得："
-
-#: ../../guides/stacks/dars.rst:26
-msgid "A |CL|-derived `DARS with OpenBlas`_ stack optimized for `OpenBLAS`_"
-msgstr "一个从 |CL| 派生且针对 `OpenBLAS`_ 优化的 `DARS with OpenBlas`_ 堆栈"
-
-#: ../../guides/stacks/dars.rst:27
-msgid "A |CL|-derived  `DARS with Intel® MKL`_ stack optimized for `MKL`_"
-msgstr "一个从 |CL| 派生且针对 `MKL`_ 优化的 `DARS with MKL`_ 堆栈"
-
-#: ../../guides/stacks/dars.rst:29
-msgid ""
-"We recommend you view the latest component versions for each image in the"
-" :file:`README` found in the `Data Analytics Reference Stack`_ GitHub\\* "
-"repository. Because |CL| is a rolling distribution, the package version "
-"numbers in the |CL|-based containers may not be the latest released by "
-"|CL|."
-msgstr "我们建议您在 `DARS repository`_ 中找到 :file:`README`，查看每个映像的最新组件版本。由于 |CL| 是滚动发行的，基于 |CL| 的容器中的软件包版本号可能不是 |CL| 最新发布的版本号。"
-
-
-#: ../../guides/stacks/dars.rst:36
-msgid ""
-"The Data Analytics Reference Stack is a collective work, and each piece "
-"of software within the work has its own license.  Please see the `DARS "
-"Terms of Use`_ for more details about licensing and usage of the Data "
-"Analytics Reference Stack."
-msgstr "数据分析参考堆栈是一项集体成果，成果中的每一个软件都有自己的许可证。有关数据分析参考堆栈的许可和使用的更多详细信息，请参阅 `DARS Terms of Use`_。"
-
-
-#: ../../guides/stacks/dars.rst:42
-msgid "Using the Docker images"
-msgstr "使用 Docker 映像"
-
-#: ../../guides/stacks/dars.rst:44
-msgid ""
-"To immediately start using the latest stable DARS images, pull an image "
-"directly from `Docker Hub`_. This example uses the `DARS with Intel® "
-"MKL`_ Docker image."
-msgstr "要立即开始使用最新的稳定版 DARS 映像，请直接从 `Docker Hub`_ 提取。在本教程中，我们将使用 `Dars with MKL`_ 版本堆栈。"
-
-
-#: ../../guides/stacks/dars.rst:48
-msgid "Once you have downloaded the image, you can run it with"
-msgstr "下载完映像后，您可以使用以下命令运行它："
-
-#: ../../guides/stacks/dars.rst:54
-msgid ""
-"This will launch the image and drop you into a bash shell inside the "
-"container.  You will see output similar to the following:"
-msgstr "此命令将启动映像，并进入容器内的 bash shell 中。您将看到类似以下内容的输出："
-
-#: ../../guides/stacks/dars.rst:75
-msgid ""
-"The :command:`--ulimit nofile` parameter is currently required in order "
-"to increase the number of open files opened at certain point by the spark"
-" engine."
-msgstr ":command:`--ulimit nofile` 参数是当前必需的参数，以便增加 spark 引擎在某一时点打开的打开文件的数量。"
-
-#: ../../guides/stacks/dars.rst:80
-msgid "Building DARS images"
-msgstr "构建 DARS 映像"
-
-#: ../../guides/stacks/dars.rst:82
-msgid ""
-"If you choose to build your own DARS container images, you can customize "
-"them as needed. Use the provided Dockerfile as a baseline."
-msgstr "如果选择构建您自己的 DARS 容器映像，您可以根据需要对它们进行自定义。将提供的 Dockerfile 用作基准。"
-
-#: ../../guides/stacks/dars.rst:85
-msgid ""
-"To construct images with |CL|, start with a |CL| development platform "
-"that has the :command:`containers-basic-dev` bundle installed. Learn more"
-" about bundles and installing them by using :ref:`swupd-guide`."
-msgstr "要使用 |CL| 构建映像，请从安装了 :command:`containers-basic-dev` 捆绑包的 |CL| 开发平台开始。使用 :ref:`swupd-guide` 了解有关捆绑包和安装捆绑包的更多信息。"
-
-#: ../../guides/stacks/dars.rst:89
-msgid "Clone the `Data Analytics Reference Stack`_ GitHub\\* repository."
-msgstr "克隆 `Data Analytics Reference Stack`_ GitHub\\* 存储库。"
-
-#: ../../guides/stacks/dars.rst:95
-msgid ""
-"Inside the DARS directory, run :command:`make` to build OpenBLAS and MKL "
-"images."
-msgstr "在 DARS 目录中，运行 :command:`make` 来构建 OpenBLAS 和 MKL 映像。"
-
-#: ../../guides/stacks/dars.rst:101
-msgid ""
-"Run :command:`make baseline` to build the baseline CentOS image. "
-"Depending on the system, it may take a while to finish building."
-msgstr "然后运行 :command:`make baseline` 构建基准 CentOS 映像。根据系统的不同，可能需要一段时间才能完成构建。完成后，使用 :command:`Docker` 检查生成的映像。"
-
-
-#: ../../guides/stacks/dars.rst:108
-msgid "Once completed, check the resulting images with :command:`Docker`"
-msgstr "完成后，使用 :command:`Docker` 检查生成的映像"
-
-#: ../../guides/stacks/dars.rst:114
-msgid ""
-"You can use any of the resulting images to launch fully functional "
-"containers. If you need to customize the containers, you can edit the "
-"provided :file:`Dockerfile`."
-msgstr "您可以使用任何一个生成的映像来启动功能齐全的容器。如果需要自定义容器，您可以编辑所提供的 :file:`Dockerfile`。"
-
-#~ msgid ""
-#~ "This tutorial shows you how to use"
-#~ " the Data Analytics Reference Stack "
-#~ "(DARS), and to optionally build your "
-#~ "own images with the baseline Dockerfiles"
-#~ " provided in the `DARS repository`_. "
-#~ "Our assumption is that |CL-ATTR| "
-#~ "is the host. However, any system "
-#~ "that supports Docker\\* containers can "
-#~ "be used to follow these steps."
-#~ msgstr ""
-#~ "本教程介绍如何使用数据分析参考堆栈 (DARS)，以及如何使用 `DARS repository`_"
-#~ " 中提供的基准 Dockerfiles 来选择构建您自己的映像。我们假设 |CL-"
-#~ "ATTR| 是主机。但是，任何支持 Docker\\* 容器的系统都可以用来执行这些步骤。"
-
-#~ msgid ""
-#~ "If you choose to build your own"
-#~ " DARS container images, you can "
-#~ "customize them as needed. Use the "
-#~ "provided Dockerfile as a baseline. To"
-#~ " construct images with |CL|, start "
-#~ "with a |CL| development platform that"
-#~ " has the :command:`containers-basic-dev`"
-#~ " bundle installed. Learn more about "
-#~ "bundles and installing them by using "
-#~ ":ref:`swupd-guide`."
-#~ msgstr ""
-#~ "如果选择构建您自己的 DARS 容器映像，您可以根据需要对它们进行自定义。将提供的 Dockerfile"
-#~ " 用作基准。要使用 |CL| 构建映像，请从安装了 :command"
-#~ ":`containers-basic-dev` 捆绑包的 |CL| 开发平台开始。使用"
-#~ " :ref:`swupd-guide` 了解有关捆绑包和安装捆绑包的更多信息。"
-
-#~ msgid "First, clone the `DARS repository`_ from GitHub."
-#~ msgstr "首先，从 GitHub 中克隆 `DARS repository`_。"
-
-

locale/zh_CN/LC_MESSAGES/guides/stacks/dlrs/dlrs.po

@@ -1,656 +0,0 @@
-# SOME DESCRIPTIVE TITLE.
-# Copyright (C) 2019, many
-# This file is distributed under the same license as the Clear Linux*
-# Project Docs package.
-# FIRST AUTHOR <EMAIL@ADDRESS>, 2019.
-#
-msgid ""
-msgstr "Project-Id-Version: Clear Linux* Project Docs latest\n"
-"Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2019-08-09 14:33-0700\n"
-"PO-Revision-Date: 2019-09-04 16:21-0008\n"
-"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
-"Language: zh-Hans\n"
-"Language-Team: zh-Hans\n"
-"Plural-Forms: nplurals=2; plural=(n != 1)\n"
-"MIME-Version: 1.0\n"
-"Content-Type: text/plain; charset=utf-8\n"
-"Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Intel® International Developer Studio Version 4.1.273.0\n"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:4
-msgid "Deep Learning Reference Stack"
-msgstr "深度学习参考堆栈"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:6
-msgid ""
-"This guide describes how to run benchmarking workloads for TensorFlow\\*,"
-" PyTorch\\*, and Kubeflow in |CL-ATTR| using the Deep Learning Reference "
-"Stack."
-msgstr "本教程介绍如何在 |CL-ATTR| 中使用深度学习参考堆栈运行 TensorFlow\\*、PyTorch\\* 和 Kubeflow 基准工作负载。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:14
-msgid "Overview"
-msgstr "概述"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:16
-msgid ""
-"We created the Deep Learning Reference Stack to help AI developers "
-"deliver the best experience on Intel® Architecture. This stack reduces "
-"complexity common with deep learning software components, provides "
-"flexibility for customized solutions, and enables you to quickly "
-"prototype and deploy Deep Learning workloads. Use this guide to run "
-"benchmarking workloads on your solution."
-msgstr "我们打造了深度学习参考堆栈来帮助 AI 开发人员在英特尔架构上获得最佳开发体验。此堆栈降低了深度学习软件组件常见的复杂性，为自定义解决方案提供了灵活性，并使您能够快速构建原型并部署深度学习工作负载。使用本教程可在您的解决方案上运行基准工作负载。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:23
-msgid "The Deep Learning Reference Stack is available in the following versions:"
-msgstr "深度学习参考堆栈有以下版本："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:25
-msgid ""
-"`Intel MKL-DNN-VNNI`_, which is optimized using Intel® Math Kernel "
-"Library for Deep Neural Networks (Intel® MKL-DNN) primitives and "
-"introduces support for Intel® AVX-512 Vector Neural Network Instructions "
-"(VNNI)."
-msgstr "`Intel MKL-DNN-VNNI`_，它使用面向深度神经网络（英特尔® MKL-DNN）原语的英特尔®数学内核库进行优化，并支持英特尔® AVX-512 矢量神经网络指令 (VNI)。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:28
-msgid ""
-"`Intel MKL-DNN`_, which includes the TensorFlow framework optimized using"
-" Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) "
-"primitives."
-msgstr "`Intel MKL-DNN`_，它包括使用面向深度神经网络（英特尔® MKL-DNN）原语的英特尔®数学内核库进行优化的 TensorFlow 框架。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:31
-msgid "`Eigen`_, which includes `TensorFlow`_ optimized for Intel® architecture."
-msgstr "`Eigen`_，它包括针对英特尔®架构优化的 `TensorFlow`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:32
-msgid "`PyTorch with OpenBLAS`_, which includes PyTorch with OpenBlas."
-msgstr "`PyTorch with OpenBLAS`_，它包括 PyTorch with OpenBlas。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:33
-msgid ""
-"`PyTorch with Intel MKL-DNN`_, which includes PyTorch optimized using "
-"Intel® Math Kernel Library (Intel® MKL) and Intel MKL-DNN."
-msgstr "`PyTorch with Intel MKL-DNN`_，它包括使用英特尔®数学内核库（英特尔® MKL）和英特尔 MKL-DNN 进行优化的 PyTorch。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:38
-msgid ""
-"To take advantage of the Intel® AVX-512 and VNNI functionality with the "
-"Deep Learning Reference Stack, you must use the following hardware:"
-msgstr "要利用英特尔® AVX-512 和 VNI 功能以及深度学习参考堆栈，您必须使用以下硬件："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:41
-msgid "Intel® AVX-512 images require an Intel® Xeon® Scalable Platform"
-msgstr "英特尔® AVX-512 映像需要使用英特尔®至强®可扩展平台"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:42
-msgid "VNNI requires a 2nd generation Intel® Xeon® Scalable Platform"
-msgstr "VNNI 需要使用第二代英特尔®至强®可扩展平台"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:45
-msgid "Stack features"
-msgstr "堆栈功能和特性"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:47
-msgid "`DLRS V3.0`_  release announcement."
-msgstr "`DLRS V3.0`_ 发布公告。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:48
-msgid "Deep Learning Reference Stack v2.0 including current `PyTorch benchmark`_."
-msgstr "深度学习参考堆栈 v2.0，包括最新的 `PyTorch benchmark results`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:50
-msgid ""
-"Deep Learning Reference Stack v1.0 including current `TensorFlow "
-"benchmark`_ results."
-msgstr "深度学习参考堆栈 v1.0，包括最新的 `TensorFlow benchmark results`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:52
-msgid ""
-"`DLRS Release notes`_  on Github\\* for the latest release of Deep "
-"Learning Reference Stack."
-msgstr "`DLRS Release notes`_ on Github\\*，了解深度学习参考堆栈的最新版本。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:57
-msgid ""
-"The Deep Learning Reference Stack is a collective work, and each piece of"
-" software within the work has its own license.  Please see the `DLRS "
-"Terms of Use`_ for more details about licensing and usage of the Deep "
-"Learning Reference Stack."
-msgstr "深度学习参考堆栈是一项集体成果，成果中的每一个软件都有自己的许可证。有关深度学习参考堆栈的许可和使用的更多详细信息，请参阅 `DLRS Terms of Use`_。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:62
-msgid "Prerequisites"
-msgstr "必备条件"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:64
-msgid ":ref:`Install <bare-metal-install-desktop>` |CL| on your host system"
-msgstr "在主机系统上 :ref:`Install <bare-metal-install-desktop>` |CL|"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:65
-msgid ":command:`containers-basic` bundle"
-msgstr ":command:`containers-basic` 捆绑包"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:66
-msgid ":command:`cloud-native-basic` bundle"
-msgstr ":command:`cloud-native-basic` 捆绑包"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:68
-msgid ""
-"In |CL|, :command:`containers-basic` includes Docker\\*, which is "
-"required for TensorFlow and PyTorch benchmarking. Use the "
-":command:`swupd` utility to check if :command:`containers-basic` and "
-":command:`cloud-native-basic` are present:"
-msgstr "在 |CL| 中，:command:`containers-basic` 包括 TensorFlow 和 PyTorch 基准测试所必需的 Docker\\*。使用 :command:`swupd` 实用程序检查 :command:`containers-basic` 和 :command:`cloud-native-basic` 是否存在："
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:77
-msgid ""
-"To install the :command:`containers-basic` or :command:`cloud-native-"
-"basic` bundles, enter:"
-msgstr "要安装 :command:`containers-basic` 或 :command:`cloud-native-basic` 捆绑包，请输入："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:84
-msgid ""
-"Docker is not started upon installation of the :command:`containers-"
-"basic` bundle. To start Docker, enter:"
-msgstr "安装 :command:`containers-basic` 捆绑包后 Docker 不会启动。要启动 Docker，请输入："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:91
-msgid ""
-"To ensure that Kubernetes is correctly installed and configured, follow "
-"the instructions in :ref:`kubernetes`."
-msgstr "要确保正确安装和配置 Kubernetes，请遵循 :ref:`kubernetes` 中的说明。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:95
-msgid "Version compatibility"
-msgstr "版本兼容性"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:97
-msgid "We validated these steps against the following software package versions:"
-msgstr "我们根据以下软件包版本验证了这些步骤："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:99
-msgid "|CL| 26240 (Minimum supported version)"
-msgstr "|CL| 26240（支持的最低版本）"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:100
-msgid "Docker 18.06.1"
-msgstr "Docker 18.06.1"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:101
-msgid "Kubernetes 1.11.3"
-msgstr "Kubernetes 1.11.3"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:102
-msgid "Go 1.11.12"
-msgstr "Go 1.11.12"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:107
-msgid ""
-"The Deep Learning Reference Stack was developed to provide the best user "
-"experience when executed on a |CL| host.  However, as the stack runs in a"
-" container environment, you should be able to complete the following "
-"sections of this guide on other Linux* distributions, provided they "
-"comply with the Docker*, Kubernetes* and Go* package versions listed "
-"above. Look for your distribution documentation on how to update packages"
-" and manage Docker services."
-msgstr "深度学习参考堆栈是为了在 |CL| 主机上执行时获得最佳用户体验而开发的。但是，该堆栈在容器环境中运行时，您应该能够在其他 Linux* 发行版上完成本教程的以下部分，只要这些发行版满足上面列出的 Docker*、Kubernetes* 和 Go* 软件包版本。查找关于如何更新软件包和管理 Docker 服务的分发版文档。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:112
-msgid "TensorFlow single and multi-node benchmarks"
-msgstr "TensorFlow 单节点和多节点基准测试"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:114
-msgid ""
-"This section describes running the `TensorFlow Benchmarks`_ in single "
-"node. For multi-node testing, replicate these steps for each node. These "
-"steps provide a template to run other benchmarks, provided that they can "
-"invoke TensorFlow."
-msgstr "本部分介绍在单节点中运行 `TensorFlow benchmarks`_。对于多节点测试，请为每个节点重复这些步骤。这些步骤提供了运行其他基准测试的模板，前提是它们可以调用 TensorFlow。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:121
-msgid ""
-"Performance test results for the Deep Learning Reference Stack and for "
-"this guide were obtained using `runc` as the runtime."
-msgstr "深度学习参考堆栈和本教程的性能测试结果是使用 `runc` 作为运行时获得的。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:124
-msgid ""
-"Download either the `Eigen`_ or the `Intel MKL-DNN`_ Docker image from "
-"`Docker Hub`_."
-msgstr "从 `Docker Hub`_ 下载 `Eigen`_ 或 `Intel MKL-DNN`_ Docker 映像。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:127 ../../guides/stacks/dlrs/dlrs.rst:169
-msgid "Run the image with Docker:"
-msgstr "使用 Docker 运行映像："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:136 ../../guides/stacks/dlrs/dlrs.rst:177
-msgid ""
-"Launching the Docker image with the :command:`-i` argument starts "
-"interactive mode within the container. Enter the following commands in "
-"the running container."
-msgstr "使用 :command:`-i` 参数启动 Docker 映像，从而在容器内启动交互模式。在正在运行的容器中输入以下命令。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:140
-msgid "Clone the benchmark repository in the container:"
-msgstr "克隆容器中的基准测试存储库："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:146 ../../guides/stacks/dlrs/dlrs.rst:187
-msgid "Execute the benchmark script:"
-msgstr "执行基准测试脚本："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:154
-msgid ""
-"You can replace the model with one of your choice supported by the "
-"TensorFlow benchmarks."
-msgstr "您可以将该模型更换为 TensorFlow 支持的其他模型。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:157
-msgid ""
-"If you are using an FP32 based model, it can be converted to an int8 "
-"model using `Intel® quantization tools`_."
-msgstr "如果使用基于 FP32 的模型，可以使用 `Intel® quantization tools`_ 将其转换为 int8 模型。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:161
-msgid "PyTorch single and multi-node benchmarks"
-msgstr "PyTorch 单节点和多节点基准测试"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:163
-msgid ""
-"This section describes running the `PyTorch benchmarks`_ for Caffe2 in "
-"single node."
-msgstr "本部分介绍在单节点中运行针对 Caffe2 的 `PyTorch benchmarks`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:166
-msgid ""
-"Download either the `PyTorch with OpenBLAS`_ or the `PyTorch with Intel "
-"MKL-DNN`_ Docker image from `Docker Hub`_."
-msgstr "从 `Docker Hub`_ 下载 `PyTorch with OpenBLAS`_ 或 `PyTorch with Intel MKL-DNN`_ Docker 映像。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:181
-msgid "Clone the benchmark repository:"
-msgstr "克隆基准测试存储库："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:197
-msgid "Kubeflow multi-node benchmarks"
-msgstr "Kubeflow 多节点基准测试"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:199
-msgid ""
-"The benchmark workload runs in a Kubernetes cluster. The guide uses "
-"`Kubeflow`_ for the Machine Learning workload deployment on three nodes."
-msgstr "基准测试工作负载在 Kubernetes 集群中运行。本教程使用 `Kubeflow`_ 在三个节点上部署机器学习工作负载。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:204
-msgid ""
-"If you choose the Intel® MKL-DNN or Intel® MKL-DNN-VNNI image, your "
-"platform must support the Intel® AVX-512 instruction set. Otherwise, an "
-"*illegal instruction* error may appear, and you won’t be able to complete"
-" this guide."
-msgstr "如果选择英特尔® MKL-DNN 或英特尔® MKL-DNN-VNNI 映像，您的平台必须支持英特尔® AVX-512 指令集。否则，可能会出现非法指令错误，导致无法完成本教程。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:210
-msgid "Kubernetes setup"
-msgstr "Kubernetes 设置"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:212
-msgid ""
-"Follow the instructions in the :ref:`kubernetes` tutorial to get set up "
-"on |CL|. The Kubernetes community also has instructions for creating a "
-"cluster, described in `Creating a single control-plane cluster with "
-"kubeadm`_."
-msgstr "按照 :ref:`kubernetes` 教程中的说明在 |CL| 上进行设置。Kubernetes 社区也提供了 `Creating a single control-plane cluster with kubeadm`_。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:217
-msgid "Kubernetes networking"
-msgstr "Kubernetes 网络连接"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:219
-msgid ""
-"We used `flannel`_ as the network provider for these tests. If you prefer"
-" a different network layer, refer to the Kubernetes network documentation"
-" described in `Creating a single control-plane cluster with kubeadm`_ for"
-" setup."
-msgstr "在这些测试中，我们使用 `flannel`_ 作为网络提供程序。如果青睐不同的网络层，请参阅 Kubernetes `Creating a single control-plane cluster with kubeadm`_ 进行设置。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:224
-msgid "Kubectl"
-msgstr "Kubectl"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:226
-msgid ""
-"You can use kubectl to run commands against your Kubernetes cluster.  "
-"Refer to the `Overview of kubectl`_ for details on syntax and operations."
-" Once you have a working cluster on Kubernetes, use the following YAML "
-"script to start a pod with a simple shell script, and keep the pod open."
-msgstr "您可以使用 kubectl 对您的 Kubernetes 集群运行命令。有关语法和操作的详细信息，请参阅 `Overview of kubectl`_。建立一个 Kubernetes 工作集群后，请使用下面的 YAML 脚本启动一个含有简单 shell 脚本的 Pod，并保持该 Pod 处于打开状态。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:231
-msgid "Copy this example.yaml script to your system:"
-msgstr "将 example.yaml 脚本复制到您的系统中："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:248
-msgid "Execute the script with kubectl:"
-msgstr "使用 kubectl 执行该脚本："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:254
-msgid ""
-"This script opens a single pod. More robust solutions would create a "
-"deployment or inject a python script or larger shell script into the "
-"container."
-msgstr "该脚本打开一个 Pod。更稳健的解决方案是创建部署，或者将 python 脚本或更大的 shell 脚本注入容器。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:258
-msgid "Images"
-msgstr "图像"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:260
-msgid ""
-"You must add `launcher.py`_ to the Docker image to include the Deep "
-"Learning Reference Stack and put the benchmarks repo in the correct "
-"location. Note that this guide uses Kubeflow v0.4.0, and cannot guarantee"
-" results if you use a different version."
-msgstr "您必须将 `launcher.py`_ 添加到 Docker 映像中，以包含深度学习参考堆栈，并将基准测试存储库放在正确的位置。请注意，本教程使用 Kubeflow v0.4.0。如果使用不同的版本，则不能保证结果。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:264
-msgid "From the Docker image, run the following:"
-msgstr "从 Docker 映像中，运行以下命令："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:273
-msgid "Your entry point becomes: :file:`/opt/launcher.py`."
-msgstr "您的入口点变成 :file:`/opt/launcher.py`。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:275
-msgid "This builds an image that can be consumed directly by TFJob from Kubeflow."
-msgstr "这会构建一个可供 TFJob 从 Kubeflow 直接使用的映像。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:278
-msgid "ksonnet\\*"
-msgstr "ksonnet\\*"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:280
-msgid ""
-"Kubeflow uses ksonnet\\* to manage deployments, so you must install it "
-"before setting up Kubeflow."
-msgstr "Kubeflow 使用 ksonnet\\* 来管理部署，因此您必须在设置 Kubeflow 之前安装它。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:283
-msgid ""
-"ksonnet was added to the :command:`cloud-native-basic` bundle in |CL| "
-"version 27550. If you are using an older |CL| version (not recommended), "
-"you must manually install ksonnet as described below."
-msgstr "ksonnet 已添加到 |CL| 版本 27550 中的 :command:`cloud-native-basic` 捆绑包中。如果使用的是较旧的 |CL| 版本（不推荐），您必须如下所述手动安装 ksonnet。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:287
-msgid "On |CL|, follow these steps:"
-msgstr "在 |CL| 上，请按照下列步骤操作："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:298
-msgid ""
-"After the ksonnet installation is complete, ensure that binary `ks` is "
-"accessible across the environment."
-msgstr "ksonnet 安装完成后，确保可在整个环境中访问 `ks` 二进制文件。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:302
-msgid "Kubeflow"
-msgstr "Kubeflow"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:304
-msgid ""
-"Once you have Kubernetes running on your nodes, set up `Kubeflow`_ by "
-"following these instructions from the `Getting Started with Kubeflow`_ "
-"guide."
-msgstr "Kubernetes 在节点上运行后，请按照 `Getting Started with Kubeflow`_ 中的说明设置 `Kubeflow`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:322
-msgid "Next, deploy the primary package for our purposes: tf-job-operator."
-msgstr "接下来，为我们的目的部署主要软件包：tf-job-operator。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:332
-msgid ""
-"This creates the CustomResourceDefinition (CRD) endpoint to launch a "
-"TFJob."
-msgstr "这将创建 CustomResourceDefinition (CRD) 端点来启动 TFJob。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:335
-msgid "Run a TFJob"
-msgstr "运行 TFJob"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:337
-msgid "Get the ksonnet registries for deploying TFJobs from `dlrs-tfjob`_."
-msgstr "从 `dlrs-tfjob`_ 获取用于部署 TFJobs 的 ksonnet 注册表。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:339
-msgid "Install the TFJob components as follows:"
-msgstr "按照以下步骤安装 TFJob 组件："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:347
-msgid "Export the image name to use for the deployment:"
-msgstr "导出用于部署的映像名称："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:355
-msgid "Replace <docker_name> with the image name you specified in previous steps."
-msgstr "将 <docker_name> 替换为前述步骤中指定的映像名称。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:357
-msgid ""
-"Generate Kubernetes manifests for the workloads and apply them using "
-"these commands:"
-msgstr "为工作负载生成 Kubernetes 清单，并使用以下命令应用这些清单："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:367
-msgid "This replicates and deploys three test setups in your Kubernetes cluster."
-msgstr "这会在 Kubernetes 集群中复制和部署三个测试设置。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:370
-msgid "Results of running this guide"
-msgstr "运行本教程的结果"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:372
-msgid ""
-"You must parse the logs of the Kubernetes pod to retrieve performance "
-"data. The pods will still exist post-completion and will be in "
-"‘Completed’ state. You can get the logs from any of the pods to inspect "
-"the benchmark results. More information about Kubernetes logging is "
-"available in the Kubernetes `Logging Architecture`_ documentation."
-msgstr "您必须解析 Kubernetes Pod 的日志来检索性能数据。完成后，Pod 仍会存在，并将处于“已完成”状态。您可以从任何一个 Pod 中获取日志来检查基准测试结果。有关 Kubernetes 日志记录的更多信息，请参见 Kubernetes `Logging Architecture`_ 文档。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:379
-msgid "Use Jupyter Notebook"
-msgstr "使用 Jupyter Notebook"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:381
-msgid ""
-"This example uses the `PyTorch with OpenBLAS`_ container image. After it "
-"is downloaded, run the Docker image with :command:`-p` to specify the "
-"shared port between the container and the host. This example uses port "
-"8888."
-msgstr "本示例使用 `PyTorch with OpenBLAS`_ 容器映像。下载后，使用 :command:`-p` 运行 Docker 映像，以指定容器和主机之间的共享端口。本示例使用端口 8888。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:389
-msgid ""
-"After you start the container, launch the Jupyter Notebook. This command "
-"is executed inside the container image."
-msgstr "启动容器后，启动 Jupyter Notebook。该命令在容器映像内执行。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:396
-msgid ""
-"After the notebook has loaded, you will see output similar to the "
-"following:"
-msgstr "加载笔记本后，您将看到类似以下内容的输出："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:404
-msgid ""
-"From your host system, or any system that can access the host's IP "
-"address, start a web browser with the following. If you are not running "
-"the browser on the host system, replace :command:`127.0.0.1` with the IP "
-"address of the host."
-msgstr "从您的主机系统或任何可以访问主机 IP 地址的系统，使用以下命令启动 Web 浏览器。如果没有在主机系统上运行浏览器，请将 :command:`127.0.0.1` 更换为主机的 IP 地址。"
-
-
-#: ../../guides/stacks/dlrs/dlrs.rst:412
-msgid "Your browser displays the following:"
-msgstr "您的浏览器会显示以下内容："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:418
-msgid "Figure 1: :guilabel:`Jupyter Notebook`"
-msgstr "图 1： :guilabel:`Jupyter Notebook`"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:421
-msgid ""
-"To create a new notebook, click :guilabel:`New` and select "
-":guilabel:`Python 3`."
-msgstr "要创建新笔记本，请点击 :guilabel:`New`，然后选择 :guilabel:`Python 3`。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:427
-msgid "Figure 2: Create a new notebook"
-msgstr "图 2：创建一个新笔记本"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:429
-msgid "A new, blank notebook is displayed, with a cell ready for input."
-msgstr "此时将显示一个新的空白笔记本，其中有一个单元格可供输入内容。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:436
-msgid ""
-"To verify that PyTorch is working, copy the following snippet into the "
-"blank cell, and run the cell."
-msgstr "要验证 PyTorch 是否正在工作，请将以下片段复制到空白单元格中，并运行该单元格。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:450
-msgid "When you run the cell, your output will look something like this:"
-msgstr "运行单元格时，您的输出将如下所示："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:456
-msgid ""
-"You can continue working in this notebook, or you can download existing "
-"notebooks to take advantage of the Deep Learning Reference Stack's "
-"optimized deep learning frameworks. Refer to `Jupyter Notebook`_ for "
-"details."
-msgstr "您可以继续在此笔记本中工作，也可以下载现有笔记本来利用深度学习参考堆栈的优化深度学习框架。详情请参阅 `Jupyter Notebook`_。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:461
-msgid "Uninstallation"
-msgstr "卸载"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:463
-msgid ""
-"To uninstall the Deep Learning Reference Stack, you can choose to stop "
-"the container so that it is not using system resources, or you can stop "
-"the container and delete it to free storage space."
-msgstr "要卸载深度学习参考堆栈，您可以选择停止容器以使其不使用系统资源，或者可以停止容器并将其删除以释放存储空间。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:467
-msgid "To stop the container, execute the following from your host system:"
-msgstr "要停止容器，请从主机系统执行以下操作："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:469
-msgid "Find the container's ID"
-msgstr "找到容器的 ID"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:475
-msgid "This will result in output similar to the following:"
-msgstr "这将产生类似于以下内容的输出："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:482
-msgid ""
-"You can then use the ID or container name to stop the container. This "
-"example uses the name \"oss\":"
-msgstr "然后，您可以使用 ID 或容器名称来停止容器。本示例使用名称 \"oss\"："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:490
-msgid "Verify that the container is not running"
-msgstr "验证容器未在运行"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:497
-msgid "To delete the container from your system you need to know the Image ID:"
-msgstr "要从系统中删除容器，您需要知道映像 ID："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:503
-msgid "This command results in output similar to the following:"
-msgstr "该命令会产生类似于以下内容的输出："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:511
-msgid "To remove an image use the image ID:"
-msgstr "要移除映像，请使用映像 ID："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:527
-msgid ""
-"Note that you can execute the :command:`docker rmi` command using only "
-"the first few characters of the image ID, provided they are unique on the"
-" system."
-msgstr "请注意，您可以只使用映像 ID 的前几个字符来执行 :command:`docker rmi` 命令，前提是它们在系统上是唯一的。"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:529
-msgid "Once you have removed the image, you can verify it has been deleted with:"
-msgstr "移除映像后，您可以通过以下方式验证它是否已被移除："
-
-#: ../../guides/stacks/dlrs/dlrs.rst:537
-msgid "Related topics"
-msgstr "相关主题"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:539
-msgid "`DLRS V3.0`_ release announcement"
-msgstr "`DLRS V3.0`_ 发布公告"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:540
-msgid "`TensorFlow Benchmarks`_"
-msgstr "`TensorFlow Benchmarks`_"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:541
-msgid "`PyTorch benchmarks`_"
-msgstr "`PyTorch benchmarks`_"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:542
-msgid "`Kubeflow`_"
-msgstr "`Kubeflow`_"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:543
-msgid ":ref:`kubernetes` tutorial"
-msgstr ":ref:`kubernetes` 教程"
-
-#: ../../guides/stacks/dlrs/dlrs.rst:544
-msgid "`Jupyter Notebook`_"
-msgstr "`Jupyter Notebook`_"
-
-#~ msgid "Deep Learning Reference Stack `V3.0 release announcement`_."
-#~ msgstr "深度学习参考堆栈 `V3.0 release announcement`_。"
-
-#~ msgid ""
-#~ "You must parse the logs of the "
-#~ "Kubernetes pod to retrieve performance "
-#~ "data. The pods will still exist "
-#~ "post-completion and will be in "
-#~ "‘Completed’ state. You can get the "
-#~ "logs from any of the pods to "
-#~ "inspect the benchmark results. More "
-#~ "information about `Kubernetes logging`_ is "
-#~ "available from the Kubernetes community."
-#~ msgstr ""
-#~ "您必须解析 Kubernetes Pod 的日志来检索性能数据。完成后，Pod "
-#~ "仍会存在，并将处于“已完成”状态。您可以从任何一个 Pod 中获取日志来检查基准测试结果。有关 "
-#~ "`Kubernetes logging`_ 的更多信息可从 Kubernetes 社区获取。"
-
-#~ msgid "Deep Learning Reference Stack `V3.0 release announcement`_"
-#~ msgstr "深度学习参考堆栈 `V3.0 release announcement`_"
-
-

locale/zh_CN/LC_MESSAGES/guides/stacks/greengrass.po

@@ -1,711 +0,0 @@
-# SOME DESCRIPTIVE TITLE.
-# Copyright (C) 2019, many
-# This file is distributed under the same license as the Clear Linux*
-# Project Docs package.
-# FIRST AUTHOR <EMAIL@ADDRESS>, 2019.
-#
-msgid ""
-msgstr "Project-Id-Version: Clear Linux* Project Docs latest\n"
-"Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2019-08-09 14:33-0700\n"
-"PO-Revision-Date: 2019-09-04 16:21-0008\n"
-"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
-"Language: zh-Hans\n"
-"Language-Team: zh-Hans\n"
-"Plural-Forms: nplurals=2; plural=(n != 1)\n"
-"MIME-Version: 1.0\n"
-"Content-Type: text/plain; charset=utf-8\n"
-"Content-Transfer-Encoding: 8bit\n"
-"Generated-By: Intel® International Developer Studio Version 4.1.273.0\n"
-
-#: ../../guides/stacks/greengrass.rst:4
-msgid "Enable AWS Greengrass\\* and OpenVINO™ toolkit"
-msgstr "启用 AWS Greengrass\\* 和 OpenVINO™ 工具包"
-
-#: ../../guides/stacks/greengrass.rst:6
-msgid ""
-"This guide explains how to enable AWS Greengrass\\* and OpenVINO™ "
-"toolkit. Specifically, the guide demonstrates how to:"
-msgstr "本指南说明了如何启用 AWS Greengrass\\* 和 OpenVINO™ 工具包。具体而言，该指南演示了如何："
-
-#: ../../guides/stacks/greengrass.rst:9
-msgid "Set up the Intel® edge device with |CL-ATTR|"
-msgstr "使用 |CL-ATTR| 设置英特尔®边缘设备"
-
-#: ../../guides/stacks/greengrass.rst:10
-msgid ""
-"Install the OpenVINO™ toolkit and Amazon Web Services\\* (AWS\\*) "
-"Greengrass\\* software stacks"
-msgstr "安装 OpenVINO™ 工具包和 Amazon Web Services\\* (AWS\\*) Greengrass\\* 软件堆栈"
-
-#: ../../guides/stacks/greengrass.rst:12
-msgid ""
-"Use AWS Greengrass\\* and AWS Lambda\\* to deploy the FaaS samples from "
-"the cloud"
-msgstr "使用 AWS Greengrass\\* 和 AWS Lambda\\* 从云中部署 FaaS 示例"
-
-#: ../../guides/stacks/greengrass.rst:20
-msgid "Overview"
-msgstr "概述"
-
-#: ../../guides/stacks/greengrass.rst:22
-msgid ""
-"Hardware accelerated Function-as-a-Service (FaaS) enables cloud "
-"developers to deploy inference functionalities [1] on Intel® IoT edge "
-"devices with accelerators (CPU, Integrated GPU, Intel® FPGA, and Intel® "
-"Movidius™ technology). These functions provide a great developer "
-"experience and seamless migration of visual analytics from cloud to edge "
-"in a secure manner using a containerized environment. Hardware-"
-"accelerated FaaS provides the best-in-class performance by accessing "
-"optimized deep learning libraries on Intel® IoT edge devices with "
-"accelerators."
-msgstr "硬件加速的功能即服务 (FaaS) 有助于云开发人员在搭载加速器的英特尔® IoT 边缘设备（CPU、集成 GPU、英特尔® FPGA 和英特尔® Movidius™ 技术）上部署推理功能 [1]。这些功能使用容器化环境，为开发人员提供了出色的体验，有助于开发人员将可视化分析从云安全地迁移到边缘。硬件加速的 FaaS 支持在搭载加速器的英特尔® IoT 边缘设备上访问经过优化的深度学习库，实现业界最佳性能。"
-
-
-#: ../../guides/stacks/greengrass.rst:32
-msgid "Supported platforms"
-msgstr "支持的平台"
-
-#: ../../guides/stacks/greengrass.rst:34
-msgid "Operating System: |CL| latest release"
-msgstr "操作系统：|CL| 最新版本"
-
-#: ../../guides/stacks/greengrass.rst:35
-msgid "Hardware:       Intel® core platforms (that support inference on CPU only)"
-msgstr "硬件：英特尔®酷睿™平台（本教程仅支持 CPU 推理。）"
-
-#: ../../guides/stacks/greengrass.rst:38
-msgid "Sample description"
-msgstr "示例说明"
-
-#: ../../guides/stacks/greengrass.rst:40
-msgid ""
-"The AWS Greengrass samples are located at `Edge-Analytics-FaaS`_. This "
-"guide uses the 1.0 version of the source code."
-msgstr "AWS Greengrass 示例位于 `Edge-Analytics-FaaS`_ 中。本教程使用 1.0 版本的源代码。"
-
-#: ../../guides/stacks/greengrass.rst:43
-msgid "|CL| provides the following AWS Greengrass samples:"
-msgstr "|CL| 提供以下 AWS Greengrass 示例："
-
-#: ../../guides/stacks/greengrass.rst:45
-msgid "`greengrass_classification_sample.py`_"
-msgstr "`greengrass_classification_sample.py`_"
-
-#: ../../guides/stacks/greengrass.rst:47
-msgid ""
-"This AWS Greengrass sample classifies a video stream using classification"
-" networks such as AlexNet and GoogLeNet and publishes top-10 results on "
-"AWS\\* IoT Cloud every second."
-msgstr "此 AWS Greengrass 示例使用 AlexNet 和 GoogLeNet 等分类网络对视频流进行分类，并每秒在 AWS\\* IoT 云上发布前十名结果。"
-
-
-#: ../../guides/stacks/greengrass.rst:51
-msgid "`greengrass_object_detection_sample_ssd.py`_"
-msgstr "`greengrass_object_detection_sample_ssd.py`_"
-
-#: ../../guides/stacks/greengrass.rst:53
-msgid ""
-"This AWS Greengrass sample detects objects in a video stream and "
-"classifies them using single-shot multi-box detection (SSD) networks such"
-" as SSD Squeezenet, SSD Mobilenet, and SSD300. This sample publishes "
-"detection outputs such as class label, class confidence, and bounding box"
-" coordinates on AWS IoT Cloud every second."
-msgstr "此 AWS Greengrass 示例会检测视频流中的对象，并使用单步多框检测 (SSD) 网络（例如 SSD Squeezenet、SSD Mobilenet 和 SSD300）对它们进行分类。此示例每秒在 AWS IoT 云上发布检测输出，如类标签、类置信度和边界框坐标。"
-
-
-#: ../../guides/stacks/greengrass.rst:61
-msgid "Install the OS on the edge device"
-msgstr "在边缘设备上安装操作系统"
-
-#: ../../guides/stacks/greengrass.rst:63
-msgid ""
-"Start with a clean installation of |CL| on a new system, using the :ref"
-":`bare-metal-install-desktop`, found in :ref:`get-started`."
-msgstr "使用 :ref:`get-started` 中的 :ref:`bare-metal-install-desktop`，在新系统上安装干净的 |CL|。"
-
-
-#: ../../guides/stacks/greengrass.rst:67
-msgid "Create user accounts"
-msgstr "创建用户帐户"
-
-#: ../../guides/stacks/greengrass.rst:69
-msgid ""
-"After |CL| is installed, create two user accounts. Create an "
-"administrative user in |CL| and create a user account for the Greengrass "
-"services to use ( see Greengrass user below)."
-msgstr "安装 |CL| 后，创建两个用户帐户。在 |CL| 中创建一个管理用户，并为要使用的 Greengrass 服务创建一个用户帐户（请参阅下面的 Greengrass 用户）。"
-
-
-#: ../../guides/stacks/greengrass.rst:73
-msgid ""
-"Create a new user and set a password for that user. Enter the following "
-"commands as ``root``:"
-msgstr "创建新用户并为该用户设置密码。以 ``root`` 用户身份输入以下命令："
-
-#: ../../guides/stacks/greengrass.rst:81
-msgid ""
-"Next, enable the :command:`sudo` command for your new <userid>. Add "
-"<userid> to the `wheel` group:"
-msgstr "接下来，为新的 <userid> 启用 :command:`sudo` 命令。将 <userid> 添加到 `wheel` 组："
-
-#: ../../guides/stacks/greengrass.rst:88
-msgid "Create a :file:`/etc/fstab` file."
-msgstr "创建一个 :file:`/etc/fstab` 文件。"
-
-#: ../../guides/stacks/greengrass.rst:96
-msgid ""
-"By default, |CL| does not create an :file:`/etc/fstab` file. You must "
-"create this file before the Greengrass service runs."
-msgstr "默认情况下，|CL| 不会创建 :file:`/etc/fstab` 文件。您必须在 Greengrass 服务运行之前创建此文件。"
-
-#: ../../guides/stacks/greengrass.rst:100
-msgid "Add required bundles"
-msgstr "添加所需的捆绑包"
-
-#: ../../guides/stacks/greengrass.rst:102
-msgid ""
-"Use the :command:`swupd` software updater utility to add the prerequisite"
-" bundles for the OpenVINO software stack:"
-msgstr "使用 :command:`swupd` 软件更新程序实用程序添加 OpenVINO 软件堆栈必备的软件包："
-
-#: ../../guides/stacks/greengrass.rst:111
-msgid "Learn more about how to :ref:`swupd-guide`."
-msgstr "详细了解如何 :ref:`swupd-guide`。"
-
-#: ../../guides/stacks/greengrass.rst:113
-msgid ""
-"The :command:`computer-vision-basic` bundle installs the OpenVINO™ "
-"toolkit, and the sample models optimized for Intel® edge platforms."
-msgstr ":command:`computer-vision-basic` 捆绑包会安装 OpenVINO™ 工具包以及针对英特尔®边缘平台优化的示例模型。"
-
-#: ../../guides/stacks/greengrass.rst:117
-msgid "Convert deep learning models"
-msgstr "转换深度学习模型"
-
-#: ../../guides/stacks/greengrass.rst:120
-msgid "Locate sample models"
-msgstr "找到示例模型"
-
-#: ../../guides/stacks/greengrass.rst:122
-msgid ""
-"There are two types of provided models that can be used in conjunction "
-"with AWS Greengrass for this guide: classification or object detection."
-msgstr "本教程中提供了两种可以与 AWS Greengrass 配合使用的模型：分类和对象检测。"
-
-#: ../../guides/stacks/greengrass.rst:125
-msgid ""
-"To complete this guide using an image classification model, download the "
-"BVLC AlexNet model files `bvlc_alexnet.caffemodel`_ and "
-"`deploy.prototxt`_ to the default model_location at "
-":file:`/usr/share/openvino/models`. Any custom pre-trained classification"
-" models can be used with the classification sample."
-msgstr "要使用图像分类模型完成本教程，请将 BVLC AlexNet 模型文件 `bvlc_alexnet.caffemodel`_ 和 `deploy.prototxt`_ 下载到 :file:`/usr/share/openvino/models` 处的默认 model_location。预先训练的任何自定义分类模型都可与分类示例配合使用。"
-
-
-#: ../../guides/stacks/greengrass.rst:131
-msgid ""
-"For object detection, the sample models optimized for Intel® edge "
-"platforms are included with the computer-vision-basic bundle installation"
-" at :file:`/usr/share/openvino/models`. These models are provided as an "
-"example; you may also use a custom SSD model with the Greengrass object "
-"detection sample."
-msgstr "对于对象检测，安装 computer-vision-basic 捆绑包时会在 :file:`/usr/share/openvino/models` 处附带针对英特尔®边缘平台优化的示例模型。这些模型作为示例提供；但是，您也可以将自定义 SSD 模型与 Greengrass 对象检测示例结合使用。"
-
-
-#: ../../guides/stacks/greengrass.rst:137
-msgid "Run model optimizer"
-msgstr "运行模型优化器"
-
-#: ../../guides/stacks/greengrass.rst:139
-msgid ""
-"Follow the instructions in the `Model Optimizer Developer Guide`_ for "
-"converting deep learning models to Intermediate Representation using "
-"Model Optimizer. To optimize either of the sample models described above,"
-" run one of the following commands."
-msgstr "遵循 `Model Optimizer Developer Guide`_ 中的说明，使用 Model Optimizer 将深度学习模型转换为 Intermediate Representation。要优化上述任一示例模型，请运行以下命令之一。"
-
-#: ../../guides/stacks/greengrass.rst:143
-msgid "For classification using BVLC AlexNet model:"
-msgstr "对于使用 BVLC AlexNet 模型的分类："
-
-#: ../../guides/stacks/greengrass.rst:152
-msgid "For object detection using SqueezeNetSSD-5Class model:"
-msgstr "对于使用 SqueezeNetSSD-5Class 模型的对象检测："
-
-#: ../../guides/stacks/greengrass.rst:161
-msgid "In these examples:"
-msgstr "在这些示例中："
-
-#: ../../guides/stacks/greengrass.rst:163
-msgid "`<model_location>` is :file:`/usr/share/openvino/models`."
-msgstr "`<model_location>` 是 :file:`/usr/share/openvino/models`。"
-
-#: ../../guides/stacks/greengrass.rst:165
-msgid "`<data_type>` is FP32 or FP16, depending on target device."
-msgstr "`<data_type>` 是 FP32 或 FP16，具体取决于目标设备。"
-
-#: ../../guides/stacks/greengrass.rst:167
-msgid ""
-"`<output_dir>` is the directory where the Intermediate Representation "
-"(IR) is stored. IR contains .xml format corresponding to the network "
-"structure and .bin format corresponding to weights. This .xml file should"
-" be passed to :command:`<PARAM_MODEL_XML>`."
-msgstr "`<output_dir>` 是存储中间表示 (IR) 的目录。IR 包含与网络结构对应的 .xml 格式以及与权重对应的 .bin 格式。此 .xml 文件应传递给 :command:`<PARAM_MODEL_XML>`。"
-
-
-#: ../../guides/stacks/greengrass.rst:172
-msgid ""
-"In the BVLC AlexNet model, the prototxt defines the input shape with "
-"batch size 10 by default. In order to use any other batch size, the "
-"entire input shape must be provided as an argument to the model "
-"optimizer. For example, to use batch size 1, you must provide: "
-"`--input_shape [1,3,227,227]`"
-msgstr "在 BVLC AlexNet 模型中，默认情况下，prototxt 会定义批处理大小为 10 的输入形状。要使用任何其他批处理大小，必须将整个输入形状作为参数提供给模型优化器。例如，要使用批处理大小 1，您必须提供 `--input_shape [1,3,227,227]`"
-
-
-#: ../../guides/stacks/greengrass.rst:180
-msgid "Configure AWS Greengrass group"
-msgstr "配置 AWS Greengrass 组"
-
-#: ../../guides/stacks/greengrass.rst:182
-msgid ""
-"For each Intel® edge platform, you must create a new AWS Greengrass group"
-" and install AWS Greengrass core software to establish the connection "
-"between cloud and edge."
-msgstr "对于每个英特尔®边缘平台，您必须创建一个新的 AWS Greengrass 组，并安装 AWS Greengrass 核心软件，以在云和边缘之间建立连接。"
-
-
-#: ../../guides/stacks/greengrass.rst:186
-msgid ""
-"To create an AWS Greengrass group, follow the instructions in `Configure "
-"AWS IoT Greengrass on AWS IoT`_."
-msgstr "要创建 AWS Greengrass 组，请按照 `Configure AWS IoT Greengrass on AWS IoT`_ 中的说明执行操作。"
-
-
-#: ../../guides/stacks/greengrass.rst:189
-msgid ""
-"To install and configure AWS Greengrass core on edge platform, follow the"
-" instructions in `Start AWS Greengrass on the Core Device`_. In step "
-"8(b), download the x86_64 Ubuntu\\* configuration of the AWS Greengrass "
-"core software."
-msgstr "要在边缘平台上安装和配置 AWS Greengrass 核心，请按照 `Start AWS Greengrass on the Core Device`_ 中的说明执行操作。在步骤 8(b) 中，下载 AWS Greengrass 核心软件的 x86_64 Ubuntu\\* 配置。"
-
-
-#: ../../guides/stacks/greengrass.rst:196
-msgid ""
-"You do not need to run the :file:`cgroupfs-mount.sh` script in step #6 of"
-" Module 1 of the `AWS Greengrass Developer Guide`_ because this is "
-"enabled already in |CL|."
-msgstr "您不需要在 `AWS Greengrass developer guide`_ 模块 1 的步骤 6 中运行 :file:`cgroupfs-mount.sh` 脚本，因为它已经在 |CL| 中启用。"
-
-
-#: ../../guides/stacks/greengrass.rst:200
-msgid ""
-"Be sure to download both the security resources and the AWS Greengrass "
-"core software."
-msgstr "请务必下载安全资源和 AWS Greengrass 核心软件。"
-
-#: ../../guides/stacks/greengrass.rst:205
-msgid "Security certificates are linked to your AWS account."
-msgstr "安全证书会链接到您的 AWS 帐户。"
-
-#: ../../guides/stacks/greengrass.rst:209
-msgid "Create and package Lambda function"
-msgstr "创建并打包 Lambda 函数"
-
-#: ../../guides/stacks/greengrass.rst:211
-msgid ""
-"Complete steps 1-4 of the AWS Greengrass guide at `Create and Package a "
-"Lambda Function`_."
-msgstr "在 `Create and Package a Lambda Function`_ 中完成 AWS Greengrass 教程的步骤 1-4。"
-
-#: ../../guides/stacks/greengrass.rst:216
-msgid ""
-"This creates the tarball needed to create the AWS Greengrass environment "
-"on the edge device."
-msgstr "这会创建必要的 tarball，以便在边缘设备上创建 AWS Greengrass 环境。"
-
-#: ../../guides/stacks/greengrass.rst:220
-msgid ""
-"In step 5, replace :file:`greengrassHelloWorld.py` with the "
-"classification or object detection Greengrass sample from `Edge-"
-"Analytics-Faas`_:"
-msgstr "在步骤 5 中，将 :file:`greengrassHelloWorld.py` 替换为 `Edge-Analytics-Faas`_ 中的分类或对象检测 Greengrass 示例："
-
-
-#: ../../guides/stacks/greengrass.rst:223
-msgid "Classification: `greengrass_classification_sample.py`_"
-msgstr "分类：`greengrass_classification_sample.py`_"
-
-#: ../../guides/stacks/greengrass.rst:225
-msgid "Object Detection: `greengrass_object_detection_sample_ssd.py`_"
-msgstr "对象检测：`greengrass_object_detection_sample_ssd.py`_"
-
-#: ../../guides/stacks/greengrass.rst:227
-msgid ""
-"Zip the selected Greengrass sample with the extracted Greengrass SDK "
-"folders from the previous step into "
-":file:`greengrass_sample_python_lambda.zip`."
-msgstr "将所选的 Greengrass 示例以及从上一步提取的 Greengrass SDK 文件夹压缩到 :file:`greengrass_sample_python_lambda.zip`。"
-
-
-#: ../../guides/stacks/greengrass.rst:230
-msgid "The zip should contain:"
-msgstr "压缩包应包含："
-
-#: ../../guides/stacks/greengrass.rst:232
-msgid "greengrasssdk"
-msgstr "greengrasssdk"
-
-#: ../../guides/stacks/greengrass.rst:234
-msgid "greengrass classification or object detection sample"
-msgstr "greengrass 分类或对象检测示例"
-
-#: ../../guides/stacks/greengrass.rst:236
-msgid "For example:"
-msgstr "例如："
-
-#: ../../guides/stacks/greengrass.rst:243
-msgid ""
-"Return to the AWS documentation section called `Create and Package a "
-"Lambda Function`_ and complete the procedure."
-msgstr "返回名为 `Create and Package a Lambda Function`_ 的 AWS 文档部分，并完成步骤。"
-
-#: ../../guides/stacks/greengrass.rst:248
-msgid ""
-"In step 9(a) of the AWS documentation, while uploading the zip file, make"
-" sure to name the handler to one of the following, depending on the AWS "
-"Greengrass sample you are using:"
-msgstr "在 AWS 文档的步骤 9(a) 中，上传 Zip 文件，并确保根据使用的 AWS Greengrass 示例将处理程序命名为以下名称之一："
-
-#: ../../guides/stacks/greengrass.rst:252
-msgid "greengrass_object_detection_sample_ssd.function_handler"
-msgstr "greengrass_object_detection_sample_ssd.function_handler"
-
-#: ../../guides/stacks/greengrass.rst:253
-msgid "greengrass_classification_sample.function_handler"
-msgstr "greengrass_classification_sample.function_handler"
-
-#: ../../guides/stacks/greengrass.rst:257
-msgid "Configure Lambda function"
-msgstr "配置 Lambda 函数"
-
-#: ../../guides/stacks/greengrass.rst:259
-msgid ""
-"After creating the Greengrass group and the Lambda function, start "
-"configuring the Lambda function for AWS Greengrass."
-msgstr "创建 Greengrass 组和 Lambda 函数后，开始为 AWS Greengrass 配置 Lambda 函数。"
-
-#: ../../guides/stacks/greengrass.rst:262
-msgid ""
-"Follow steps 1-8 in `Configure the Lambda Function for AWS IoT "
-"Greengrass`_ in the AWS documentation."
-msgstr "按照 AWS 文档中 `Configure the Lambda Function for AWS IoT Greengrass`_ 中的步骤 1-8 执行操作。"
-
-
-#: ../../guides/stacks/greengrass.rst:265
-msgid ""
-"In addition to the details mentioned in step 8, change the Memory limit "
-"to 2048 MB to accommodate large input video streams."
-msgstr "除了步骤 8 中提到的细节之外，将内存限制更改为 2048 MB，以容纳较大的输入视频流。"
-
-#: ../../guides/stacks/greengrass.rst:268
-msgid ""
-"Add the following environment variables as key-value pairs when editing "
-"the Lambda configuration and click on update:"
-msgstr "编辑 Lambda 配置时，添加以下环境变量作为键值对，然后点击更新："
-
-#: ../../guides/stacks/greengrass.rst:271
-msgid "**Table 1.  Environment variables: Lambda configuration**"
-msgstr "**表 1.环境变量：Lambda 配置**"
-
-#: ../../guides/stacks/greengrass.rst:275
-msgid "Key"
-msgstr "键"
-
-#: ../../guides/stacks/greengrass.rst:276
-msgid "Value"
-msgstr "值"
-
-#: ../../guides/stacks/greengrass.rst:277
-msgid "PARAM_MODEL_XML"
-msgstr "PARAM_MODEL_XML"
-
-#: ../../guides/stacks/greengrass.rst:278
-msgid ""
-"<MODEL_DIR>/<IR.xml>, where <MODEL_DIR> is user specified and contains "
-"IR.xml, the Intermediate Representation file from Intel® Model Optimizer."
-" For this guide, <MODEL_DIR> should be set to "
-"'/usr/share/openvino/models' or one of its subdirectories."
-msgstr "<MODEL_DIR>/<IR.xml>，其中 <MODEL_DIR> 是用户指定的，包含来自英特尔®模型优化器的中间表示文件 IR.xml。在本教程中，<MODEL_DIR> 应设置为 '/usr/share/openvino/models' 或其某个子目录。"
-
-
-#: ../../guides/stacks/greengrass.rst:282
-msgid "PARAM_INPUT_SOURCE"
-msgstr "PARAM_INPUT_SOURCE"
-
-#: ../../guides/stacks/greengrass.rst:283
-msgid "<DATA_DIR>/input.webm to be specified by user. Holds both input and"
-msgstr "<DATA_DIR>由用户指定的 /input.webm。保存输入和"
-
-#: ../../guides/stacks/greengrass.rst:284
-msgid "output data. For webcam, set PARAM_INPUT_SOURCE to ‘/dev/video0’"
-msgstr "输出数据。对于网络摄像头，请将 PARAM_INPUT_SOURCE 设置为 ‘/dev/video0’"
-
-#: ../../guides/stacks/greengrass.rst:285
-msgid "PARAM_DEVICE"
-msgstr "PARAM_DEVICE"
-
-#: ../../guides/stacks/greengrass.rst:286
-msgid "\"CPU\""
-msgstr "\"CPU\""
-
-#: ../../guides/stacks/greengrass.rst:287
-msgid "PARAM_CPU_EXTENSION_PATH"
-msgstr "PARAM_CPU_EXTENSION_PATH"
-
-#: ../../guides/stacks/greengrass.rst:288
-msgid "/usr/lib64/libcpu_extension.so"
-msgstr "/usr/lib64/libcpu_extension.so"
-
-#: ../../guides/stacks/greengrass.rst:289
-msgid "PARAM_OUTPUT_DIRECTORY"
-msgstr "PARAM_OUTPUT_DIRECTORY"
-
-#: ../../guides/stacks/greengrass.rst:290
-msgid "<DATA_DIR> to be specified by user. Holds both input and output data"
-msgstr "<DATA_DIR> 由用户指定。保存输入和输出数据"
-
-#: ../../guides/stacks/greengrass.rst:292
-msgid "PARAM_NUM_TOP_RESULTS"
-msgstr "PARAM_NUM_TOP_RESULTS"
-
-#: ../../guides/stacks/greengrass.rst:293
-msgid ""
-"User specified for classification sample. (e.g. 1 for top-1 result, 5 for"
-" top-5 results)"
-msgstr "为分类示例指定的用户。（例如，1 为 前 1 名结果，5 为前 5 名结果）"
-
-#: ../../guides/stacks/greengrass.rst:296
-msgid ""
-"Add subscription to subscribe, or publish messages from AWS Greengrass "
-"Lambda function by completing the procedure in `Configure the Lambda "
-"Function for AWS IoT Greengrass`_."
-msgstr "完成 `Configure the Lambda Function for AWS IoT Greengrass`_ 中的步骤，添加订阅以进行订阅或发布来自 AWS Greengrass Lambda 函数的消息。"
-
-
-#: ../../guides/stacks/greengrass.rst:301
-msgid ""
-"The optional topic filter field is the topic mentioned inside the Lambda "
-"function. In this guide, sample topics include the following: "
-":command:`openvino/ssd` or :command:`openvino/classification`"
-msgstr "可选主题过滤器字段是 Lambda 函数中提到的主题。在本教程中，示例主题包括以下 :command:`openvino/ssd` 或 :command:`openvino/classification`"
-
-
-#: ../../guides/stacks/greengrass.rst:305
-msgid "Add local resources"
-msgstr "添加本地资源"
-
-#: ../../guides/stacks/greengrass.rst:307
-msgid ""
-"Refer to the AWS documentation `Access Local Resources with Lambda "
-"Functions and Connectors`_ for details about local resources and access "
-"privileges."
-msgstr "有关 `Access Local Resources with Lambda Functions and Connectors`_ 的详细信息，请参阅 AWS 文档。"
-
-#: ../../guides/stacks/greengrass.rst:310
-msgid "The following table describes the local resources needed for the CPU:"
-msgstr "下表列出了 CPU 所需的本地资源："
-
-#: ../../guides/stacks/greengrass.rst:312
-msgid "**Local resources**"
-msgstr "**本地资源**"
-
-#: ../../guides/stacks/greengrass.rst:316
-msgid "Name"
-msgstr "名称"
-
-#: ../../guides/stacks/greengrass.rst:317
-msgid "Resource type"
-msgstr "资源类型"
-
-#: ../../guides/stacks/greengrass.rst:318
-msgid "Local path"
-msgstr "本地路径"
-
-#: ../../guides/stacks/greengrass.rst:319
-msgid "Access"
-msgstr "访问"
-
-#: ../../guides/stacks/greengrass.rst:321
-msgid "ModelDir"
-msgstr "ModelDir"
-
-#: ../../guides/stacks/greengrass.rst:322
-#: ../../guides/stacks/greengrass.rst:332
-msgid "Volume"
-msgstr "卷"
-
-#: ../../guides/stacks/greengrass.rst:323
-msgid "<MODEL_DIR> to be specified by user"
-msgstr "<MODEL_DIR> 由用户指定"
-
-#: ../../guides/stacks/greengrass.rst:324
-#: ../../guides/stacks/greengrass.rst:329
-msgid "Read-Only"
-msgstr "只读"
-
-#: ../../guides/stacks/greengrass.rst:326
-msgid "Webcam"
-msgstr "网络摄像头"
-
-#: ../../guides/stacks/greengrass.rst:327
-msgid "Device"
-msgstr "设备"
-
-#: ../../guides/stacks/greengrass.rst:328
-msgid "/dev/video0"
-msgstr "/dev/video0"
-
-#: ../../guides/stacks/greengrass.rst:331
-msgid "DataDir"
-msgstr "DataDir"
-
-#: ../../guides/stacks/greengrass.rst:333
-msgid "<DATA_DIR> to be specified by user. Holds both input and output data."
-msgstr "<DATA_DIR> 由用户指定。保存输入和输出数据。"
-
-#: ../../guides/stacks/greengrass.rst:335
-msgid "Read and Write"
-msgstr "读取和写入"
-
-#: ../../guides/stacks/greengrass.rst:338
-msgid "Deploy Lambda function"
-msgstr "部署 Lambda 函数"
-
-#: ../../guides/stacks/greengrass.rst:340
-msgid ""
-"Refer to the AWS documentation `Deploy Cloud Configurations to an AWS IoT"
-" Greengrass Core Device`_ for instructions on how to deploy the lambda "
-"function to AWS Greengrass core device. Select *Deployments* on the group"
-" page and follow the instructions."
-msgstr "有关如何 `Deploy Cloud Configurations to an AWS IoT Greengrass Core Device`_ 的说明，请参阅 AWS 文档。在组页面上选择 *Deployments*，并按照说明执行操作。"
-
-
-#: ../../guides/stacks/greengrass.rst:344
-msgid "Output consumption"
-msgstr "输出的使用"
-
-#: ../../guides/stacks/greengrass.rst:346
-msgid ""
-"There are four options available for output consumption. These options "
-"are used to report, stream, upload, or store inference output at an "
-"interval defined by the variable :command:`reporting_interval` in the AWS"
-" Greengrass samples."
-msgstr "使用输出时有四种可用选项。这些选项用于按 AWS Greengrass 示例中 :command:`reporting_interval` 变量定义的间隔，报告、流式传输、上传或存储推理输出。"
-
-
-#: ../../guides/stacks/greengrass.rst:350
-msgid "IoT cloud output:"
-msgstr "IoT 云输出："
-
-#: ../../guides/stacks/greengrass.rst:352
-msgid ""
-"This option is enabled by default in the AWS Greengrass samples using the"
-" :command:`enable_iot_cloud_output` variable. You can use it to verify "
-"the lambda running on the edge device. It enables publishing messages to "
-"IoT cloud using the subscription topic specified in the lambda. (For "
-"example, topics may include :command:`openvino/classification` for "
-"classification and :command:`openvino/ssd` for object detection samples.)"
-" For classification, top-1 result with class label are published to IoT "
-"cloud. For SSD object detection, detection results such as bounding box "
-"coordinates of objects, class label, and class confidence are published."
-msgstr "在 AWS Greengrass 示例中，默认情况下使用 :command:`enable_iot_cloud_output` 变量启用此选项。您可以使用它来验证在边缘设备上运行的 lambda。它支持使用 lambda 中指定的订阅主题向 IoT 云发布消息。（例如，主题可能包括用于分类示例的 :command:`openvino/classification` 以及用于对象检测示例的 :command:`openvino/ssd`。） 对于分类，具有类标签的前 1 名结果会发布到 IoT 云。对于 SSD 对象检测，则发布对象的边界框坐标、类标签和类置信度等检测结果。"
-
-
-#: ../../guides/stacks/greengrass.rst:362
-msgid ""
-"Refer to the AWS documentation `Verify the Lambda Function Is Running on "
-"the Device`_ for instructions on how to view the output on IoT cloud."
-msgstr "有关如何在 IoT 云上查看输出的说明，请参考 AWS 文档 `Verify the Lambda Function Is Running on the Device`_。"
-
-#: ../../guides/stacks/greengrass.rst:366
-msgid "Kinesis streaming:"
-msgstr "Kinesis 流式传输："
-
-#: ../../guides/stacks/greengrass.rst:368
-msgid ""
-"This option enables inference output to be streamed from the edge device "
-"to cloud using Kinesis [3] streams when :command:`enable_kinesis_output` "
-"is set to True. The edge devices act as data producers and continually "
-"push processed data to the cloud. You must set up and specify Kinesis "
-"stream name, Kinesis shard, and AWS region in the AWS Greengrass samples."
-msgstr ":command:`enable_kinesis_output` 设置为 True 时，此选项支持使用 Kinesis [3] 流将推理输出从边缘设备流式传输到云。边缘设备充当数据生产者，并将处理后的数据不断推送到云中。您必须在 AWS Greengrass 示例中设置和指定 Kinesis 流名称、Kinesis shard 和 AWS 区域。"
-
-
-#: ../../guides/stacks/greengrass.rst:375
-msgid "Cloud storage using AWS S3 bucket:"
-msgstr "使用 AWS S3 存储桶的云存储："
-
-#: ../../guides/stacks/greengrass.rst:377
-msgid ""
-"When the :command:`enable_s3_jpeg_output` variable is set to True, it "
-"enables uploading and storing processed frames (in jpeg format) in an AWS"
-" S3 bucket. You must set up and specify the S3 bucket name in the AWS "
-"Greengrass samples to store the JPEG images. The images are named using "
-"the timestamp and uploaded to S3."
-msgstr "将 :command:`enable_s3_jpeg_output` 变量设置为 True 时，它允许在 AWS S3 存储桶中上传和存储已处理的帧（jpeg 格式）。您必须在 AWS Greengrass 示例中设置和指定用来存储 JPEG 图像的 S3 存储桶名称。这些映像使用时间戳命名，并上传到 S3。"
-
-
-#: ../../guides/stacks/greengrass.rst:383
-msgid "Local storage:"
-msgstr "本地存储："
-
-#: ../../guides/stacks/greengrass.rst:385
-msgid ""
-"When the :command:`enable_s3_jpeg_output` variable is set to True, it "
-"enables storing processed frames (in jpeg format) on the edge device. The"
-" images are named using the timestamp and stored in a directory specified"
-" by :command:`PARAM_OUTPUT_DIRECTORY`."
-msgstr "将 :command:`enable_s3_jpeg_output` 变量设置为 True 时，它允许在边缘设备上存储已处理的帧（jpeg 格式）。这些映像使用时间戳命名，并存储在由 :command:`PARAM_OUTPUT_DIRECTORY` 指定的目录中。"
-
-
-#: ../../guides/stacks/greengrass.rst:391
-msgid "References"
-msgstr "参考"
-
-#: ../../guides/stacks/greengrass.rst:393
-msgid "AWS Greengrass: https://aws.amazon.com/greengrass/"
-msgstr "AWS Greengrass：https://aws.amazon.com/greengrass/"
-
-#: ../../guides/stacks/greengrass.rst:394
-msgid "AWS Lambda: https://aws.amazon.com/lambda/"
-msgstr "AWS Lambda：https://aws.amazon.com/lambda/"
-
-#: ../../guides/stacks/greengrass.rst:395
-msgid "AWS Kinesis: https://aws.amazon.com/kinesis/"
-msgstr "AWS Kinesis：https://aws.amazon.com/kinesis/"
-
-#~ msgid "This tutorial demonstrates how to:"
-#~ msgstr "本教程演示了如何："
-
-#~ msgid "Refer to the following topics:"
-#~ msgstr "请参阅以下主题："
-
-#~ msgid ""
-#~ "Follow these instructions for `converting "
-#~ "deep learning models to Intermediate "
-#~ "Representation using Model Optimizer`_. To "
-#~ "optimize either of the sample models "
-#~ "described above, run one of the "
-#~ "following commands."
-#~ msgstr ""
-#~ "按照 `converting deep learning models to"
-#~ " Intermediate Representation using Model "
-#~ "Optimizer`_ 中的说明执行操作。要优化上述任一示例模型，请运行以下命令之一。"
-
-#~ msgid "Follow the instructions here to `view the output on IoT cloud`_."
-#~ msgstr "按照这里的说明`view the output on IoT cloud`_。"
-
-

locale/zh_CN/LC_MESSAGES/index.po

@@ -64,7 +64,7 @@ msgstr ":ref:`guides`"
 #: ../../index.rst:31
 msgid ""
 "Guides cover a range of topics from |CL| features and tooling, to system "
-"maintenance, network, and stacks."
+"maintenance, and network."
 msgstr "指南页面涵盖了从 |CL| 功能和工具到系统维护、网络和堆栈的一系列主题。"
 
 #: ../../index.rst:34

make.bat

@@ -5,7 +5,9 @@ REM Command file for Sphinx documentation
 if "%SPHINXBUILD%" == "" (
 	set SPHINXBUILD=sphinx-build
 )
-set BUILDDIR=build
+
+set SCRIPTDIR=source\_scripts\_python
+set BUILDDIR=source\_build
 set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source
 set I18NSPHINXOPTS=%SPHINXOPTS% source
 if NOT "%PAPER%" == "" (
@@ -30,6 +32,7 @@ if "%1" == "help" (
 	echo.  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter
 	echo.  text       to make text files
 	echo.  man        to make manual pages
+	echo.  clean-man  to clean up after man page generation
 	echo.  texinfo    to make Texinfo files
 	echo.  gettext    to make PO message catalogs
 	echo.  changes    to make an overview over all changed/added/deprecated items
@@ -77,6 +80,17 @@ if "%1" == "html" (
 	if errorlevel 1 exit /b 1
 	echo.
 	echo.Build finished. The HTML pages are in %BUILDDIR%/html.
+	copy source\_scripts\js\copybutton.js %BUILDDIR%\html\_static
+	goto end
+)
+
+if "%1" == "py" (
+	cd %SCRIPTDIR%
+	python.exe bundle_lister.py
+	copy bundles.html.txt ..\..\reference\bundles
+	for /d %%i in (cloned_repo\*) do rmdir /q /s %%i
+	del /q /s bundles.html.txt
+	echo "Python bundle script finished successfully!"
 	goto end
 )
 
@@ -186,10 +200,14 @@ if "%1" == "text" (
 )
 
 if "%1" == "man" (
-	%SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man
-	if errorlevel 1 exit /b 1
-	echo.
-	echo.Build finished. The manual pages are in %BUILDDIR%/man.
+    cd source/_scripts/_python/manpages
+    man.bat man
+	goto end
+)
+
+if "%1" == "clean-man" (
+	cd source/_scripts/_python/manpages
+	man.bat clean-man
 	goto end
 )
 

requirements.txt

@@ -1,9 +1,10 @@
 breathe==4.9.1
-sphinx==1.8
+sphinx==2.2.0
 docutils==0.14
 sphinx_rtd_theme
 sphinx-intl==2.0.0
 sphinx-sitemap==1.0.2
-Jinja2==2.10.1
+Jinja2==2.11.3
 GitPython==3.0.8
 sphinx-tabs
+sphinx-copybutton

setup.py

@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+
+from distutils.core import setup
+
+setup(name='Clear Linux Documentation',
+      version='',
+      description='Sphinx build of Clear Linux documentation',
+      author='Many',
+      author_email='kevin.putnam@intel.com',
+      url='https://github.com/clearlinux/clear-linux-documentation/',
+      )

source/FAQ/index.rst

@@ -198,6 +198,8 @@ Visual Studio Code
 
 |
 
+.. _licensing_restrict:
+
 Is FFmpeg available?
 ====================
 
@@ -224,8 +226,8 @@ ZFS is not available with |CL| because of copyright and licensing
 complexities. BTRFS is an alternative filesystem that is available in |CL|
 natively.
 
-A user on GitHub notes that the `ZFS kernel module can be compiled, built, and
-installed manually <https://github.com/clearlinux/distribution/issues/631>`_.
+A community contributed tutorial has been shared on how to :ref:`manually
+install ZFS <zfs>`.
 
 |
 

source/Makefile

@@ -50,6 +50,8 @@ help:
 	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
 	@echo "  coverage   to run coverage check of the documentation (if enabled)"
 	@echo "  py         to trigger an update of bundle content"
+	@echo "  man        to pull and create manpage rst files in reference section"
+	@echo "  clean-man  to clean up manpage generation"
 
 clean:
 	rm -rf $(BUILDDIR)/*
@@ -71,6 +73,7 @@ htmlzh:
 
 html:
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	cp _scripts/js/copybutton.js $(BUILDDIR)/html/_static
 	@echo
 	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
 
@@ -210,3 +213,9 @@ pseudoxml:
 	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
 	@echo
 	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+
+man:
+	$(MAKE) -C $(SCRIPTDIR)/manpages man
+
+clean-man:
+	$(MAKE) -C $(SCRIPTDIR)/manpages clean-man

source/_scripts/_python/manpages/Makefile

@@ -0,0 +1,25 @@
+man:
+	git clone https://github.com/clearlinux/clr-man-pages.git
+	git clone https://github.com/clearlinux/clr-power-tweaks.git
+	git clone https://github.com/clearlinux/clrtrust.git
+	git clone https://github.com/clearlinux/mixer-tools.git
+	git clone https://github.com/clearlinux/swupd-client.git
+	git clone https://github.com/clearlinux/telemetrics-client.git
+	git clone https://github.com/clearlinux/tallow.git
+	git clone https://github.com/clearlinux/micro-config-drive.git
+	python3 manpages.py
+	- mkdir ../../../reference/manpages
+	cp *.rst ../../../reference/manpages
+
+clean-man:
+	- rm -rf clr-man-pages
+	- rm -rf clr-power-tweaks
+	- rm -rf clrtrust
+	- rm -rf mixer-tools
+	- rm -rf swupd-client
+	- rm -rf telemetrics-client
+	- rm -rf tallow
+	- rm -rf micro-config-drive
+	- rm *.rst
+	- rm ../../../reference/manpages/*.rst
+	- rm ../../../reference/man-pages.rst

source/_scripts/_python/manpages/man.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+REM Command file for building man pages
+
+if "%1" == "man" (
+    git clone https://github.com/clearlinux/clr-man-pages.git
+    git clone https://github.com/clearlinux/clr-power-tweaks.git
+    git clone https://github.com/clearlinux/clrtrust.git
+    git clone https://github.com/clearlinux/mixer-tools.git
+    git clone https://github.com/clearlinux/swupd-client.git
+    git clone https://github.com/clearlinux/telemetrics-client.git
+    git clone https://github.com/clearlinux/tallow.git
+    git clone https://github.com/clearlinux/micro-config-drive.git
+    python.exe manpages.py
+    mkdir ..\..\..\reference\manpages
+    copy *.rst ..\..\..\reference\manpages
+    goto end
+)
+
+if "%1" == "clean-man" (
+    rmdir /q /s clr-man-pages
+    rmdir /q /s clr-power-tweaks
+    rmdir /q /s clrtrust
+    rmdir /q /s mixer-tools
+    rmdir /q /s swupd-client
+    rmdir /q /s telemetrics-client
+    rmdir /q /s tallow
+    rmdir /q /s micro-config-drive
+    del *.rst
+    del ..\..\..\reference\manpages\*.rst
+    del ..\..\..\reference\man-pages.rst
+    goto end
+)
+
+:end

source/_scripts/_python/manpages/manpages.py

@@ -0,0 +1,196 @@
+#
+# manpages.py
+#
+# maintainer: intelkevinputnam
+#
+# usage: python3 manpages.py
+#
+# dependencies: 1. clone of Clear Linux documentation https://github.com/clearlinux/clear-linux-documentation
+#               2. Makefile or make.bat to create directories and clone repositories (``make man`` to generate. ``make clean-man`` to clean up.)
+#
+# This script does 4 things:
+#
+# 1. Find and move reStructuredText versions of man pages (looks for manName.sectionNumber.fileExtension: mixer.1.rst) to reference/manpages directory of Clear Linux docs.
+# 2. Massage markdown and reStructuredText man pages into a normalized format.
+# 3. Add cross reference links wherever man pages reference each other.
+# 4. Create manpages.rst in reference directory (already included in toctree of reference/index.rst)
+#
+#
+
+import sys
+import re
+import subprocess
+from os import listdir
+from os.path import join, isfile
+from shutil import copyfile
+
+paths = ["clr-man-pages","clr-power-tweaks/man","clrtrust/man","micro-config-drive/docs/","mixer-tools/docs","swupd-client/docs","tallow/man","telemetrics-client/docs/man"]
+pathToRefSection = "../../../reference/"
+manPageRegex = '.[0-9]+.rst'
+mdManPageRegex = '.[0-9]+.md'
+mdBoldItalicRegex = '\\*\\*\\`[a-z-._]+\\`\\*\\*'
+manFiles = []
+manGroups = {}
+manNamePerms = {}
+seeAlsoRefs = {}
+gitHubGroup = "https://github.com/clearlinux/"
+TOC = "\n.. contents::\n    :local:\n"
+manPageRest = """.. _man-pages:
+
+Man pages
+#########
+
+These pages are generated from `Clear Linux\\* tool
+repositories <https://github.com/clearlinux>`__. Updated |today|.
+
+
+"""
+manTOC = """.. toctree::
+   :maxdepth: 1
+
+"""
+
+def getPages(paths):
+    for path in paths:
+        manGroups[path] = []
+        files = listdir(path)
+        for file in files:
+            if file.endswith(".rst"):
+                regex_found = re.search(manPageRegex,file)
+                if regex_found:
+                    filePath = join(path,file)
+                    copyfile(filePath,file)
+                    manFiles.append(file)
+                    manGroups[path].append(file)
+                    addManNamePermutations(file)
+            elif file.endswith(".md"):
+                regex_found = re.search(mdManPageRegex,file)
+                if regex_found:
+                    filePath = join(path,file)
+                    rstFileName = processMDFile(file,filePath)
+                    manFiles.append(rstFileName)
+                    manGroups[path].append(rstFileName)
+                    addManNamePermutations(rstFileName)
+
+def processMDFile(fileName,filePath):
+    manName = fileName.rstrip('.md')
+    manSection = manName.split('.')[-1]
+    manName = manName.rstrip('.' + manSection)
+    lines = []
+    headerInsert = ""
+    with open(filePath,'r') as m:
+        lines = m.readlines()
+        if "SYNOPSIS" in lines[0]:
+                header='='*len(manName)
+                lines.insert(0,"# " + manName + "\n\n:Manual section: "+manSection+"\n\n")
+        index = 0
+        first = True
+        for line in lines:
+            newLine = line.replace("**`","`") #Fix some markdown formatting weirdness that doesn't translate to reST
+            newLine = newLine.replace("`**","`")
+            if newLine.startswith("#") and not newLine.startswith("##"): #Fix if all headers are first level
+                if first:
+                    first = False
+                else:
+                    newLine = newLine.replace("#","##")
+            if "===" in newLine: #fix rst style header that is actually description in some docs
+                newLine = ""
+                headerInsert = manName + "\n" + '='*len(manName) + "\n\n"
+            lines[index] = newLine
+            index += 1
+        lines.insert(0,headerInsert)
+
+    with open(filePath,'w') as md:
+        md.writelines(lines)
+        rstFilePath = filePath.replace(".md",".rst")
+        rstFileName = fileName.replace(".md",".rst")
+
+    command = "pandoc " +  filePath + " -o " + rstFilePath
+    subprocess.run(command, shell=True)
+    if isfile(rstFilePath):
+        copyfile(rstFilePath,rstFileName)
+    return rstFileName
+
+def addManNamePermutations(fileName): #Based on all the syntactical permutations of references to man pages in the documents.
+    (manName,subsection) = getNameAndSubsection(fileName)
+    manNames = []
+    manNames.append("``" + manName + "``\\(" + subsection +")") # ``mixer.init``\(1)
+    manNames.append("``" + manName + "``\\ (" + subsection +")") # ``mixer.init``\ (1)
+    manNames.append("**``" + manName + "(" + subsection + ")``**")  # **`mixer.init(1)`**
+    manNames.append("``" + manName + "(" + subsection + ")``") # ``mixer.init(1)``
+    manNames.append("**" + manName + "(" + subsection + ")**") # **mixer.init(1)**
+    manNames.append("`" + manName + "(" + subsection + ")`") # `mixer.init(1)`
+    manNamePerms[(manName,subsection)] = manNames
+
+def getNameAndSubsection(manFileName):
+    manName = manFileName.rstrip('.rst')
+    manSection = manName.split('.')[-1]
+    manName = manName.rstrip('.'+manSection)
+    return (manName,manSection)
+
+def linkToMan(manName,manSection):
+    return "`" + manName + "(" + manSection + ") <" + manName + "." + manSection + ".html>`__"
+ 
+def buildManName(name,section):
+    return name + "(" + section + ")"
+
+def updateManPages():
+    #makeSeeAlsoReplacements()
+    for file in manFiles:
+        manFile = ""
+        with open(file,'r',encoding="utf8") as f:
+            manFile = f.read()
+            #manFile = addTOC(manFile,file) # Not convinced adding TOC adds value.
+            #
+            # Add linked cross referencing for all manpages discovered.
+            # 1. the manNamePerms dictionary is created once all of the man page source files are discovered
+            # 2. Each document is checked for each permutation. 
+            # 3. When a permuation is found it is replaced with a normalized version
+            # 4. Once normalized it is turned into a reStructuredText link (def linkToMan)
+            # 
+            for nameAndSection, listOfPerms in manNamePerms.items():
+                for perm in listOfPerms:
+                    manFile = manFile.replace(perm,buildManName(nameAndSection[0],nameAndSection[1]))
+            for manName, doNotUse in manNamePerms.items():
+                manFile = manFile.replace(buildManName(manName[0],manName[1]),linkToMan(manName[0],manName[1]))
+        with open(file,'w',encoding="utf8") as w:
+            w.write(manFile)
+
+def addTOC(manContent,file):
+# Find the first instance of the manpage Name
+# Skip the underline of the header
+# Add a new line
+# Add contents directive with local modifier
+# Add a new line
+    manSectionMeta = ":Manual section:"
+    manContentLines = manContent.split('\n')
+    index = 0
+    for line in manContentLines:
+        if manSectionMeta in line:
+            manContentLines.insert(index + 2,TOC)
+            break
+        index += 1
+    output = ""
+    for line in manContentLines:
+        output = output + line + "\n"
+    return output
+
+def createManpagesRST():
+    filePath = join(pathToRefSection,"man-pages.rst")
+    with open(filePath,'w') as f:
+        manGrouping = manPageRest
+        for path, fileList in manGroups.items():
+            repoName = path.split("/")[0]
+            repoLink = gitHubGroup + repoName
+            repoReST = "`" + repoName + " <" + repoLink + ">`__"
+            manGrouping += repoReST + "\n"
+            manGrouping += "="*len(repoReST) + "\n\n"
+            manGrouping += manTOC
+            for file in fileList:
+                manGrouping += "   manpages/" + file + "\n"
+            manGrouping += "\n"
+        f.write(manGrouping)
+
+getPages(paths)
+updateManPages()
+createManpagesRST()

source/_scripts/js/copybutton.js

@@ -0,0 +1,147 @@
+// introduces special behavior for ShellSession
+
+// Localization support
+const messages = {
+  'en': {
+    'copy': 'Copy',
+    'copy_to_clipboard': 'Copy to clipboard',
+    'copy_success': 'Copied!',
+    'copy_failure': 'Failed to copy',
+  },
+  'es' : {
+    'copy': 'Copiar',
+    'copy_to_clipboard': 'Copiar al portapapeles',
+    'copy_success': '¡Copiado!',
+    'copy_failure': 'Error al copiar',
+  },
+  'de' : {
+    'copy': 'Kopieren',
+    'copy_to_clipboard': 'In die Zwischenablage kopieren',
+    'copy_success': 'Kopiert!',
+    'copy_failure': 'Fehler beim Kopieren',
+  }
+}
+
+let locale = 'en'
+if( document.documentElement.lang !== undefined
+    && messages[document.documentElement.lang] !== undefined ) {
+  locale = document.documentElement.lang
+}
+
+/**
+ * Set up copy/paste for code blocks
+ */
+
+const runWhenDOMLoaded = cb => {
+  if (document.readyState != 'loading') {
+    cb()
+  } else if (document.addEventListener) {
+    document.addEventListener('DOMContentLoaded', cb)
+  } else {
+    document.attachEvent('onreadystatechange', function() {
+      if (document.readyState == 'complete') cb()
+    })
+  }
+}
+
+const codeCellId = index => `codecell${index}`
+
+// Clears selected text since ClipboardJS will select the text when copying
+const clearSelection = () => {
+  if (window.getSelection) {
+    window.getSelection().removeAllRanges()
+  } else if (document.selection) {
+    document.selection.empty()
+  }
+}
+
+// Changes tooltip text for two seconds, then changes it back
+const temporarilyChangeTooltip = (el, newText) => {
+  const oldText = el.getAttribute('data-tooltip')
+  el.setAttribute('data-tooltip', newText)
+  setTimeout(() => el.setAttribute('data-tooltip', oldText), 2000)
+}
+
+// Callback when a copy button is clicked. Will be passed the node that was clicked
+// should then grab the text and replace pieces of text that shouldn't be used in output
+var copyTargetText = (trigger) => {
+  var target = document.querySelector(trigger.attributes['data-clipboard-target'].value);
+  var textContent = target.innerText.split('\n');
+  var copybuttonPromptText = '$ '; // Inserted from config
+  var onlyCopyPromptLines = true; // Inserted from config
+  var removePrompts = true; // Inserted from config
+
+  grandParent = target.parentElement.parentElement;
+  blockType = grandParent.classList;
+  if (blockType[0].includes("ShellSession")) {
+    onlyCopyPromptLines = false;
+  }
+
+  // Text content line filtering based on prompts (if a prompt text is given)
+  if (copybuttonPromptText.length > 0) {
+    // If only copying prompt lines, remove all lines that don't start w/ prompt
+    if (onlyCopyPromptLines) {
+      linesWithPrompt = textContent.filter((line) => {
+        return line.startsWith(copybuttonPromptText) || (line.length == 0); // Keep newlines
+      });
+      // Check to make sure we have at least one non-empty line
+      var nonEmptyLines = linesWithPrompt.filter((line) => {return line.length > 0});
+      // If we detected lines w/ prompt, then overwrite textContent w/ those lines
+      if ((linesWithPrompt.length > 0) && (nonEmptyLines.length > 0)) {
+        textContent = linesWithPrompt;
+      }
+    }
+    // Remove the starting prompt from any remaining lines
+    if (removePrompts) {
+      textContent.forEach((line, index) => {
+        if (line.startsWith(copybuttonPromptText)) {
+          textContent[index] = line.slice(copybuttonPromptText.length);
+        }
+      });
+    }
+  }
+  textContent = textContent.join('\n');
+  // Remove a trailing newline to avoid auto-running when pasting
+  if (textContent.endsWith("\n")) {
+     textContent = textContent.slice(0, -1)
+  }
+  return textContent
+}
+
+const addCopyButtonToCodeCells = () => {
+  // If ClipboardJS hasn't loaded, wait a bit and try again. This
+  // happens because we load ClipboardJS asynchronously.
+  if (window.ClipboardJS === undefined) {
+    setTimeout(addCopyButtonToCodeCells, 250)
+    return
+  }
+
+  // Add copybuttons to all of our code cells
+  const codeCells = document.querySelectorAll('div.highlight pre')
+  codeCells.forEach((codeCell, index) => {
+    const id = codeCellId(index)
+    codeCell.setAttribute('id', id)
+    const pre_bg = getComputedStyle(codeCell).backgroundColor;
+
+    const clipboardButton = id =>
+    `<a class="copybtn o-tooltip--left" style="background-color: ${pre_bg}" data-tooltip="${messages[locale]['copy']}" data-clipboard-target="#${id}">
+      <img src="${DOCUMENTATION_OPTIONS.URL_ROOT}_static/copy-button.svg" alt="${messages[locale]['copy_to_clipboard']}">
+    </a>`
+    codeCell.insertAdjacentHTML('afterend', clipboardButton(id))
+  })
+
+  // Initialize with a callback so we can modify the text before copy
+  const clipboard = new ClipboardJS('.copybtn', {text: copyTargetText})
+
+  // Update UI with error/success messages
+  clipboard.on('success', event => {
+    clearSelection()
+    temporarilyChangeTooltip(event.trigger, messages[locale]['copy_success'])
+  })
+
+  clipboard.on('error', event => {
+    temporarilyChangeTooltip(event.trigger, messages[locale]['copy_failure'])
+  })
+}
+
+runWhenDOMLoaded(addCopyButtonToCodeCells)

source/_themes/otc_tcs_sphinx_theme/static/tcs_theme.css

@@ -306,9 +306,9 @@ button:hover a.headerlink:after {
 /*End support for custom Clear Linux header*/
 
 /*Adds a bit of spacing after the last paragraph in a bulleted list*/
-.wy-plain-list-disc li p:last-child, .rst-content .section ul li p:last-child, .rst-content .toctree-wrapper ul li p:last-child, article ul li p:last-child {
+/*.wy-plain-list-disc li p:last-child, .rst-content .section ul li p:last-child, .rst-content .toctree-wrapper ul li p:last-child, article ul li p:last-child {
     margin-bottom: 10px;
-}
+}*/
 
 div.admonition ul {
   margin-top: 20px;
@@ -368,6 +368,15 @@ div.highlight-python .highlight:before{
   white-space: pre;
 }
 
+div.highlight-ShellSession .highlight:before{
+  background: #909090;
+  color: white;
+  content: " Shell ";
+  font-family: SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",Courier,monospace;
+  font-size: 14px;
+  white-space: pre;
+}
+
 div.highlight-console .highlight:before{
   background: #909090;
   color: white;
@@ -494,6 +503,11 @@ div.linenodiv:before { /*add extra new line to make sure code and line numbers a
   background: white;
 }
 
+.column.narrow {
+  width: 300px;
+  height: 450px;
+}
+
 .column.featurecard {
   background: #CCE9F5;
   width: 300px;
@@ -511,6 +525,12 @@ div.linenodiv:before { /*add extra new line to make sure code and line numbers a
   clear: both;
 }
 
+.colh3 {
+  font-size: 125%;
+  font-weight: 700;
+  font-family: "Roboto Slab","ff-tisa-web-pro","Georgia",Arial,sans-serif;
+}
+
 .colh2 {
   font-size: 150%;
   font-weight: 700;

source/about.rst

@@ -3,57 +3,131 @@
 About
 #####
 
-The |CL| delivers a secure, hardware optimized OS. Its easy updates ensure that
-software dependencies remain mutually compatible.
-
-|CL| does this via custom infrastructure components and process innovations.
+|CL-ATTR| does things differently. Our software architecture provides a
+unique and innovative platform for Linux* developers focused on
+performance and security for compute, server, and the cloud.
 
 .. contents::
    :local:
    :depth: 1
 
+What is |CL|?
+*************
 
-For detailed information on these topics, refer to the :ref:`cl-guides` guides.
+|CL| is an open source, rolling-release Linux distribution, optimized for
+performance and security from the cloud to the Edge. Designed from the ground up,
+|CL| provides an industry blueprint on how to incorporate Intel® architecture
+features for a modern, modular Linux OS. |CL| is not based on any other Linux
+distro.
 
+What |CL| isn't?
+****************
+
+|CL| is not intended to be a general-purpose Linux distribution, suitable 
+for novice end-users. While we ship common applications, our purpose isn’t 
+to make an OS for routine desktop tasks and provide immunity from all 
+security threats in all situations. Our unique focus means that what we consider *essential* use cases, *optional* use cases, or even *unsupported* use cases, differs from other Linux distros. See our :ref:`target audience <target-audience>` below. 
+
+Is |CL| completely Open Source?
+*******************************
+
+|CL| aims to be completely open source. Our project `source code`_ and 
+`packages source code`_ are available on GitHub\*. When considering projects 
+for inclusion, we check that they are in active development and are well 
+maintained. We have a very strict requirement for not accepting proprietary 
+packages and non-open source components.  For example, many Linux distros 
+may not be able to include certain media codecs due to 
+:ref:`licensing restrictions <licensing_restrict>`, but manual installation and `third party alternatives`_ are available. 
+
+.. _target-audience:
+
+Who is the target audience?
+***************************
+
+|CL| mainly targets professionals in IT, DevOps, Cloud/Container deployments, and :abbr:`AI (Artificial Intelligence)`. 
+
+Rather than making a standard Linux distribution, the |CL| team decided to
+build a unique Linux distro. Developing a distro in house allows us to experiment and iterate faster, which means we continually optimize performance and deliver security patches, :ref:`several times per week <release-cadence>`. Yet our experiments are only valuable if our software architecture gives you the freedom to innovate, too. To improve manageability, |CL| employs a :ref:`stateless` design, separating user and system management.  
+
+We leverage the pool of knowledge and skills at Intel to drive improvements to |CL|.
+
+Intel has worked with the Linux community and other distros for many years.
+Understanding what it takes to integrate features in our own Linux distro
+helps us collaborate with other distro owners and submit enhancements to
+upstream. We demonstrate the value of our distro by offering users the same
+tools we use. For example, :ref:`mixer`, a tool unique to |CL|, allows users
+to build custom derivatives and act as their own :abbr:`OSV (Operating System
+Vendor)`. 
+
+For more details on |CL| features, visit our :ref:`cl-guides` guides.
+
+How does |CL| address security?
+*******************************
+
+Several :ref:`security features <security>` are designed to work 
+out-of-the-box, yet they're not intended to be intrusive. We focus on 
+*essential* use cases and ignore *unwanted* or *unsupported* use cases. 
+For example, while |CL| does not enable antivirus by default, we provide a 
+bundle for it (``clamav``). We leave antivirus configuration to our users.  
+In addition, firewalls are less important if the OS doesn’t expose services 
+to the outside by default. In |CL|, we enforce this strategy by disabling 
+network services by default - e.g. ``mariadb`` listens on a UNIX socket; 
+``nginx`` won’t listen at all; and other services similarly are restricted 
+from being accessed over the network. This strategy alone makes firewall 
+software much less urgent--there simply isn’t anything that a firewall could 
+easily block.
+
+What’s the thinking around Server vs. Desktop?
+**********************************************
+
+|CL| focuses on performance for server and cloud use-cases first because 
+many design decisions associated with them are applicable to other 
+use-cases, such as IoT and the desktop client. While our initial focus was 
+on the command line, we realized that many people valued the ease-of-use of 
+a desktop environment.  Whereas in the past we tried to accommodate those 
+interested in a desktop version, we were forced to confront clear limits as
+to how we could meet this need. |CL| minimizes the customizations and patches in support of the desktop and provides a generic GNOME implementation. Other window managers or desktops are available; however, testing in |CL| is focused on GNOME. 
+
+What makes |CL| different?
+**************************
+
+.. _release-cadence:
 
 Release Cadence
-***************
+===============
 
 |CL| updates are based on a rolling release that can occur daily, up to a few
-times per week. Each release has a unique version number that
-identifies every component in the OS from kernel, to driver, to tool, to GUI
-application. Most components are included in entities called *bundles*.
+times per week. Each release has a unique version number that identifies
+every component in the OS from kernel, to driver, to tool, to GUI
+application. Most components are included in entities called :ref:`bundles<bundles>`.
 
 Updates
-*******
+=======
 
-By default, |CL| automatically checks for updates, ensuring that the latest
+By default, |CL| automatically checks for updates, ensuring the latest
 performance and security fixes are installed as soon as they are available.
-:ref:`swupd-guide` is the custom tool designed to manage updates and bundles.
+|CL| stays in lockstep with upstream for current security upgrades and is 
+designed to rapidly deliver security mitigations to customers.
+:ref:`swupd-guide` is designed to manage updates and bundles.
 
-|CL| is :ref:`stateless` to make sure that system components can be updated
-without impacting user settings.
-
-
-Ease of Use
-***********
+Ease of Use 
+===========
 
 |CL| makes it easier to manage a number of difficult problems.
 
 * :ref:`autoproxy` makes it possible for |CL| tools to operate in some proxy
   environments without needing to be configured.
 
-* Being :ref:`stateless` means that configuration settings are easier to manage
+* :ref:`stateless` means that configuration settings are easier to manage
   and remain untouched when system software is updated.
 
-* :ref:`swupd-guide` simplifies managing software and maintaining compatibility.
+* :ref:`swupd-guide` simplifies managing software and maintaining
+  compatibility.
 
 Custom Derivatives
-******************
+==================
 
-The same tools used to build the |CL| are available *in* the OS. These tools can
-be used to create a custom distribution that continues to benefit from upstream
-rolling releases.
+The same tools used to build the |CL| are available *in* the OS. These tools can be used to create a custom distribution that continues to benefit from upstream rolling releases.
 
 .. figure:: /_figures/about/clear-lifecycle.png
    :scale: 75%
@@ -66,10 +140,7 @@ Create
 ======
 
 To create a custom distribution you need to understand how to use the
-:ref:`autospec` and :ref:`mixer` tools.
-
-Additional training materials are available in the `how-to-clear`_ GitHub\*
-project to help you get started with |CL| tools.
+:ref:`autospec` and :ref:`mixer` tools. Additional training materials are available in the `how-to-clear`_ GitHub project to help you get started with |CL| tools.
 
 Deploy
 ======
@@ -82,4 +153,159 @@ Administrate
 |CL| provides a :ref:`telem-guide` solution for collecting useful information
 about a deployment, as well as :ref:`debug` capabilities.
 
+Why create new components rather than modifying existing projects?
+******************************************************************
+
+One question that's often asked: “Why did you develop your own solution 
+instead of using <XYZ>?” (e.g. `swupd post`_).  We do evaluate existing 
+projects for inclusion in |CL|, yet there are cases where our unique 
+architecture and components would require too much customization to use 
+off-the-shelf projects.  In other situations, we may feel that using a new 
+language to develop the component would give us a performance advantage, 
+ease code development and maintenance, and grow the skills of our engineers 
+on new and upcoming programming languages.  And yes, sometimes there are 
+personal biases for and against some projects by the architects and 
+engineers.  We tend to move fast, and sometimes it’s easier to live with 
+suboptimal choices until we have the time or incentive to re-architect them 
+properly. 
+
+Which Components are used in Clear Linux?
+*****************************************
+
+.. list-table::
+   :widths: 33,33,33
+   :header-rows: 1
+
+   * - Component
+     - Enabled in OS/Bundle
+     - Optional
+
+   * - OS Installer
+     - `Clear Linux installer`_
+     - 
+
+   * - Bootloader
+     - `systemd-boot`_ (UEFI) / `syslinux`_ (Legacy)
+     - 
+
+   * - Boot Manager
+     - `Clear Linux Boot Manager`_
+     - 
+
+   * - Configuration initialization and management
+     - *NA*
+     - `micro-config-drive`_ (minimal cloud-init), Ansible
+
+   * - Software component installer, manager, updater
+     - `swupd`_
+     - 
+
+   * - Software bundle generator - 
+     - `mixer`_ and `Clear Linux Distro Factory`_
+     - 
+
+   * - Software package builder
+     - `autospec`_
+     - 
+
+   * - Software debugging
+     - *NA*
+     - `clr-debug-info`_ 
+
+   * - Unified TLS Trust Store Management
+     - `clrtrust`_
+     - 
+
+   * - System and software telemetry
+     - *NA*
+     - `Telemetrics`_ (disabled by default)
+
+   * - File system
+     - `EXT4`_ (default for rootfs), `VFAT`_, `EXT2 and EXT3`_, `F2FS`_
+     - 
+
+   * - Disk encryption
+     - *NA*
+     - `LUKS`_ 
+
+   * - System /Service manager  
+     - `systemd`_
+     - 
+
+   * - Display manager 
+     - `GNOME`_
+     - ``KDE``, ``Xfce``, ``lightdm``, ``sddm``  (see `Clear Linux store`_)
+
+   * - Display services (Desktop installed)
+     - `X.Org`_
+     - `Wayland`_ compositor
+
+   * - Network services
+     - `NetworkManager`_ by default, `systemd-networkd`_ See Note below.
+     - 
+
+   * - SSH Port scanning blocker
+     - `Tallow`_
+     - 
+
+   * - Firewall
+     - *NA*
+     - iptables and `firewalld`_
+
+   * - Antivirus
+     - *NA*
+     - `ClamAV*`_
+
+   * - Web browser
+     - `Lynx`_ or `links`_ for text environments, `Firefox*`_ for GUI
+     - 
+
+   * - Additional Software
+     - `Supplied Bundles`_
+     - Flatpak, 3rd-party software bundles
+
+.. note::
+   
+   The |CL| OS images targeted for cloud deployments continue to use
+   ``systemd-networkd`` to manage network connections.  In earlier |CL|, 
+   ``systemd-networkd`` was used to manage Ethernet interfaces and NetworkManager was used for wireless interfaces.
+
+
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
+.. _third party alternatives: https://community.clearlinux.org/t/about-the-3rd-party-sw-category/4072
 .. _how-to-clear: https://github.com/clearlinux/how-to-clear
+.. _Clear Linux store: https://clearlinux.org/software
+.. _source code: https://github.com/clearlinux
+.. _swupd post: https://community.clearlinux.org/t/why-does-clearlinux-use-swupd-and-not-apt-deb-rpm/
+.. _swupd: https://github.com/clearlinux/swupd-client
+.. _Clear Linux installer: https://github.com/clearlinux/clr-installer/
+.. _systemd-boot: https://www.freedesktop.org/software/systemd/man/systemd-boot.html
+.. _syslinux: https://wiki.syslinux.org/wiki/index.php?title=The_Syslinux_Project
+.. _Clear Linux Boot Manager: https://github.com/clearlinux/clr-boot-manager
+.. _mixer: https://github.com/clearlinux/mixer-tools
+.. _Clear Linux Distro Factory: https://github.com/clearlinux/clr-distro-factory
+.. _autospec: https://github.com/clearlinux/common
+.. _clr-debug-info: https://github.com/clearlinux/clr-debug-info
+.. _clrtrust: https://github.com/clearlinux/clrtrust
+.. _EXT4: https://ext4.wiki.kernel.org/index.php/Main_Page
+.. _VFAT: https://www.kernel.org/doc/html/latest/filesystems/vfat.html
+.. _EXT2 and EXT3: https://ext4.wiki.kernel.org/index.php/Main_Page
+.. _F2FS: https://www.kernel.org/doc/Documentation/filesystems/f2fs.txt
+.. _LUKS: https://gitlab.com/cryptsetup/cryptsetup/
+.. _systemd: https://www.freedesktop.org/wiki/Software/systemd/
+.. _GNOME: https://www.gnome.org/
+.. _X.Org: https://www.x.org/
+.. _Wayland: https://wayland.freedesktop.org/
+.. _NetworkManager: https://wiki.gnome.org/Projects/NetworkManager
+.. _systemd-networkd: https://www.freedesktop.org/software/systemd/man/systemd.network.html
+.. _Tallow: https://github.com/clearlinux/tallow
+.. _firewalld: https://docs.01.org/clearlinux/latest/guides/network/firewall.html#firewalld
+.. _ClamAV*: https://www.clamav.net/
+.. _Lynx: https://lynx.invisible-island.net/
+.. _links: http://links.twibright.com/
+.. _Firefox*: https://www.mozilla.org/en-US/firefox/
+.. _Supplied Bundles: https://clearlinux.org/software
+.. _micro-config-drive: https://github.com/clearlinux/micro-config-drive
+.. _Telemetrics: https://github.com/clearlinux/telemetrics-backend
+.. _packages source code: https://github.com/clearlinux-pkgs/

source/collaboration/collaboration.rst

@@ -41,7 +41,7 @@ Contribution guidelines
 ***********************
 
 The |CL| documentation is written using reStructuredText. Use our guidelines
-and best practices to write consistent, readable documentation.
+and best practices to write consistent, readable documentation. If you're writing a tutorial, review our skill levels to better target a user group.
 
 .. toctree::
    :maxdepth: 1

source/conf.py

@@ -15,6 +15,9 @@
 import sys
 import os
 import shlex
+#support for modified code block
+from pygments.lexers.shell import BashSessionLexer
+from sphinx.highlighting import lexers
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -26,6 +29,17 @@ import shlex
 # If your documentation needs a minimal Sphinx version, state it here.
 #needs_sphinx = '1.0'
 
+#############
+#
+# Add a special lexer to add a class to console lexer
+#
+#############
+
+class copyAllConsole (BashSessionLexer):
+    name = 'ShellSession'
+
+lexers['ShellSession'] = copyAllConsole(startinLine=True)
+
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
@@ -34,7 +48,8 @@ import shlex
 #]
 
 extensions = [
-    'sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx_sitemap', 'sphinx_tabs.tabs'
+    'sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx_sitemap', 
+    'sphinx_tabs.tabs', 'sphinx_copybutton'
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -54,7 +69,7 @@ master_doc = 'index'
 # General information about the project.
 #project = u'Clear Linux* project'
 project = u'Clear Linux* Project Docs'
-copyright = u'2020.'
+copyright = u'2022 Intel Corporation. All Rights Reserved.'
 author = u'many'
 
 # The version info for the project you're documenting, acts as replacement for
@@ -81,7 +96,7 @@ language = None
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
-exclude_patterns = ['_build','_themes']
+exclude_patterns = ['_build','_themes','.tox','_scripts']
 
 # The reST default role (used for this markup: `text`) to use for all
 # documents.
@@ -174,7 +189,8 @@ html_favicon = '_images/favicon.ico'
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-#html_static_path = ['_static']
+# html_static_path = ['_scripts']
+copybutton_prompt_text = "$ "
 
 # Add any extra paths that contain custom files (such as robots.txt or
 # .htaccess) here, relative to this directory. These files are copied

source/get-started/bare-metal-install-desktop.rst

@@ -4,7 +4,9 @@ Install |CL-ATTR| from the live desktop
 #######################################
 
 This page explains how to boot the |CL-ATTR| live desktop image, from which
-you can install |CL| or explore without modifying the host system.
+you can install |CL| or explore without modifying the host system. 
+Alternatively, use a :ref:`YAML configuration file <install-configfile>` 
+to install |CL|. 
 
 .. contents::
    :local:
@@ -61,28 +63,44 @@ these steps.
 
 .. _preliminary-steps-install-desktop-end:
 
-#. Select :guilabel:`Clear Linux OS` in the boot menu, shown in Figure 1.
+Choose boot menu option
+=======================
 
-   .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-01.png
-      :scale: 100%
-      :alt: Clear Linux OS in boot menu
+#. Choose one of the options shown in Figure 1.
 
-      Figure 1: Clear Linux OS in boot menu
+   a. Follow `Verify integrity of installer media (optional)`_.
+
+   #. Select :guilabel:`Clear Linux OS` in the boot menu.
+
+      .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-01.png
+         :scale: 100%
+         :alt: Clear Linux OS in boot menu
+
+         Figure 1: Clear Linux OS in boot menu
+
+      .. note::
+
+         If no action is taken, the live image starts by default.
 
 .. _install-on-target-end:
 
-Software (optional)
-===================
+Verify integrity of installer media (optional)
+==============================================
 
-Explore |CL| bundles and other software. Double-click the
-:guilabel:`Software` icon from the Activities menu, shown in Figure 2.
-Ensure a network connection exists before launching the Software application.
+Use :guilabel:`Verify ISO Integrity` to verify the checksum of 
+the image burned to the installer media. The checksum ensures that the ISO 
+is uncorrupted (see Figure 1). For every ISO generated, the 
+:guilabel:`clr-installer` implants checksums, which are verified during 
+early boot stage as part of :command:`initrd`. 
+ 
+#. Select :guilabel:`Verify ISO Integrity`. The media will be validated.
 
-.. note::
+#. If the check passes, it will boot into the live image. Continue in 
+   the next section.
 
-   While running the |CL| live desktop image, the Software application is
-   intended only for exploration. Do not attempt to install applications during
-   exploration.
+#. If the check fails, a failure message appears. 
+   
+   * Restart the process at `Preliminary Steps`_. 
 
 .. _install-clr-desktop-start:
 
@@ -344,8 +362,10 @@ boot partition
 
 #. Select :guilabel:`Add`.
 
-swap partition
---------------
+swap partition (optional)
+-------------------------
+
+A swapfile is generated by default during installation. However, if you prefer to create a swap partition, follow the steps below.
 
 #. With :guilabel:`unallocated` highlighted, select from the menu
    :menuselection:`Partition --> New`.
@@ -643,19 +663,24 @@ Create partitions per requirements in Table 1.
      - /boot
      - 150MB
 
-   * - ``linux-swap``
-     - swap
-     -
-     - 256MB
-
    * - ``ext[234], XFS, or f2fs``
      - root
      - /
      - *Size depends upon use case/desired bundles.*
 
+.. note:: 
+   
+   A 64MiB swapfile is generated by default. The default size may be set
+   manually with the ``--swap-file-size`` command-line option.
+
 Troubleshooting
 ***************
 
 :ref:`erase-lvm-troubleshooting-tip`
 
+Related topics
+**************
+
+* :ref:`install-configfile`
+
 .. _Downloads: https://clearlinux.org/downloads

source/get-started/bare-metal-install-server.rst

@@ -4,7 +4,7 @@ Install |CL-ATTR| from the live server
 ######################################
 
 This page explains how to install |CL-ATTR| on bare metal from a bootable USB
-drive using a live server image.
+drive using a live server image. Alternatively, use a :ref:`YAML configuration file <install-configfile>` to install |CL|. 
 
 .. contents::
    :local:
@@ -20,8 +20,8 @@ installation:
 * :ref:`system-requirements`
 * :ref:`compatibility-check`
 
-Download the latest |CL| live server image
-******************************************
+Preliminary steps
+*****************
 
 #. Visit our `Downloads`_ page.
 
@@ -62,7 +62,14 @@ Follow these steps to install |CL| on the target system:
 
 #. Reboot the target system.
 
-#. This action launches the |CL| installer boot menu, shown in Figure 1.
+Choose boot menu option
+=======================
+
+#. Choose one of the options shown in Figure 1.
+
+   a. Follow `Verify integrity of installer media (optional)`_.
+
+   #. Select :guilabel:`Clear Linux OS` in the boot menu.
 
    .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-01.png
       :scale: 100%
@@ -70,7 +77,27 @@ Follow these steps to install |CL| on the target system:
 
       Figure 1: Clear Linux OS Installer boot menu
 
-#. With :guilabel:`Clear Linux OS` highlighted, select :kbd:`Enter`.
+   .. note::
+
+      If no action is taken, the live image starts by default.
+
+Verify integrity of installer media (optional)
+==============================================
+
+Use :guilabel:`Verify ISO Integrity` to verify the checksum of 
+the image burned to the installer media. The checksum ensures that the ISO 
+is uncorrupted (see Figure 1). For every ISO generated, the 
+:guilabel:`clr-installer` implants checksums, which are verified during 
+early boot stage as part of :command:`initrd`. 
+ 
+#. Select :guilabel:`Verify ISO Integrity`. The media will be validated. 
+
+#. If the check passes, it will boot into the live image. Continue in 
+   the next section.
+
+#. If the check fails, a failure message appears. 
+   
+   * Restart the process at `Preliminary Steps`_. 
 
 .. _install-clr-server-start:
 
@@ -318,8 +345,11 @@ boot partition
 
 Now follow the same process to configure the remaining partitions.
 
-swap partition
---------------
+swap partition (optional)
+-------------------------
+
+A swapfile is generated by default during installation. However, if you
+prefer to create a swap partition, follow the steps below. 
 
 #. Use the :kbd:`Up/Down` arrow to select free space.
 
@@ -915,16 +945,16 @@ Create partitions per requirements in Table 1.
      - /boot
      - 150MB
 
-   * - ``linux-swap``
-     - swap
-     -
-     - 256MB
-
    * - ``ext[234], `XFS`, or f2fs``
      - root
      - /
      - *Size depends upon use case/desired bundles.*
 
+.. note:: 
+   
+   A 64MiB swapfile is generated by default. The default size may be set
+   manually with the ``--swap-file-size`` command-line option.
+
 Troubleshooting
 ***************
 
@@ -1001,4 +1031,12 @@ commands:
    sudo dmsetup remove_all --force
    sudo partprobe
 
+Related topics
+**************
+
+* :ref:`install-configfile`
+
+
 .. _Downloads: https://clearlinux.org/downloads
+
+

source/get-started/cloud-install/import-clr-aws.rst

@@ -63,7 +63,7 @@ Create an S3 bucket
    See Figure 1. 
 
    .. figure:: ../../_figures/aws/import-clr-aws-01.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS Services - S3 Management Console
 
       Figure 1: AWS Services - S3 Management Console
@@ -71,7 +71,7 @@ Create an S3 bucket
 #. Click :guilabel:`+ Create bucket`.
 
    .. figure:: ../../_figures/aws/import-clr-aws-02.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Create bucket
 
       Figure 2: AWS S3 - Create bucket
@@ -80,22 +80,22 @@ Create an S3 bucket
    See Figure 3.
 
    .. figure:: ../../_figures/aws/import-clr-aws-03.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Create bucket - Set bucket name and region
 
       Figure 3: AWS S3 - Create bucket - Set bucket name and region
 
-#. Leave the :guilabel:`Configure options" and :guilabel:`Set permissions`
+#. Leave the :guilabel:`Configure options` and :guilabel:`Set permissions`
    settings as is or configure as desired.  See Figure 4 and 5.
 
    .. figure:: ../../_figures/aws/import-clr-aws-04.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Create bucket - Configure options
 
       Figure 4: AWS S3 - Create bucket - Configure options
 
    .. figure:: ../../_figures/aws/import-clr-aws-05.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Create bucket - Set permissions
 
       Figure 5: AWS S3 - Create bucket - Set permissions
@@ -103,7 +103,7 @@ Create an S3 bucket
 #. At the :guilabel:`Review` screen, click :guilabel:`Create bucket`.
 
    .. figure:: ../../_figures/aws/import-clr-aws-06.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Create bucket - Review
 
       Figure 6: AWS S3 - Create bucket - Review
@@ -111,7 +111,7 @@ Create an S3 bucket
    The created bucket should appear.  See Figure 7.
 
    .. figure:: ../../_figures/aws/import-clr-aws-07.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Created bucket
 
       Figure 7: AWS S3 - Created bucket
@@ -123,7 +123,7 @@ Upload the |CL| image into the bucket
    See Figure 8.
 
    .. figure:: ../../_figures/aws/import-clr-aws-08.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Select bucket
 
       Figure 8: AWS S3 - Select bucket
@@ -132,7 +132,7 @@ Upload the |CL| image into the bucket
    See Figure 9.
 
    .. figure:: ../../_figures/aws/import-clr-aws-09.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Upload
 
       Figure 9: AWS S3 - Upload
@@ -141,7 +141,7 @@ Upload the |CL| image into the bucket
    See Figure 10.
 
    .. figure:: ../../_figures/aws/import-clr-aws-10.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Add files
 
       Figure 10: AWS S3 - Add files
@@ -150,19 +150,19 @@ Upload the |CL| image into the bucket
    See Figure 11, Figure 12, and Figure 13.
 
    .. figure:: ../../_figures/aws/import-clr-aws-11.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Add files
 
       Figure 11: AWS S3 - Add files
 
    .. figure:: ../../_figures/aws/import-clr-aws-12.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Set permissions
 
       Figure 12: AWS S3 - Set permissions
 
    .. figure:: ../../_figures/aws/import-clr-aws-13.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Set properties
 
       Figure 13: AWS S3 - Set properties
@@ -171,7 +171,7 @@ Upload the |CL| image into the bucket
    See Figure 14.
 
    .. figure:: ../../_figures/aws/import-clr-aws-14.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS S3 - Upload
 
       Figure 14: AWS S3 - Upload
@@ -183,8 +183,8 @@ Add a user to IAM with AWS_CLI privilege
    and select :guilabel:`IAM`.
    See Figure 15. 
 
-   .. figure:: ../../_figures/aws/import-clr-aws-08.png
-      :scale: 100%
+   .. figure:: ../../_figures/aws/import-clr-aws-15.png
+      :scale: 70%
       :alt: AWS Services - IAM
 
       Figure 15: AWS Services - IAM
@@ -194,7 +194,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 16.
 
    .. figure:: ../../_figures/aws/import-clr-aws-16.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Access management
 
       Figure 16: AWS AIM - Access management
@@ -203,7 +203,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 17.
    
    .. figure:: ../../_figures/aws/import-clr-aws-17.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Add user
 
       Figure 17: AWS AIM - Add user
@@ -212,7 +212,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 18.
 
    .. figure:: ../../_figures/aws/import-clr-aws-18.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Enter user name and select access type
 
       Figure 18: AWS AIM - Enter user name and select access type
@@ -227,7 +227,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 19.
 
    .. figure:: ../../_figures/aws/import-clr-aws-19.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Set user permissions
 
       Figure 19: AWS AIM - Set user permissions
@@ -244,7 +244,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 20.
 
    .. figure:: ../../_figures/aws/import-clr-aws-20.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Create user
 
       Figure 20: AWS AIM - Create user
@@ -255,7 +255,7 @@ Add a user to IAM with AWS_CLI privilege
    See Figure 21.
 
    .. figure:: ../../_figures/aws/import-clr-aws-21.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS AIM - Access key ID and secret access key
 
       Figure 21: AWS AIM - Access key ID and secret access key
@@ -401,29 +401,29 @@ There are 2 methods to create an AMI from the snapshot.
      :guilabel:`EC2`.
      See Figure 22. 
 
-   .. figure:: ../../_figures/aws/import-clr-aws-22.png
-      :scale: 100%
-      :alt: AWS Services - EC2
+     .. figure:: ../../_figures/aws/import-clr-aws-22.png
+        :scale: 70%
+        :alt: AWS Services - EC2
 
-      Figure 22: AWS Services - EC2
+        Figure 22: AWS Services - EC2
     
   #. Click :guilabel:`Snapshots`.
      See Figure 23. 
 
-   .. figure:: ../../_figures/aws/import-clr-aws-23.png
-      :scale: 100%
-      :alt: AWS Services - Snapshots
+     .. figure:: ../../_figures/aws/import-clr-aws-23.png
+        :scale: 70%
+        :alt: AWS Services - Snapshots
 
-      Figure 23: AWS Services - Snapshots
+        Figure 23: AWS Services - Snapshots
 
   #. Locate the snaphot using the `Snapshot ID`.
      See Figure 24.
 
-   .. figure:: ../../_figures/aws/import-clr-aws-24.png
-      :scale: 100%
-      :alt: AWS Services - Snapshots
+     .. figure:: ../../_figures/aws/import-clr-aws-24.png
+        :scale: 70%
+        :alt: AWS Services - Snapshots
 
-      Figure 24: AWS Services - Snapshots
+        Figure 24: AWS Services - Snapshots
     
   #. Right-click it and select :guilabel:`Create Image`.
 
@@ -438,7 +438,7 @@ There are 2 methods to create an AMI from the snapshot.
      See Figure 25.
 
      .. figure:: ../../_figures/aws/import-clr-aws-25.png
-        :scale: 100%
+        :scale: 70%
         :alt: AWS Services - Snapshots
 
         Figure 25: AWS Services - Snapshots
@@ -453,7 +453,7 @@ Launch an instance
    See Figure 26. 
 
    .. figure:: ../../_figures/aws/import-clr-aws-26.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS Services - EC2
 
       Figure 26: AWS Services - EC2
@@ -463,7 +463,7 @@ Launch an instance
    See Figure 27. 
 
    .. figure:: ../../_figures/aws/import-clr-aws-27.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS Services - Launch instance
 
       Figure 27: AWS Services - Launch instance
@@ -472,7 +472,7 @@ Launch an instance
    See Figure 28. 
 
    .. figure:: ../../_figures/aws/import-clr-aws-28.png
-      :scale: 100%
+      :scale: 70%
       :alt: AWS Services - Select AMI
 
       Figure 28: AWS Services - Select AMI

source/get-started/compatibility-check.rst

@@ -1,57 +1,49 @@
 .. _compatibility-check:
 
-Check processor and EFI firmware compatibility
-##############################################
+Check Processor Compatibility
+#############################
 
-Before installing |CL-ATTR|, check your host system's processor and EFI firmware
-compatibility. To check compatibility, choose one of the following paths:
-
-* From a system with a Linux\* OS installed, follow the instructions to :ref:`check-compatibility-steps`.
-
-* From a non-Linux OS, first :ref:`bare-metal-install-desktop` and then follow
-  the instructions to :ref:`check-compatibility-steps`.
+Before installing |CL-ATTR|, check your host system's processor compatibility using one of 
+the following options:
 
 .. note::
    This does not check other system components (for example: storage and
    graphics) for compatibility with |CL|.
 
-.. _check-compatibility-steps:
-
-Check compatibility
-*******************
+Option 1: Use the :command:`clear-linux-check-config.sh` script on an existing Linux system
+*******************************************************************************************
 
 #. Download the `clear-linux-check-config.sh`_ file.
 
    If a browser is not available, use:
 
-   .. code-block:: console
+   .. code-block:: bash
 
       curl -O https://cdn.download.clearlinux.org/current/clear-linux-check-config.sh
 
 #. Make the script executable.
 
-   .. code-block:: console
+   .. code-block:: bash
 
       chmod +x clear-linux-check-config.sh
 
 #. Run the script.
 
-   #. Check to see if the host's processor and EFI firmware is capable of
-      running |CL|.
+   #. Check to see if the host's processor is capable of running |CL|.
 
-      .. code-block:: console
+      .. code-block:: bash
 
          ./clear-linux-check-config.sh host
 
    #. Check to see if the host is capable of running |CL| in a container.
 
-      .. code-block:: console
+      .. code-block:: bash
 
          ./clear-linux-check-config.sh container
 
    The script prints a list of test results similar to the output below.
    All items should return a `SUCCESS` status. This example indicates the
-   host's processor and EFI firmware support running |CL|.
+   host's processor supports running |CL|.
 
    .. code-block:: console
 
@@ -61,8 +53,52 @@ Check compatibility
       SUCCESS: Supplemental Streaming SIMD Extensions 3 (ssse3)
       SUCCESS: Streaming SIMD Extension v4.1 (sse4_1)
       SUCCESS: Streaming SIMD Extensions v4.2 (sse4_2)
-      SUCCESS: Advanced Encryption Standard instruction set (aes)
       SUCCESS: Carry-less Multiplication extensions (pclmulqdq)
-      SUCCESS: EFI Firmware
 
-.. _clear-linux-check-config.sh: https://cdn.download.clearlinux.org/current/clear-linux-check-config.sh
+Option 2: Use a |CL| live image on a non-Linux system
+=====================================================
+
+#. `Download`_ either the `Desktop` or `Server` version of the live image ISO.
+
+#. Follow the instruction to :ref:`bootable-usb`.
+
+#. Boot up the |CL| live image on the USB.
+
+#. Check compatibility as follows:
+
+   * *Desktop version:*
+     
+     a. Open a terminal.
+
+     #. Check compatibility.
+
+        .. code-block:: bash
+
+	   sudo clr-installer --system-check
+
+   * *Server version:*
+
+     a. Log in as `root` and set a password.
+
+     #. Check compatibility.
+
+        .. code-block:: bash
+
+	   clr-installer --system-check
+
+   Expected output for a compatible host processor:
+
+   .. code-block:: console
+
+      Checking for required CPU feature: lm [success]
+      Checking for required CPU feature: sse4_2 [success]
+      Checking for required CPU feature: sse4_1 [success]
+      Checking for required CPU feature: pclmulqdq [success]
+      Checking for required CPU feature: ssse3 [success]
+      Success: System is compatible
+
+.. _clear-linux-check-config.sh: 
+   https://cdn.download.clearlinux.org/current/clear-linux-check-config.sh
+
+.. _Download:
+   https://clearlinux.org/downloads   

source/get-started/ipxe-install.rst

@@ -285,12 +285,12 @@ YAML syntax.
       - login: clrlinux
         username: Clear Linux
         # Password is "clear123"
-        password: \$6\$SJJMfnInWQg.CvMA\$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/        
+        password: $6$SJJMfnInWQg.CvMA$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/        
         admin: true
       - login: root
         username: Root Root
         # Password is "clear123"
-        password: \$6\$SJJMfnInWQg.CvMA\$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/        
+        password: $6$SJJMfnInWQg.CvMA$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/        
         admin: true
 
       pre-install: [

source/get-started/virtual-machine-install/hyper-v.rst

@@ -15,8 +15,8 @@ Overview
 ********
 
 Hyper-V is a type 1 bare-metal hypervisor that runs directly on system
-hardware. It is available for `Windows\* server`_ and client operating systems,
-including `Windows 10`_.
+hardware. It is available for `Windows\* server`_ and client operating 
+systems, including `Windows 10`_.
 
 |CL| provides a virtual disk image for Hyper-V, which also includes
 a :ref:`Hyper-V specific kernel <compatible-kernels>` and drivers.
@@ -43,14 +43,16 @@ Prerequisites
 Download the |CL| disk image for Hyper-V
 ****************************************
 
-#. Download the :file:`clear-[VERSION]-hyperv.vhdx.gz` for Microsoft* Hyper-V
-   from the `downloads`_ website.
+#. Download the :file:`clear-[VERSION]-azure-hyperv.vhd.gz` for Microsoft* 
+   Hyper-V from the `downloads`_ website.
 
 #. Verify and extract the image using these instructions:
    :ref:`download-verify-decompress`.
 
-   After extraction, the file should be named :file:`clear-[VERSION]-hyperv.vhdx`.
+#. Extract the compressed file using software such as the 
+   7-Zip\* tool or the WinZip\* tool. 
 
+   After extraction, the file should be named :file:`clear-[VERSION]-azure-hyperv.vhd`.
 
 Create and configure new VM
 ****************************
@@ -63,6 +65,10 @@ Create and configure new VM
 
       Figure 1: Hyper-V Manager from the Start menu
 
+   .. note::
+
+      You may need to manually enable Hyper-V on a Windows\* machine. Review 
+      ``Windows Features``.
 
 #. Create a *New Virtual Machine* by clicking the :guilabel:`Action` menu,
    then selecting :guilabel:`New` and :guilabel:`Virtual Machine...`.
@@ -77,23 +83,22 @@ Create and configure new VM
    specifying the options below:
 
    - **Name**: Choose name (for example, ClearLinuxOS-VM)
-   - **Generation**: Generation 2
+   - **Specify Generation**: Generation 1
    - **Startup memory**: 2048 MB or more
+   - **Configure Networking**: Change :guilabel:`Connection` to `Default Switch`
    - **Connect Virtual Hard Disk**: Select :guilabel:`Use an existing virtual
-     hard disk` and browse to find the :file:`clear-[VERSION]-hyperv.vhdx`
-     file.
+     hard disk` and browse to find the 
+     :file:`clear-[VERSION]-azure-hyperv.vhd` file.
 
    After finishing the wizard, the VM will be created but not powered on.
 
 #. Configure the VM by right-clicking it in the Hyper-V Manager and selecting
-   :guilabel:`Settings...`.
-   Figure 3 shows the Settings page after configuration selections.
+   :guilabel:`Settings...`. Figure 3 shows the Settings page after configuration selections.
 
-   - Under :guilabel:`Firmware`, select the Virtual disk and click
-     :guilabel:`Move Up...` until it is at the top of the list.
+   **Optional**
 
-   - Under :guilabel:`Security`, uncheck the :guilabel:`Enable Secure Boot`
-     checkbox.
+   - If you wish to `Encrypt state and virtual machine traffic, under 
+     :guilabel:`Security`, select :guilabel:`Add Key Storage Drive`.
 
    - Under :guilabel:`Processor`, consider increasing the number of virtual
      processors assigned to the |CL| VM to improve performance.
@@ -112,12 +117,12 @@ Create and configure new VM
 Start the VM
 ************
 
-#. Start the |CL| VM by right-clicking the VM in Hyper-V Manager and selecting
-   :guilabel:`Start`.
+#. Start the |CL| VM by right-clicking the VM in Hyper-V Manager and 
+   selecting :guilabel:`Start`.
 
 #. Connect to the VM console by right-clicking the VM in Hyper-V Manager and
-   selecting :guilabel:`Connect...`. A new *Virtual Machine Connection* window
-   is displayed.
+   selecting :guilabel:`Connect...`. A new *Virtual Machine Connection* 
+   window is displayed.
 
 #. After |CL| is booted, log in to the console with user *root*. You are
    prompted to set a new password immediately.
@@ -133,7 +138,7 @@ Related topics
 
 * :ref:`increase-virtual-disk-size`
 
-
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
 
 .. _`Windows\* Server`: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/hyper-v-on-windows-server
 .. _`Windows 10`: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/index

source/get-started/virtual-machine-install/kvm.rst

@@ -41,13 +41,13 @@ Download and launch the virtual machine image
 
    .. code-block:: bash
 
-      curl -O https://cdn.download.clearlinux.org/image/$(curl https://cdn.download.clearlinux.org/image/latest-images | grep '[0-9]'-kvm'\.')
+      curl -o clear.img.xz https://cdn.download.clearlinux.org/image/$(curl https://cdn.download.clearlinux.org/image/latest-images.json | grep -o clear-'[0-9]'*-kvm.img.xz | head -1)
 
 #. Uncompress the downloaded image:
 
    .. code-block:: bash
 
-      unxz -v clear-<version>-kvm.img.xz
+      xz -dv clear.img.xz
 
 #. Download the 3 OVMF files (`OVMF.fd`, `OVMF_CODE.fd`, `OVMF_VARS.fd`) that
    provides UEFI  support for virtual machines.
@@ -75,17 +75,11 @@ Download and launch the virtual machine image
 
       curl -O https://cdn.download.clearlinux.org/image/start_qemu.sh
 
-#. Make the script executable:
-
-   .. code-block:: bash
-
-      chmod +x start_qemu.sh
-
 #. Start the |CL| KVM virtual machine:
 
    .. code-block:: bash
 
-      sudo ./start_qemu.sh clear-<version>-kvm.img
+      sudo bash ./start_qemu.sh clear.img
 
 #. Log in as ``root`` user and set a new password.
 
@@ -178,7 +172,7 @@ To add :abbr:`GDM (GNOME Display Manager)` to the |CL| VM, follow these steps:
 
    .. code-block:: bash
 
-      sudo ./start_qemu.sh clear-<version>-kvm.img
+      sudo ./start_qemu.sh clear.img
 
 #. Determine the IP address of the host on which you will launch the VM.
    Substitute <ip-addr-of-kvm-host> in the next step with this information.
@@ -186,6 +180,7 @@ To add :abbr:`GDM (GNOME Display Manager)` to the |CL| VM, follow these steps:
    .. code-block:: bash
 
       ip a
+      
 
 #. From the local host or remote system, open a new terminal emulator window
    and connect into the |CL| VM using the Spice viewer:
@@ -218,6 +213,8 @@ To add :abbr:`GDM (GNOME Display Manager)` to the |CL| VM, follow these steps:
    corner).
 
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _Intel® Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html
 .. _Intel® Virtualization Technology for Directed I/O: https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices
 .. _start_qemu.sh: https://cdn.download.clearlinux.org/image/start_qemu.sh

source/get-started/virtual-machine-install/virt-manager.rst

@@ -214,6 +214,8 @@ Congratulations! You have successfully installed |CL| in your new VM and can
 begin using it immediately. The `virt-manager` tool is maintained on GitHub\*
 at `virt-manager-github`_.
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _virt-mgr: https://www.virt-manager.org
 
 .. _Downloads: https://clearlinux.org/downloads

source/get-started/virtual-machine-install/virtualbox-cl-installer.rst

@@ -389,6 +389,9 @@ Troubleshooting
 
       bcdedit /set {current} hypervisorlaunchtype Auto
 
+
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _VirtualBox Installation Instructions: https://www.virtualbox.org/manual/ch02.html
 
 .. _VirtualBox: https://www.virtualbox.org

source/get-started/virtual-machine-install/vmw-player.rst

@@ -3,7 +3,7 @@
 |CL-ATTR| on VMware\* Workstation Player
 ########################################
 
-This guide explains how to set up the VMware Workstation Player 15.5.1
+This guide explains how to set up the VMware\* Workstation Player 15.5.1
 hypervisor and instantiate a VM instance of |CL| by installing it using 
 an ISO or using a pre-built image.
 
@@ -33,8 +33,8 @@ it, see :ref:`vmware-esxi-install-cl`.
 Install the VMware Workstation Player hypervisor
 ************************************************
 
-#. Enable :abbr:`Intel® VT (Intel® Virtualization Technology)` and
-   :abbr:`Intel® VT-d (Intel® Virtualization Technology for Directed I/O)` in
+#. Enable Intel® Virtualization Technology (Intel® VT) and
+   Intel® Virtualization Technology for Directed I/O (Intel® VT-d) in
    your system's BIOS.
 
 #. `VMware Workstation Player`_ is available for Windows and Linux.
@@ -407,6 +407,8 @@ For other guides on using the VMWare Player and ESXi, see:
 
 * :ref:`vmware-esxi-install-cl`
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html
 
 .. _VMware Workstation Player:

source/guides/clear/autospec.rst

@@ -594,6 +594,7 @@ Related topics
 **************
 
 * :ref:`Mixer tool <mixer>`
+* :ref:`Proxy Configuration <proxy>`
 
 .. _contributing to an existing software package: https://github.com/clearlinux/distribution/blob/master/contributing.md#contributing-to-an-existing-software-package
 
@@ -607,4 +608,4 @@ Related topics
 .. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/
 
 
-.. TODO:  Add link to how to submit a new package: https://github.com/clearlinux/distribution/blob/master/contributing.md#contributing-a-new-software-package
+.. TODO:  Add link to how to submit a new package: https://github.com/clearlinux/distribution/blob/master/contributing.md#contributing-a-new-software-package

source/guides/clear/compatible-kernels.rst

@@ -12,7 +12,7 @@ Bare metal only
 
 Kernel native
     The *kernel-native* bundle focuses on the bare metal platforms. It is
-    optimized for fast booting and performs best on the Intel® architectures
+    optimized for fast booting and performs best on the Intel® Architecture Processors
     described on the :ref:`supported hardware list<system-requirements>`. The
     optimization patches are found in our `Linux`_ GitHub\* repo.
 
@@ -24,8 +24,8 @@ Also compatible with VMs
 Kernel LTS
     The *kernel-lts* bundle focuses on the bare metal platforms but uses the
     latest :abbr:`LTS (Long Term Support)` Linux kernel. It is optimized for
-    fast booting and performs best on the Intel® architectures described on the
-    :ref:`supported hardware list<system-requirements>`. Additionally, this
+    fast booting and performs best on the Intel® Architecture Processors described
+    on the :ref:`supported hardware list<system-requirements>`. Additionally, this
     kernel includes the VirtualBox\* kernel modules, see our
     :ref:`instructions on using Virtualbox<virtualbox-cl-installer>` for more
     information. The optimization patches are found in our `Linux-LTS`_ GitHub
@@ -37,8 +37,8 @@ VM only
 Kernel KVM
     The *kernel-kvm* bundle focuses on the Linux
     :abbr:`KVM (Kernel-based Virtual Machine)`. It is optimized for fast
-    booting and performs best on Virtual Machines running on the Intel®
-    architectures described on the
+    booting and performs best on Virtual Machines running on the Intel® Architecture
+    Processors described on the
     :ref:`supported hardware list<system-requirements>`. Use this kernel when
     running |CL| as the guest OS on top of *qemu/kvm*. Use this kernel with
     **cloud orchestrators** using *qemu/kvm* internally as their **hypervisor**
@@ -49,7 +49,7 @@ Kernel KVM
 Kernel Hyper-V\*
     The *kernel-hyperv* bundle focuses on running Linux on Microsoft\*
     Hyper-V. It is optimized for fast booting and performs best on Virtual
-    Machines running on the Intel® architectures described on the
+    Machines running on the Intel® Architecture Processors described on the
     :ref:`supported hardware list<system-requirements>`.
     Use this kernel when running |CL| as the guest OS of **Cloud Instances** in
     projects such as Microsoft `Azure`_\*. This kernel can be used in a
@@ -57,18 +57,7 @@ Kernel Hyper-V\*
     for more information. The optimization patches are found in our
     `Linux-HyperV`_ GitHub repo.
 
-Kernel Hyper-V LTS
-    The *kernel-hyperv-lts* bundle focuses on running Linux on Microsoft
-    Hyper-V but uses the latest :abbr:`LTS (Long Term Support)` Linux kernel.
-    It is optimized for fast booting and performs best on Virtual
-    Machines running on the Intel® architectures described on the
-    :ref:`supported hardware list<system-requirements>`.
-    Use this kernel when running |CL| as the guest OS of **Cloud Instances** in
-    projects such as Microsoft `Azure`_. This kernel can be used in a
-    standalone |CL| VM, see our :ref:`instructions on using Hyper-V<hyper-v>`
-    for more information. The optimization patches are found in our
-    `Linux-HyperV-LTS`_ GitHub repo.
-
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
 
 .. _Linux: https://github.com/clearlinux-pkgs/linux
 .. _Linux-LTS: https://github.com/clearlinux-pkgs/linux-lts

source/guides/clear/k8s-migration.rst

@@ -0,0 +1,321 @@
+.. _kubernetes-migration:
+
+Kubernetes\* migration
+######################
+
+This guide describes how to migrate `Kubernetes container orchestration system`_ on |CL-ATTR| from 1.17.x to 1.19.x.
+
+.. contents::
+   :local:
+   :depth: 1
+
+Background
+**********
+
+The version of Kubernetes\* was bumped from 1.17.7 to 1.19.4 in |CL-ATTR|
+release 34090. This guide and the |CL| bundle `k8s-migration` were created
+to help facilitate migration of a cluster from 1.17.x to the latest 1.19.x .
+
+The new |CL| bundle `k8s-migration` was added in |CL-ATTR| release 34270.
+
+Prerequisites
+*************
+
+* Make sure you check any updates to kubernetes upgrade doc for caveats related to the version that is running in the cluster.
+* Make sure ALL the nodes are in Ready state. Without that, the cluster cannot be upgraded.
+  Either fix the broken nodes or remove them from the cluster.
+
+.. contents::
+   :local:
+   :depth: 1
+
+Upgrade 1.17.x ---> 1.18.15
+***************************
+
+#. Upgrade Control Node to 1.18.15 first
+
+   First step would be to upgrade one of the main control node and
+   update kubernetes components on them. You will need to have a newer
+   version of :command:`kubeadm` for the upgrade to work. Please consult
+   `kubeadm upgrade guide`_
+   for any caveats from your current version to the new one.
+
+   Update |CL| to the latest release to update the kubernetes version.
+
+   .. code-block:: bash
+
+      sudo -E swupd update
+
+   .. note::
+      Note: PLEASE DO NOT REBOOT YOUR SYSTEM AT THIS TIME. |CL| is awesome and
+      your stuff will work just fine.
+
+#. Add the new Kubernetes migration bundle which contains the 1.18.15 binaries.
+
+   .. code-block:: bash
+
+      sudo -E swupd bundle-add k8s-migration
+
+#. Find the upgrade version of kubeadm that can used. This should be 1.18.15.
+
+   This command will show the command and possible jumps that can be made from the current kubernetes version.
+
+   .. code-block:: bash
+
+      sudo -E /usr/k8s-migration/bin/kubeadm upgrade plan
+
+   Sample output:
+
+   .. code-block:: console
+
+      [upgrade/config] Making sure the configuration is correct:
+      [upgrade/config] Reading configuration from the cluster...
+      [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
+      [preflight] Running pre-flight checks.
+      [upgrade] Running cluster health checks
+      [upgrade] Fetching available versions to upgrade to
+      [upgrade/versions] Cluster version: v1.17.17
+      [upgrade/versions] kubeadm version: v1.18.15
+      I0209 21:12:49.868786  832739 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.18
+      [upgrade/versions] Latest stable version: v1.18.15
+      [upgrade/versions] Latest stable version: v1.18.15
+      [upgrade/versions] Latest version in the v1.17 series: v1.17.17
+      [upgrade/versions] Latest version in the v1.17 series: v1.17.17
+
+      Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
+      COMPONENT   CURRENT       AVAILABLE
+      Kubelet     3 x v1.17.7   v1.18.15
+
+      Upgrade to the latest stable version:
+
+      COMPONENT            CURRENT    AVAILABLE
+      API Server           v1.17.17   v1.18.15
+      Controller Manager   v1.17.17   v1.18.15
+      Scheduler            v1.17.17   v1.18.15
+      Kube Proxy           v1.17.17   v1.18.15
+      CoreDNS              1.6.5      1.6.7
+      Etcd                 3.4.3      3.4.3-0
+
+      You can now apply the upgrade by executing the following command:
+
+        kubeadm upgrade apply v1.18.15
+
+#. Upgrade the node to the intermediate 1.18.15 version of Kubernetes.
+
+   .. code-block:: bash
+
+      sudo -E /usr/k8s-migration/bin/kubeadm upgrade apply v1.18.15
+
+   .. note::
+      Note: Do **not** reboot the system yet.
+
+#. Upgrade Additional Control Nodes to 1.18.15
+
+   In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs.
+
+#. Upgrade Other Nodes to 1.18.15
+
+   For each of the other nodes:
+
+   a. Update |CL| to the latest release to update the kubernetes version.
+
+      .. code-block:: bash
+
+         sudo -E swupd update
+
+   #. Add the new Kubernetes migration bundle which contains the 1.18.15 binaries.
+
+      .. code-block:: bash
+
+         sudo -E swupd bundle-add k8s-migration
+
+   #. On the **Admin node**, drain the Client node *FIRST*
+
+      .. code-block:: bash
+
+         /usr/k8s-migration/bin/kubectl drain <CLIENT_NODE_NAME> --ignore-daemonsets --delete-local-data
+
+   #. Back on the **Client node**, upgrade Kubernetes on the Client
+
+      .. code-block:: bash
+
+         sudo -E /usr/k8s-migration/bin/kubeadm upgrade node
+
+   #. On the **Admin node**, re-enable the Client
+
+      .. code-block:: bash
+
+         /usr/k8s-migration/bin/kubectl uncordon <CLIENT_NODE_NAME>
+
+
+   #. Back on the **Client node**, restart Kubernetes on the Client
+
+      .. code-block:: bash
+
+         sudo -E systemctl restart kubelet
+
+#. Restart Kubernetes on the Admin node(s) to finish the 1.18.x upgrade
+
+   .. code-block:: bash
+
+      sudo -E systemctl restart kubelet
+
+   .. note::
+      Note: Wait for all nodes to be Ready and showing the 1.19.x version.
+      This version will now show as it is the released version the
+      service files will see and use, but the Nodes are *not* upgraded yet.
+
+Upgrade 1.18.15 ---> 1.19.x
+***************************
+
+#. Upgrade Control Node to 1.19.x
+
+   Now that systems are upgraded to the intermediate release of 1.18.15
+   each of the nodes can be upgraded to the latest 1.19.x release.
+
+#. Find the upgrade version of kubeadm that can used. This should be 1.19.x.
+
+   This command will show the command and possible jumps that can be made from the current kubernetes version.
+
+   .. code-block:: bash
+
+      sudo -E kubeadm upgrade plan
+
+   Sample output:
+
+   .. code-block:: console
+
+      [upgrade/config] Making sure the configuration is correct:
+      [upgrade/config] Reading configuration from the cluster...
+      [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
+      [preflight] Running pre-flight checks.
+      [upgrade] Running cluster health checks
+      [upgrade] Fetching available versions to upgrade to
+      [upgrade/versions] Cluster version: v1.18.15
+      [upgrade/versions] kubeadm version: v1.19.7
+      I0209 23:08:23.810900  925910 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.19
+      [upgrade/versions] Latest stable version: v1.19.7
+      [upgrade/versions] Latest stable version: v1.19.7
+      [upgrade/versions] Latest version in the v1.18 series: v1.18.15
+      [upgrade/versions] Latest version in the v1.18 series: v1.18.15
+
+      Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
+      COMPONENT   CURRENT       AVAILABLE
+      kubelet     3 x v1.17.7   v1.19.7
+
+      Upgrade to the latest stable version:
+
+      COMPONENT                 CURRENT    AVAILABLE
+      kube-apiserver            v1.18.15   v1.19.7
+      kube-controller-manager   v1.18.15   v1.19.7
+      kube-scheduler            v1.18.15   v1.19.7
+      kube-proxy                v1.18.15   v1.19.7
+      CoreDNS                   1.6.7      1.7.0
+      etcd                      3.4.3-0    3.4.13-0
+
+      You can now apply the upgrade by executing the following command:
+
+        kubeadm upgrade apply v1.19.7
+
+      The table below shows the current state of component configs as understood by this version of kubeadm.
+      Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or
+      resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually
+      upgrade to is denoted in the "PREFERRED VERSION" column.
+
+      API GROUP                 CURRENT VERSION   PREFERRED VERSION   MANUAL UPGRADE REQUIRED
+      kubeproxy.config.k8s.io   v1alpha1          v1alpha1            no
+      kubelet.config.k8s.io     v1beta1           v1beta1             no
+
+#. Upgrade the node to the latest 1.19.x version of Kubernetes.
+
+   .. code-block:: bash
+
+      sudo -E /usr/bin/kubeadm upgrade apply v1.19.7
+
+   .. note::
+
+      Note: Do **not** reboot the system yet.
+
+#. Upgrade Additional Control Nodes to 1.19.x
+
+   In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs.
+
+#. Upgrade Other Nodes to 1.19.x
+
+   For each of the other nodes:
+
+   a. On the **Admin node**, drain the Client *FIRST*
+
+      .. code-block:: bash
+
+         kubectl drain <CLIENT_NODE_NAME> --ignore-daemonsets
+
+   #. Back on the **Client node**, upgrade Kubernetes on the Client
+
+      .. code-block:: bash
+
+         sudo -E kubeadm upgrade node
+
+   #. On the **Admin node**, re-enable the Client
+
+      .. code-block:: bash
+
+         kubectl uncordon <CLIENT_NODE_NAME>
+
+   #. Back on the **Client node**, if you wish reboot the Client, it is now safe to do so.
+
+      .. code-block:: bash
+
+         sudo reboot
+
+#. Reboot the Control Node (optional)
+
+  *If you wish reboot the nodes, it is now safe to do so.*
+
+   .. code-block:: bash
+
+      sudo reboot
+
+**Congratulations!**
+
+You've successfully installed and set up Kubernetes in |CL| using CRI-O and kata-runtime. You are now ready to follow on-screen instructions to deploy a pod network to the cluster and join worker nodes with the displayed token and IP information.
+
+Clean up: Remove the migration bundle for each node
+
+.. code-block:: bash
+
+   sudo -E swupd bundle-remove k8s-migration
+
+Related topics
+**************
+
+Read the Kubernetes documentation to learn more about:
+
+*  `Kubernetes tutorial <tutorials/kubernetes>`_
+
+*  `Kubernetes best practices <tutorials/kubernetes-bp>`_
+
+*  Deploying Kubernetes with a `cloud-native-setup`_
+
+* `Understanding basic Kubernetes architecture`_
+
+* `Deploying an application to your cluster`_
+
+* Installing a `pod network add-on`_
+
+* `Joining your nodes`_
+
+
+.. _kubeadm upgrade guide: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
+
+.. _Kubernetes container orchestration system: https://kubernetes.io/
+
+.. _Understanding basic Kubernetes architecture: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3
+
+.. _Deploying an application to your cluster: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2
+
+.. _pod network add-on: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network
+
+.. _Joining your nodes: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes
+
+.. _cloud-native-setup: https://github.com/clearlinux/cloud-native-setup/tree/master/clr-k8s-examples

source/guides/clear/mixer.rst

@@ -52,6 +52,9 @@ Prerequisites
   Add the mixer tool by installing the :command:`mixer` bundle. Refer to
   :ref:`swupd-guide` for more information on installing bundles.
 
+* If you're working behind a corporate proxy, configure proxy settings using
+  the :ref:`General proxy settings for many applications <proxy>` steps.
+
 * Location to host the update content and images
 
   In order for :command:`swupd` to make use of your mix, the update content for your mix must be hosted on a web server. Your mix will be configured with an update location URL, which :command:`swupd` will use to pull down updates.

source/guides/clear/performance.rst

@@ -0,0 +1,241 @@
+.. _performance:
+
+Performance
+###########
+
+|CL-ATTR| is built with optimizations across the whole stack for improved
+performance. |CL| achieves its performance through a variety of design decisions
+and software building techniques.
+
+.. contents:: :local:
+   :depth: 1
+
+Overview
+********
+
+The |CL| philosophy is to do everything with performance in mind. The |CL| team
+applies this philosophy in the project's codebase and operating culture.
+
+Below are some examples of the |CL| philosophy:
+
+**Consider performance holistically.**
+  Performance optimizations are considered across hardware and software. |CL|
+  shows the performance potential of a holistic approach on Linux, using Intel®
+  architecture with optimizations across the full stack.
+
+**Optimize for runtime performance.**
+  In general, |CL| will trade the one-time cost of longer build time and larger
+  storage footprint for the repeated benefit of improved runtime performance.
+  |CL| users benefit from the optimized software but aren't affected by the
+  increased build time because the |CL| team builds the software before
+  distributing it to |CL| clients.
+
+**Optimize performance for server and cloud use cases first.**
+  Design decisions that optimize performance for server and cloud also benefit
+  other use cases, such as IoT devices and desktop clients.
+
+|CL| has become well-known for the performance it can deliver.
+`Phoronix publishes
+Linux performance comparisons <https://www.phoronix.com/scan.php?page=news_topic&q=Clear+Linux>`_
+that include |CL|.
+
+Software build toolchain
+************************
+
+|CL| uses many techniques in its software build toolchain to improve software
+performance, such as aggressive compiler flags and CPU-specific optimizations.
+If maintained manually, these techniques can become complex to support due to
+the volume of packages and the potential for technical drift of package
+performance configurations. The |CL| team built the :ref:`autospec` tool to
+manage this complexity and to apply the techniques used in the software build
+toolchain across the entire project. autospec is available as part of the OS for
+developers to use when they build their own projects on |CL|.
+
+Latest versions of compilers and low-level libraries
+====================================================
+
+|CL| is a rolling release distribution and follows upstream software
+repositories, including compilers and libraries, for updates. |CL| includes
+upstream source-level optimizations as soon as they're available.
+
+A benchmark approach to compiler performance
+============================================
+
+|CL| chooses the compiler used to build each software package on a case-by-case
+basis to maximize performance. Typically, |CL| uses the open source `GNU Compiler
+Collection <https://gcc.gnu.org/>`_ (GCC) with the standard low-level
+libraries `Glibc <https://www.gnu.org/software/libc/>`_ and
+`libstdc++ <https://gcc.gnu.org/onlinedocs/libstdc++/>`_ for C and C++
+programming languages. If there is a performance advantage, |CL| will build
+packages with `Clang / LLVM <https://clang.llvm.org/>`_.
+
+|CL| uses patched compilers and low-level libraries for exact control of the
+software build. Patches include changes that default to more aggressive
+optimizations or optimizations that haven't yet been merged upstream.
+
+View the full list of patches in the autospec repositories on GitHub:
+
+* https://github.com/clearlinux-pkgs/gcc
+* https://github.com/clearlinux-pkgs/glibc
+* https://github.com/clearlinux-pkgs/llvm
+
+Aggressive compiler flags
+=========================
+
+|CL| uses aggressive
+`compiler flags <https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html>`_ to
+optimize software builds for runtime performance. Some significant flags that
+|CL| often implements are:
+
+`mtune and march <https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html>`_
+	Options used to tune generated code with optimized instructions for specific
+	CPU types instead of creating generic code for maximum compatibility.
+
+	|CL| defines its minimum hardware requirements to be second-generation
+	Intel® microarchitecture code name Westmere (released in 2010) or later.
+	This enables compiler optimizations that are available only on newer
+	architectures. Whenever possible, |CL| tunes code for the Haswell generation
+	processors or newer.
+
+	|CL| sets	:command:`march=westmere` and :command:`mtune=haswell`.
+
+	.. note::
+		|CL| doesn't require Advanced Encryption Standard (AES), so it should
+		run on some Intel CPUs from the first generation of Intel® microarchitecture code name Nehalem (released in 2008). Refer to the
+		`recommended minimum system requirements <https://docs.01.org/clearlinux/latest/reference/system-requirements.html>`_ for specific requirements.
+
+`O3 <https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html>`_
+	The largest preset of compiler options optimizations for performance. O3
+	favors runtime performance.
+
+	View the "Optimize Options" section of the GCC man page for additional
+	information: :command:`man gcc`
+
+`LTO <https://gcc.gnu.org/onlinedocs/gccint/LTO.html>`_
+	Link-time optimization that performs an optimization between compiled object
+	files and creation of executable binaries by adding extra information to the
+	compiled object to help the linker.
+
+`PGO <https://en.wikipedia.org/wiki/Profile-guided_optimization>`_
+	Profile guided optimization or field guided optimization performs
+	optimization based on information sampled during the execution of the program.
+
+
+Compiler flags are set at different levels in the |CL| build environment:
+
+User flags
+	The set of default flags used by |CL| when a user compiles software
+	from source. The flags are exported as system-wide environment variables from
+	the
+	`/usr/share/defaults/etc/profile <https://github.com/clearlinux-pkgs/filesystem/blob/master/profile.x86_64>`_ file to the user’s shell by default. These are the
+	standard variables read by the compiler, named :command:`*FLAGS`, depending
+	on the compiler.
+
+	.. note::
+		Source code may come with software build systems that
+		override these values. This will cause a difference in expected flags.
+		The |CL| autospec tooling will attempt to ignore these overrides, but
+		the build system may still need patching. A manual build will not ignore
+		the build system override values if they exist.
+
+Global flags
+	Compiler flags applied at a global level for all packages. The |CL| RPM
+	configuration (`clr-rpm-config <https://github.com/clearlinux/clr-rpm-config>`_)
+	contains global compiler flags. Search the :file:`macros` file for
+	:command:`global_cflags` and search the :file:`rpmrc` file for
+	:command:`optflags`. Global compiler flags may be overridden.
+
+	.. note::
+		|CL| doesn't use RPMs to install software. |CL|
+		distributes software in the form of :ref:`bundles-guide`. The RPM format
+		is only used during the |CL| build process as a way to resolve
+		dependencies.
+
+Per-package flags
+	Compiler flags applied at a per-package level. The package's autospec
+	repository contains the package-specific compiler flags. Search the
+	:file:`.spec` file for the
+	section starting with :command:`export CFLAGS`.
+
+Multiple builds of libraries with CPU-specific optimizations
+============================================================
+
+To fully use the capabilities in different generations of CPU hardware, |CL|
+will perform multiple builds of libraries with CPU-specific optimizations. For
+example, |CL| builds libraries with Intel® Advanced Vector Extensions 2 (Intel®
+AVX2) and Intel® Advanced Vector Extensions 512 (Intel® AVX-512). |CL| can then
+dynamically link to the library with the newest optimization based on the
+processor in the running system. Runtime libraries used by ordinary applications
+benefit from these CPU specific optimizations.
+
+The autospec repository for Python* shows an example of this optimization:
+https://github.com/clearlinux-pkgs/python3
+
+Kernel
+******
+
+A modern kernel with variants optimized for different platforms
+===============================================================
+
+|CL| is a rolling release distribution that uses the newest upstream Linux
+kernel. The Linux kernel has frequent updates which can include performance
+enhancements. It's a policy of the |CL| team to try to upstream any performance
+enhancements in the Linux kernel for all to use.
+
+|CL| `builds different kernel variants <https://docs.01.org/clearlinux/latest/guides/clear/compatible-kernels.html>`_ for compatibility with specific platforms.
+For example, kernels meant to run on virtual machines skip support for much of
+the physical hardware that doesn’t show up in VM environments and will slow down
+boot.
+
+View the kernel configuration and patches to the default native kernel in the
+autospec repository: https://github.com/clearlinux-pkgs/linux/
+
+Utility to enforce kernel runtime parameters
+============================================
+
+The Linux kernel exposes parameters for tuning the behavior of drivers and
+devices such as certain buffers and resource management strategies. |CL| uses a
+small utility, `clr-power-tweaks <https://github.com/clearlinux-pkgs/clr-power-tweaks>`_,
+to set and enforce kernel parameter values weighted towards performance upon
+boot. View the set performance values by running :command:`sudo clr_power --debug`.
+
+Operating system
+****************
+
+Operating system and software build-time optimizations set the stage for high
+performance. Decisions made after the installation of |CL| are equally as
+important.
+
+
+CPU performance governor
+========================
+
+|CL| uses the performance CPU governor which calls for the CPU to operate at
+maximum clock frequency. In other words, P-state P0. The idea behind prioritizing
+maximum CPU performance is that the faster a program finishes execution, the
+faster the CPU can return to a low energy idle state. See the `CPU Power and
+Performance documentation <https://docs.01.org/clearlinux/latest/guides/maintenance/cpu-performance.html>`_
+for further details.
+
+Restructured boot sequence
+==========================
+
+To optimize boot speed, |CL| uses a restructured order for boot processes that
+minimizes the time services wait on slow operations and the time boot processes
+wait on each other.
+
+Systemd-bootchart is a tool for graphing the boot sequence and writes logs to a
+file under :file:`/run/log`. The tool and corresponding log file make diagnosing slow
+boot problems easier. All |CL| systems have `systemd-bootchart <https://github.com/systemd/systemd-bootchart>`_ enabled by default for every boot. systemd-bootchart configuration is
+non-blocking to not materially slow down boot performance.
+
+Related topics
+**************
+
+* :ref:`cpu-performance`
+* `A Linux* OS for Linux Developers <https://clearlinux.org/blogs-news/linux-os-linux-developers>`_
+* `The Performance Race <https://clearlinux.org/news-blogs/performance-race>`_
+* `Boosting Python* from profile-guided to platform-specific optimizations <https://clearlinux.org/news-blogs/boosting-python-profile-guided-platform-specific-optimizations>`_
+* `Transparent use of library packages optimized for Intel® architecture <https://clearlinux.org/news-blogs/transparent-use-library-packages-optimized-intel-architecture>`_
+
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*

source/guides/clear/security.rst

@@ -138,6 +138,8 @@ some examples:
 * `Tallow`_, a lightweight service which monitors and blocks suspicious SSH
   login patterns, is installed with the :command:`openssh-server` bundle.
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _`Security for software update in Clear Linux* OS`: https://clearlinux.org/blogs/security-software-update-clear-linux-os-intel-architecture
 .. _`Recent GNU* C library improvements`: https://clearlinux.org/blogs/recent-gnu-c-library-improvements
 .. _`rolling release`: https://en.wikipedia.org/wiki/Rolling_release

source/guides/clear/swupd-3rd-party.rst

@@ -23,6 +23,8 @@ client system.
    :local:
    :depth: 1
 
+Also, see our `general guidelines`_ on sharing 3rd-party bundles.  
+
 Prerequisite
 *************
 
@@ -198,7 +200,7 @@ All installed 3rd-party bundles reside in :file:`/opt/3rd-party/bundles/<repo-na
 
       tree /opt/3rd-party
 
-   Example out:
+   Example output:
    
    .. code-block:: console
 
@@ -245,6 +247,11 @@ On the client side:
 
 #. Run :command:`sudo swupd 3rd-party update` to update to the latest version of your mix.
 
+   .. note::
+      
+      If `swupd autoupdate` is enabled, 3rd-party repositories will update 
+      automatically as well during regular swupd update.  
+
 #. Now, you can see and add the new bundles.
 
 Some limitations of 3rd-party bundles
@@ -270,4 +277,5 @@ Related topics
    https://clearlinux.org/software
 .. _bundle definition:
    https://docs.01.org/clearlinux/latest/guides/clear/mixer.html#id16
-
+.. _general guidelines:
+   https://community.clearlinux.org/t/about-the-3rd-party-sw-category/4072

source/guides/index.rst

@@ -47,11 +47,3 @@ Kernel
 
    kernel/*
 
-Stacks
-=======
-
-.. toctree::
-   :maxdepth: 1
-   :glob:
-
-   stacks/*

source/guides/kernel/kernel-boot-msg.rst

@@ -0,0 +1,69 @@
+.. _kernel-boot-msg:
+
+Capture Kernel Boot Messages in the Journal
+###########################################
+
+By default |CL| does not capture kernel boot messages in the journal logs, 
+where they're reported as "Missed" messages.  This design decision was made 
+to provide a faster boot performance.  On the other hand, if you wish to 
+see the messages, follow this guide.
+
+Here's an example a journal log with "Missed" messages:
+
+.. code-block:: console
+   :linenos:
+   :emphasize-lines: 4
+
+   -- Reboot --
+   Apr 10 19:55:43 kernel systemd-journald[300]: Journal started
+   Apr 10 19:55:43 kernel systemd-journald[300]: Runtime Journal (/run/log/journal/d01862ca79d1064ea379cd715cfdd53a) is 5.8M, max 47.0M, 41.1M free.
+   Apr 10 19:55:43 kernel systemd-journald[300]: Missed 2233 kernel messages
+   Apr 10 19:55:43 kernel systemd[1]: Started Journal Service.
+
+.. contents::
+   :local:
+   :depth: 1
+ 
+Prerequisites
+*************
+
+* `systemd-journald` version 245 and higher 
+
+Enable journaling of kernel boot messages
+*****************************************
+
+#. Open a terminal window.
+
+#. Create a base journald configuration file.
+
+   .. code-block:: bash
+
+      sudo mkdir -p /etc/systemd/journald.conf.d
+      sudo cp /usr/lib/systemd/journald.conf.d/clear.conf /etc/systemd/journald.conf.d/
+
+#. Append :command:`BootKMsg=true` to it.
+   
+   .. code-block:: bash
+
+      echo "BootKMsg=true" | sudo tee -a /etc/systemd/journald.conf.d/clear.conf
+
+#. Reboot.  
+
+.. tip::
+
+   If you need to increase the kernel buffer length (for example, 1M), do this: 
+
+   .. code-block:: bash
+   
+      sudo mkdir -p /etc/kernel/cmdline.d/
+      echo "log_buf_len=1M" | sudo tee /etc/kernel/cmdline.d/log_buf_len.conf
+      sudo clr-boot-manager update
+
+Alternative
+***********
+
+An alternative is to use :command:`dmesg`.  
+
+.. code-block:: bash
+
+   sudo dmesg 

source/guides/kernel/kernel-modules-dkms.rst

@@ -64,14 +64,16 @@ The :command:`kernel-native-dkms` bundle also:
    kernel. This is especially important for systems where a successful boot
    relies on a kernel module.
 
+.. _kernel-modules-dkms-install-begin-alt:
+
 Install the :command:`kernel-native-dkms` or :command:`kernel-lts-dkms`
-bundle:
+bundle.
 
 #. Determine which kernel variant is running on |CL|. Only the *native*
    and *lts* kernels are enabled to build and load out-of-tree kernel modules
    with DKMS.
 
-   .. code-block:: bash
+   .. code-block:: console
 
       $ uname -r
       5.XX.YY-ZZZZ.native
@@ -93,7 +95,8 @@ bundle:
       sudo swupd bundle-add kernel-lts-dkms
 
 
-#. Update the |CL| bootloader and reboot.
+#. Update the |CL| bootloader and reboot, and 
+   ensure that you can start the new kernel.
 
    .. code-block:: bash
 
@@ -225,9 +228,9 @@ The instructions below show a generic example:
 #. Create or modify the :file:`dkms.conf` file inside of the extracted source
    code directory.
 
-   .. code-block:: bash
+   .. code-block:: ShellSession
 
-      $EDITOR dkms.conf
+      $ EDITOR dkms.conf
 
       MAKE="make -C src/ KERNELDIR=/lib/modules/${kernelver}/build"
       CLEAN="make -C src/ clean"

source/guides/maintenance/configure-hugepages.rst

@@ -23,7 +23,7 @@ to add hugepages to the system and how to change the default hugepage size.
       
       The active option is enclosed in brackets. In this case, always is active, 
       which means hugepages are enabled for every process. The `madvise` 
-      option means that hugepages are enabled for processes that explicitely 
+      option means that hugepages are enabled for processes that explicitly 
       call `madvise`_. 
 
 #. To check the size of hugepages, run the below command.
@@ -71,4 +71,4 @@ to add hugepages to the system and how to change the default hugepage size.
       sudo clr-boot-manager update
       sudo reboot
 
-   .. _madvise: https://linux.die.net/man/2/madvise
+   .. _madvise: https://linux.die.net/man/2/madvise

source/guides/maintenance/cpu-performance.rst

@@ -203,11 +203,10 @@ Better thermal control and performance can be achieved by providing platform
 specific configuration to :command:`thermald`.
 
 `Linux DPTF Extract Utility`_ is a companion tool to :command:`thermald`,
-This tool uses Intel® 
-:abbr:`DPTF (Dynamic Platform and Thermal Framework)` technology and
-can convert to the :file:`thermal_conf.xml` configuration format used by
-:command:`thermald`. Closed-source projects, like this one, cannot be packaged
-as a bundle in |CL|, so you must install it manually:
+This tool uses Intel® Dynamic Platform and Thermal Framework (Intel® DPTF)
+technology and can convert to the :file:`thermal_conf.xml` configuration format
+used by :command:`thermald`. Closed-source projects, like this one, cannot be
+packaged as a bundle in |CL|, so you must install it manually:
 
 #. Make sure your machine's BIOS has DPTF feature and is enabled. It will usually be in the :guilabel:`Advanced` or :guilabel:`Advanced>Power` section of the BIOS. 
 
@@ -248,15 +247,13 @@ The following output means the configuration has already been applied:
 
    thermald[*]: [WARN]Using generated /etc/thermald/thermal-conf.xml.auto
 
-.. admonition:: Disclaimer
+*Intel® Turbo Boost Technology requires a PC with a processor with Intel Turbo
+Boost Technology capability. Intel Turbo Boost Technology performance varies
+depending on hardware, software and overall system configuration. Check with
+your PC manufacturer on whether your system delivers Intel Turbo Boost Technology.
+For more information, see http://www.intel.com/technology/turboboost*
 
-   Intel® Turbo Boost Technology requires a PC with a processor with Intel
-   Turbo Boost Technology capability. Intel Turbo Boost Technology performance
-   varies depending on hardware, software and overall system configuration.
-   Check with your PC manufacturer on whether your system delivers Intel Turbo
-   Boost Technology. For more information, see http://www.intel.com/technology/turboboost
-
-   Intel SpeedStep is a trademark of Intel Corporation or its subsidiaries.
+*Intel, Intel SpeedStep, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
 
 
 .. _`Intel P-state driver`: https://www.kernel.org/doc/Documentation/cpu-freq/intel-pstate.txt

source/guides/maintenance/developer-workstation.rst

@@ -78,9 +78,6 @@ tools you need to start. Consider these profiles as a starting point.
          * - Work with deep learning and edge-optimized models.
            - `computer-vision-models <https://clearlinux.org/software/bundle/computer-vision-models/>`_
 
-         * - Basic OpenVINO™ toolkit.
-           - `computer-vision-openvino <https://clearlinux.org/software/bundle/computer-vision-openvino/>`_
-
          * - API helper for cloud access.
            - `cloud-api <https://clearlinux.org/software/bundle/cloud-api/>`_
 

source/guides/maintenance/fix-broken-install.rst

@@ -37,7 +37,8 @@ Boot a live desktop image to fix target system
 Mount root partition, verify, and fix
 *************************************
 
-#. Ensure the system is connected to the network.
+#. Ensure the system is connected to the Internet in order to access the 
+   the |CL| update server.
 
 #. Open a terminal window.
 
@@ -51,9 +52,8 @@ Mount root partition, verify, and fix
    Example output:
 
    .. code-block:: console
-      :emphasize-lines: 10
+      :emphasize-lines: 9 
 
-      clrlinux@clr-live~ $ lsblk -po NAME,SIZE,LABEL,PARTTYPE,PARTLABEL
       NAME          SIZE LABEL       PARTTYPE                             PARTLABEL
       /dev/loop0  643.6M                                                  
       /dev/sda     14.3G CLR_ISO                                          
@@ -73,7 +73,7 @@ Mount root partition, verify, and fix
       sudo mount /dev/sdb3 /mnt
 
 #. Verify that you mounted the correct root partition by verifying the content
-   of ``/usr/lib/os-release`` looks similar to the example below.  
+   of ``/mnt/usr/lib/os-release`` looks similar to the example below.  
 
    .. code-block:: bash
 
@@ -83,7 +83,6 @@ Mount root partition, verify, and fix
 
    .. code-block:: console
 
-      clrlinux@clr-live~ $ cat /mnt/usr/lib/os-release 
       NAME="Clear Linux OS"
       VERSION=1
       ID=clear-linux-os
@@ -96,15 +95,15 @@ Mount root partition, verify, and fix
       BUG_REPORT_URL="mailto:dev@lists.clearlinux.org"
       PRIVACY_POLICY_URL="http://www.intel.com/privacy"
 
-#. Next, run :command:`swupd` to fix any issues on the target system.
+#. Next, run :command:`swupd repair` to fix any issues on the target system.
 
    .. code-block:: bash
 
-      sudo swupd repair --picky --path=/mnt
+      sudo swupd repair --picky --path=/mnt --statedir=/mnt/var/lib/swupd
 
    :ref:`Learn more about how swupd works <swupd-guide>`.
 
-#. After the process is complete, unmount the root partition:
+#. After the process is complete, unmount the root partition.
 
    .. code-block:: bash
 

source/guides/network/proxy.rst

@@ -0,0 +1,116 @@
+.. _proxy:
+
+Proxy Configuration
+###################
+
+When working behind a corporate proxy server, one typically has to configure
+proxy settings for applications to reach the Internet.  |CL-ATTR| has 
+implemented an :ref:`autoproxy` feature to try and eliminate manual 
+configurations as much as possible.  However, there are still some applications 
+that cannot take full advantage of the :ref:`autoproxy` feature due to their 
+own ways of configuring proxy settings.  This guide shows you how to configure 
+proxy settings for some of the known applications manually.  
+
+.. contents::
+    :local:
+    :depth: 1
+
+Prerequisites
+*************
+
+* You have installed |CL| on your host system.
+
+  For detailed instructions on installing |CL| on a bare metal system, visit
+  the :ref:`bare metal installation guide <bare-metal-install-desktop>`.
+
+General proxy settings for many applications
+============================================
+
+#. First, apply these general proxy settings which should work for many 
+   applications.  If they do not work for a specific application, such as the 
+   ones listed below, apply application-specific proxy settings as needed.
+
+   Proxy settings:
+
+   .. code-block:: none
+
+      export http_proxy=http://<YOUR.HTTP-PROXY.URL:PORT>
+      export https_proxy=http://<YOUR.HTTPS-PROXY.URL:PORT>
+      export ftp_proxy=http://<YOUR.FTP-PROXY.URL:PORT>
+      export socks_proxy=http://<YOUR.SOCKS-PROXY.URL:PORT>
+      export no_proxy="<YOUR-DOMAIN>,localhost"
+      export HTTP_PROXY=$http_proxy
+      export HTTPS_PROXY=$https_proxy
+      export FTP_PROXY=$ftp_proxy
+      export SOCKS_PROXY=$socks_proxy
+      export NO_PROXY=$no_proxy
+
+   * *User-specific*, put them in :file:`$HOME/.bashrc`.
+
+   * *For all users*, put them in :file:`/etc/profile.d/proxy.conf`. 
+
+#. For the proxies to take effect, either :command:`source` the file manually
+   or log out and log back in.
+
+Docker\*
+========
+
+Please refer the official Docker links on how to configure proxies:
+
+* `Docker client`_
+* `Docker daemon`_
+
+git over SSH
+============
+
+Add the following to your :file:`~/.ssh/config` file:
+
+.. code-block:: none
+
+   host github.com
+        port 22
+        user git
+        ProxyCommand connect-proxy -S <YOUR.SSH-PROXY.URL:PORT> %h %p
+
+.. note::
+
+   Though :command:`netcat` is included with |CL|, it is not the BSD version,
+   which is the one usually used to enable git over SSH.
+
+autospec/mock
+=============
+
+:ref:`autospec` uses mock to do builds.  Configure mock's proxy settings with
+these steps:  
+
+#. Override the general mock configuration file with a custom one, otherwise
+   your settings will get overwritten each time autospec is updated.
+
+   .. code-block:: bash
+
+      sudo mkdir -p /etc/mock
+      sudo cp ~/clearlinux/projects/common/conf/clear.cfg /etc/mock/clear-custom.cfg
+
+#. :command:`sudoedit` :file:`/etc/mock/clear-custom.cfg` and add the highlighted
+   lines.
+
+   .. code-block:: none
+      :emphasize-lines: 3-5
+
+      ...
+      config_opts['use_bootstrap_container'] = False
+      config_opts['http_proxy'] = '<YOUR.HTTP.PROXY.URL>:<PORT>'
+      config_opts['https_proxy'] = '<YOUR.HTTPS.PROXY.URL>:<PORT>'
+      config_opts['no_proxy'] = '<YOUR.DOMAIN>,192.168.0.0/16,localhost,127.0.0.0/8'
+
+Kubernetes
+==========
+
+See :ref:`Setting proxy servers for Kubernetes <kubernetes>`.
+
+.. _Docker client:
+   https://docs.docker.com/network/proxy/#configure-the-docker-client
+
+.. _Docker daemon:
+   https://docs.docker.com/config/daemon/systemd/#httphttps-proxy
+

source/guides/network/vnc.rst

@@ -639,7 +639,7 @@ For Method 2:
 
    .. code-block:: bash
 
-      sudo systemctl daemon-load
+      sudo systemctl daemon-reload
       sudo systemctl restart vncserver@:5.service
 
 For Method 3:

source/guides/stacks/dars.rst

@@ -1,701 +0,0 @@
-.. _dars:
-
-Data Analytics Reference Stack
-##############################
-
-This guide explains how to use the :abbr:`DARS (Data Analytics Reference Stack)`, and to optionally build your own DARS container image.
-
-Any system that supports Docker\* containers can be used with DARS. The steps
-in this guide use |CL-ATTR| as the host system.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Overview
-********
-
-The Data Analytics Reference Stack (DARS) provides developers and enterprises a straightforward, highly optimized software stack for storing and processing large amounts of data.  More detail is available on the `DARS architecture and performance benchmarks`_.
-
-Stack Features
-==============
-
-The Data Analytics Reference Stack provides two pre-built Docker images,
-available on `Docker Hub`_:
-
-* A |CL|-derived `DARS with OpenBlas`_ stack optimized for `OpenBLAS`_
-* A |CL|-derived  `DARS with Intel® MKL`_ stack optimized for `MKL`_ (Intel® Math Kernel Library)
-
-We recommend you view the latest component versions for each image in the
-:file:`releasenote` found in the `Data Analytics Reference Stack`_ GitHub\*
-repository. Because |CL| is a rolling distribution, the package version numbers
-in the |CL|-based containers may not be the latest released by |CL|.
-
-.. note::
-
-   The Data Analytics Reference Stack is a collective work, and each piece
-   of software within the work has its own license.  Please see the
-   `DARS Terms of Use`_ for more details about licensing and usage of the Data
-   Analytics Reference Stack.
-
-Using the Docker images
-***********************
-
-Launching the Image
-===================
-
-#. To use the latest stable DARS images, pull an image
-   directly from `Docker Hub`_. This example uses the
-   `DARS with Intel® MKL`_ Docker image.
-
-   .. code-block:: bash
-
-      docker pull clearlinux/stacks-dars-mkl
-
-
-#. Once you have downloaded the image, you can run it with this command, which will launch the image and drop you into a bash shell inside the container.
-
-   .. code-block:: bash
-
-      docker run -it --ulimit nofile=1000000:1000000 --name mkl --network host --rm -i -t <name-of-image>
-
-
-   Command Flags
-
-   :command:`--ulimit nofile=` is required in order to increase the allowed number of open files for the Apache Spark\* engine.
-
-   :command:`--name` can be any name of your choice.  This guide is using `mkl`
-
-   :command:`--network host` enables the host machine's IP address to be used to access the container.
-
-   If you need to verify the name of the DARS image for the <name-of-image> flag, you can use the :command:`docker image ls` command to see which images reside on your system.
-
-   .. code-block:: bash
-
-      docker image ls
-
-
-   .. code-block:: console
-
-      REPOSITORY                                                   TAG                 IMAGE ID            CREATED             SIZE
-      clearlinux/stacks-dars-mkl                                   test-img            49a70a22231f        23 hours ago        2.66GB
-      ubuntu                                                       latest              2ca708c1c9cc        7 days ago          64.2MB
-      katadocker/kata-deploy                                       latest              bd6dc92f8060        7 days ago          673MB
-      clearlinux/stacks-dars-mkl                                   latest              2c9555536d5f        4 weeks ago         2.62GB
-
-
-
-
-.. note::
-
-   All of the DARS components are compiled on Open JDK11\*. The container will have preinstalled JDK11 at :file:`/usr/lib/jvm/java-1.11.0-openjdk/` and it has been set as the default Java version. While the DARS containers also contain Open JDK8, it is not covered in this guide.
-
-
-Building DARS images
-====================
-
-If you choose to build your own DARS container images, you can customize them as needed. Use the :file:`Dockerfile` included in the Github\* repository as your baseline.
-
-To construct images with |CL|, start with a |CL| development platform that has the :command:`containers-basic-dev` bundle installed. Learn more about bundles and installing them by using :ref:`swupd-guide`.
-
-#. The `Data Analytics Reference Stack`_ is part of the Intel® stacks GitHub\* repository. Clone the :file:`stacks` repository.
-
-   .. code-block:: bash
-
-      git clone https://github.com/intel/stacks.git
-
-#. Inside the :file:`stacks/dars/clearlinux/mkl` directory, use docker with the :file:`Dockerfile` to build the  MKL image.
-
-   .. code-block:: bash
-
-      cd ./stacks/dars/clearlinux/mkl
-      docker build --no-cache -t clearlinux/stacks-dars-mkl .
-
-
-#. Once completed, check the resulting images with :command:`Docker`
-
-   .. code-block:: bash
-
-       docker images | grep dars
-
-#. You can use any of the resulting images to launch fully functional containers. If you need to customize the containers, you can edit the provided :file:`Dockerfile`.
-
-.. note::
-
-   The environment variables for Apache Hadoop* and Apache Spark have been configured in the Dockerfile for the DARS container. For Apache Hadoop\* use :file:`/etc/hadoop` as `HADOOP_CONF_DIR` folder. For Apache Spark use :file:`/etc/spark` as `SPARK_CONF_DIR` folder.
-
-
-Using Apache Spark\* in DARS
-****************************
-
-After launching the container, you can start Apache Spark with either the Scala or PySpark environment.  For these examples we will use PySpark, which is the Python\* API for Apache Spark.
-
-.. code-block:: bash
-
-  pyspark
-
-
-Launching is as simple as this.  Depending on your system configuration and capabilities, you may need to define proxy or memory allocation settings on the command line or in a config file for optimal performance. Refer to the `Apache Spark documentation`_ for more detail.
-
-After executing :command:`pyspark`, you will see output similar to this.
-
-.. code-block:: console
-
-  root@fd5155b89857 /root # pyspark
-    Welcome to
-        ____              __
-       / __/__  ___ _____/ /__
-       _\ \/ _ \/ _ `/ __/  '_/
-      /__ / .__/\_,_/_/ /_/\_\   version 2.4.0
-         /_/
-
-    Using Python version 3.7.4 (default, Jul 13 2019 06:59:17)
-    SparkSession available as 'spark'.
-    >>>
-
-
-Execute code directly in PySpark
-================================
-
-A simple example for verifying that pyspark is working correctly is to run a small python function from a `PySpark getting started guide`_ to estimate the value of Pi. Run these lines in the PySpark shell.
-
-.. code-block:: console
-
-   import random
-   NUM_SAMPLES = 100000000
-   def inside(p):
-    x, y = random.random(), random.random()
-    return x*x + y*y < 1
-
-   count = sc.parallelize(range(0, NUM_SAMPLES)).filter(inside).count()
-   pi = 4 * count / NUM_SAMPLES
-   print(“Pi is roughly”, pi)
-
-
-Run Python programs with spark-submit
-=====================================
-
-You can also run python scripts in Apache Spark from the command line.  We'll use the Apache Spark example found in the :file:`/usr/share/apache-spark/examples/src/main/python/pi.py` file.  Note that we have turned off the INFO and WARN messages in Apache Spark for this example.
-
-.. code-block:: console
-
-   #spark-submit /usr/share/apache-spark/examples/src/main/python/pi.py
-   Config directory: /usr/share/defaults/spark/
-   Pi is roughly 3.134700
-
-DARS Usecase example
-====================
-
-The DARS container is used in conjunction with the Deep Learning Reference Stack container to implement a real world use case.  Refer to the `Github Issue Classification`_ Usecase found in the `stacks-usecase`_ repository for a walkthrough.  This usecase is implemented using the Scala environment, rather than PySpark.
-
-
-Using Apache Hadoop in DARS
-***************************
-
-Apache Hadoop is an open source framework allowing for distributed processing of large data sets across clusters of computers using simple programming models. This framework is designed to scale up from a few servers to thousands of machines, each offering local computation and storage.
-
-Single Node Hadoop Cluster Setup
-================================
-
-In this mode, all the daemons involved (e.g., the DataNode, NameNode, TaskTracker, JobTracker) run as Java processes on the same machine. This setup is useful for developing and testing Apache Hadoop applications.
-
-The components of an Apache Hadoop Cluster are described below:
-
-* NameNode manages HDFS storage. HDFS exposes a filesystem namespace and allows user data to be stored in files. Internally a file is split into one or more blocks and these blocks are stored in a set of DataNodes.
-* DataNode is also known as Slave node. It is responsible for storing and managing the data in that node and responds to the NameNode for all filesystem operations.
-* JobTracker is a master which creates and runs the job through tasktrackers. It also tracks resource availability and task lifecycle management.
-* TaskTracker manages the processing resources on each worker node and send status updates to the JobTracker periodically.
-
-
-Configuration
-=============
-
-#. To setup a single node cluster, run a DARS container with the following flags:
-
-   .. code-block:: bash
-
-      docker run --ulimit nofile=1000000:1000000 -ti --rm --network host clearlinux/stacks-dars-mkl cp -r -n /usr/share/defaults/hadoop/* /etc/hadoop
-
-#. In the running container, set configuration in the :file:`/etc/hadoop/mapred-site.xml` file
-
-   .. code-block:: xml
-
-      <configuration>
-          <property>
-              <name>mapreduce.framework.name</name>
-              <value>yarn</value>
-          </property>
-
-          <property>
-              <name>yarn.app.mapreduce.am.env</name>
-              <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
-          </property>
-
-          <property>
-              <name>mapreduce.map.env</name>
-              <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
-          </property>
-
-          <property>
-              <name>mapreduce.reduce.env</name>
-              <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
-          </property>
-       </configuration>
-
-#. Set up the :file:`/etc/hadoop/yarn-site.xml` as follows
-
-   .. code-block:: xml
-
-      <configuration>
-          <property>
-              <name>yarn.nodemanager.aux-services</name>
-              <value>mapreduce_shuffle</value>
-          </property>
-
-          <property>
-              <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
-              <value>org.apache.hadoop.mapred.ShuffleHandler</value>
-          </property>
-      </configuration>
-
-Start the Apache Hadoop daemons
-===============================
-
-#. Format the NameNode server using this command:
-
-   .. code-block:: bash
-
-      hdfs namenode -format
-
-#. Start the Apache Hadoop services
-
-   HDFS Namenode service :
-
-   .. code-block:: bash
-
-      hdfs --daemon start namenode
-
-
-   HDFS Datanode service :
-
-   .. code-block:: bash
-
-      hdfs --daemon start datanode
-
-
-   Yarn ResourceManager :
-
-   .. code-block:: bash
-
-      yarn --daemon start resourcemanager
-
-
-   Yarn NodeManager :
-
-   .. code-block:: bash
-
-      yarn --daemon start nodemanager
-
-
-   jobhistory service :
-
-   .. code-block:: bash
-
-      mapred --daemon start historyserver
-
-#. Verify the nodes are alive with this command:
-
-
-   .. code-block:: bash
-
-      yarn node -list 2
-
-   Your output will look similar to:
-
-   .. code-block:: console
-
-      Total Nodes:1
-         Node-Id             Node-State Node-Http-Address       Number-of-Running-Containers
-      <hostname>:43489            RUNNING <hostname>:8042                      0
-
-
-Example application
-===================
-
-Apache Hadoop comes packages with a set of example  applications. In this example we will show how to use the cluster to calculate Pi. The JAR file containing the compiled class can be found on your running DARS container at :file:`/usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar`
-
-
-.. code-block:: bash
-
-   hadoop jar /usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-$(hadoop version | grep Hadoop | cut -d ' ' -f2).jar pi 16 100
-
-
-Deploy DARS on Kubernetes\*
-***************************
-
-Many containerized workloads are deployed in clusters managed by orchestration software like Kubernetes.
-
-Prerequisites
-=============
-
-* A running Kubernetes cluster at version >= 1.6 with access configured to it using kubectl.
-* You must have appropriate permissions to list, create, edit and delete pods in your cluster.
-* The service account credentials used by the driver pods must be allowed to create pods, services and configmaps.
-* You must have Kubernetes DNS configured in your cluster.
-
-.. note::
-
-  To ensure that Kubernetes is correctly installed and configured for |CL|, follow the instructions in :ref:`kubernetes`.
-
-
-#. For this example we will create the following Dockerfile
-
-   .. code-block:: bash
-
-        cat > $(pwd)/Dockerfile << 'EOF'
-        ARG DERIVED_IMAGE
-        FROM ${DERIVED_IMAGE}
-
-        RUN mkdir -p /etc/passwd /etc/pam.d /opt/spark/conf /opt/spark/work-dir
-
-        RUN set -ex && \
-            rm /bin/sh && \
-            ln -sv /bin/bash /bin/sh && \
-            touch /etc/pam.d/su \
-            echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \
-            chgrp root /etc/passwd && chmod ug+rw /etc/passwd
-
-        RUN ln -s /usr/share/apache-spark/jars/ /opt/spark/ && \
-            ln -s /usr/share/apache-spark/bin/ /opt/spark/ && \
-            ln -s /usr/share/apache-spark/sbin/ /opt/spark/ && \
-            ln -s /usr/share/apache-spark/examples/ /opt/spark/ && \
-            ln -s /usr/share/apache-spark/kubernetes/tests/ /opt/spark/ && \
-            ln -s /usr/share/apache-spark/data/ /opt/spark/ && \
-            ln -s /etc/spark/* /opt/spark/conf/
-
-        COPY entrypoint.sh /opt/
-        ENV JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk
-        ENV PATH="${JAVA_HOME}/bin:${PATH}"
-        ENV SPARK_HOME /opt/spark
-        WORKDIR /opt/spark/work-dir
-        ENTRYPOINT [ "/opt/entrypoint.sh" ]
-        EOF
-
-
-#. Create the :file:`entrypoint.sh` file. The Dockerfile requires an entrypoint script, to allow spark-submit to interact with the container.
-
-   .. code-block:: bash
-
-        cat > $(pwd)/entrypoint.sh << 'EOF'
-        #!/bin/bash
-        #
-        # Licensed to the Apache Software Foundation (ASF) under one or more
-        # contributor license agreements.  See the NOTICE file distributed with
-        # this work for additional information regarding copyright ownership.
-        # The ASF licenses this file to You under the Apache License, Version 2.0
-        # (the "License"); you may not use this file except in compliance with
-        # the License.  You may obtain a copy of the License at
-        #
-        #    http://www.apache.org/licenses/LICENSE-2.0
-        #
-        # Unless required by applicable law or agreed to in writing, software
-        # distributed under the License is distributed on an "AS IS" BASIS,
-        # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-        # See the License for the specific language governing permissions and
-        # limitations under the License.
-        #
-
-        # echo commands to the terminal output
-        set -ex
-
-        # Check whether there is a passwd entry for the container UID
-        myuid=$(id -u)
-        mygid=$(id -g)
-        # turn off -e for getent because it will return error code in anonymous uid case
-        set +e
-        uidentry=$(getent passwd $myuid)
-        set -e
-
-        # If there is no passwd entry for the container UID, attempt to create one
-        if [ -z "$uidentry" ] ; then
-            if [ -w /etc/passwd ] ; then
-                echo "$myuid:x:$myuid:$mygid:anonymous uid:$SPARK_HOME:/bin/false" >> /etc/passwd
-            else
-                echo "Container ENTRYPOINT failed to add passwd entry for anonymous UID"
-            fi
-        fi
-
-        SPARK_K8S_CMD="$1"
-        case "$SPARK_K8S_CMD" in
-            driver | driver-py | driver-r | executor)
-              shift 1
-              ;;
-            "")
-              ;;
-            *)
-              echo "Non-spark-on-k8s command provided, proceeding in pass-through mode..."
-              exec /sbin/tini -s -- "$@"
-              ;;
-        esac
-
-        SPARK_CLASSPATH="$SPARK_CLASSPATH:${SPARK_HOME}/jars/*"
-        env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt
-        readarray -t SPARK_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt
-
-        if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then
-          SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH"
-        fi
-
-        if [ -n "$PYSPARK_FILES" ]; then
-            PYTHONPATH="$PYTHONPATH:$PYSPARK_FILES"
-        fi
-
-        PYSPARK_ARGS=""
-        if [ -n "$PYSPARK_APP_ARGS" ]; then
-            PYSPARK_ARGS="$PYSPARK_APP_ARGS"
-        fi
-
-        R_ARGS=""
-        if [ -n "$R_APP_ARGS" ]; then
-            R_ARGS="$R_APP_ARGS"
-        fi
-
-        if [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "2" ]; then
-            pyv="$(python -V 2>&1)"
-            export PYTHON_VERSION="${pyv:7}"
-            export PYSPARK_PYTHON="python"
-            export PYSPARK_DRIVER_PYTHON="python"
-        elif [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "3" ]; then
-            pyv3="$(python3 -V 2>&1)"
-            export PYTHON_VERSION="${pyv3:7}"
-            export PYSPARK_PYTHON="python3"
-            export PYSPARK_DRIVER_PYTHON="python3"
-        fi
-
-        case "$SPARK_K8S_CMD" in
-          driver)
-            CMD=(
-              "$SPARK_HOME/bin/spark-submit"
-              --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
-              --deploy-mode client
-              "$@"
-            )
-            ;;
-          driver-py)
-            CMD=(
-              "$SPARK_HOME/bin/spark-submit"
-              --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
-              --deploy-mode client
-              "$@" $PYSPARK_PRIMARY $PYSPARK_ARGS
-            )
-            ;;
-            driver-r)
-            CMD=(
-              "$SPARK_HOME/bin/spark-submit"
-              --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
-              --deploy-mode client
-              "$@" $R_PRIMARY $R_ARGS
-            )
-            ;;
-          executor)
-            CMD=(
-              ${JAVA_HOME}/bin/java
-              "${SPARK_EXECUTOR_JAVA_OPTS[@]}"
-              -Xms$SPARK_EXECUTOR_MEMORY
-              -Xmx$SPARK_EXECUTOR_MEMORY
-              -cp "$SPARK_CLASSPATH"
-              org.apache.spark.executor.CoarseGrainedExecutorBackend
-              --driver-url $SPARK_DRIVER_URL
-              --executor-id $SPARK_EXECUTOR_ID
-              --cores $SPARK_EXECUTOR_CORES
-              --app-id $SPARK_APPLICATION_ID
-              --hostname $SPARK_EXECUTOR_POD_IP
-            )
-            ;;
-
-          *)
-            echo "Unknown command: $SPARK_K8S_CMD" 1>&2
-            exit 1
-        esac
-
-        # Execute the container CMD
-        exec "${CMD[@]}"
-        EOF
-
-
-#. Make :file:`entrypoint.sh` executable
-
-   .. code-block:: bash
-
-      sudo chmod +x $(pwd)/entrypoint.sh
-
-#. Build the Docker image, for this example we will use dars_k8s_spark for the name of the image.
-
-   .. code-block:: bash
-
-      docker build . --build-arg DERIVED_IMAGE=clearlinux/stacks-dars-mkl -t dars_k8s_spark
-
-
-#. Verify your built image. Execute the following command looking for the given name dars_k8s_spark
-
-   .. code-block:: bash
-
-     docker images | grep "dars_k8s_spark"
-
-   You should see something like:
-
-   .. code-block:: console
-
-     dars_k8s_spark                               latest              1fa3278a3421        1 minutes ago       6.56GB
-
-#. Use a variable to store the image's given name:
-
-   .. code-block:: bash
-
-     DARS_K8S_IMAGE=dars_k8s_spark
-
-
-Configure RBAC
-==============
-
-Create the Spark service account and cluster role binding to allow Spark on Kubernetes to create Executors as required. For this example use the default namespace.
-
-.. code-block:: bash
-
-   kubectl create serviceaccount spark-serviceaccount --namespace default
-   kubectl create clusterrolebinding spark-rolebinding --clusterrole=edit --serviceaccount=default:spark-serviceaccount --namespace=default
-
-
-Prepare to Submit the Spark Job
-===============================
-
-#. Determine the Kubernetes master address:
-
-   .. code-block:: bash
-
-      kubectl cluster-info
-
-   You should see something like:
-
-   .. code-block:: console
-
-      Kubernetes master is running at https://192.168.39.127:8443
-
-#. Use a variable to store the master address:
-
-   .. code-block:: bash
-
-      MASTER_ADDRESS='https://192.168.39.127:8443'
-
-#. Submit the Spark Job on Minikube using the MASTER_ADDRESS and DARS_K8S variables. The driver pod will be called spark-pi-driver.
-
-   .. code-block:: bash
-
-      spark-submit \
-      --master k8s://${MASTER_ADDRESS} \
-      --deploy-mode cluster \
-      --name spark-pi \
-      --class org.apache.spark.examples.SparkPi \
-      --conf spark.executor.instances=2 \
-      --conf spark.kubernetes.container.image=${DARS_K8S_IMAGE} \
-      --conf spark.kubernetes.driver.pod.name=spark-pi-driver \
-      --conf spark.kubernetes.namespace=default \
-      --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark-serviceaccount \
-      local:///usr/share/apache-spark/examples/jars/spark-examples_2.12-2.4.0.jar
-
-
-#. Check the Job. Read the logs and look for the Pi result:
-
-   .. code-block:: bash
-
-      kubectl logs spark-pi-driver | grep "Pi is roughly"
-
-   You should see something like:
-
-   .. code-block:: console
-
-      Pi is roughly 3.1418957094785473
-
-More information about spark-submit configuration is available in the  `running-on-kubernetes`_ documentation.
-
-
-Troubleshooting
-***************
-
-Dropped or refused connection
-=============================
-
-If Pyspark / Spark-shell warns of a dropped connection exception or Connection refused, check if the `HADOOP_CONF_DIR` environment variable is set. These APIs assume they will use Hadoop Distributed File System.
-You can unset `HADOOP_CONF_DIR` and use Spark RDDs, or start Hadoop services and then create your directories and files as required using hdfs.
-
-It is also possible to change the file system to local without unsetting `HADOOP_CONF_DIR` using one of these commands.
-
-.. code-block:: bash
-
-   pyspark --conf "spark.hadoop.fs.defaultFS=file:///"
-
-.. code-block:: bash
-
-   spark-shell --conf "spark.hadoop.fs.defaultFS=file:///"
-
-Using Spark with proxy settings
-===============================
-
-There are two ways to work with proxies:
-
-#. Add the following line to  :file:`$SPARK_CONF_DIR/spark-defaults.conf` for both `spark.executor.extraJavaOptions` and `spark.driver.extraJavaOptions` variables:
-
-.. code-block:: console
-
-   -Dhttp.proxyHost=<URL> -Dhttp.proxyPort=<PORT> -Dhttps.proxyHost=<URL> -Dhttps.proxyPort=<PORT>
-
-
-
-#. Give the proxies URL and Port as a configuration parameter
-
-.. code-block:: bash
-
-   pyspark --conf "spark.hadoop.fs.defaultFS=file:///" --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=example.proxy -Dhttp.proxyPort=111 -Dhttps.proxyHost=example.proxy -Dhttps.proxyPort=112"
-
-.. code-block:: bash
-
-   spark-shell --conf "spark.hadoop.fs.defaultFS=file:///" --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=example.proxy -Dhttp.proxyPort=111 -Dhttps.proxyHost=example.proxy -Dhttps.proxyPort=112"
-
-
-Known issues
-============
-
-#. There is an exception message `Unrecognized Hadoop major version number: 3.2.0 at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion.`
-
-This exception can be disregarded because DARS does not use hadoop.hive.shims. Hive binaries installed from Apache on |CL| with JDK11 does not work at the time of this writing.
-
-#. There is an exception message `Exception in thread "Thread-3" java.lang.ExceptionInInitializerError at org.apache.hadoop.hive.conf.HiveConf` This is related to the same issue with |CL| and JDK11 noted above, and does not affect DARS for the same reason.
-
-
-.. _Data Analytics Reference Stack: https://github.com/intel/stacks/tree/master/dars/clearlinux
-
-.. _Docker Hub: https://hub.docker.com/
-
-.. _OpenBLAS: http://www.openblas.net/
-
-.. _MKL: https://software.intel.com/en-us/mkl
-
-.. _CentOS: https://www.centos.org/
-
-.. _DARS with OpenBLAS: https://hub.docker.com/r/clearlinux/stacks-dars-openblas/
-
-.. _DARS with Intel® MKL: https://hub.docker.com/r/clearlinux/stacks-dars-mkl/
-
-.. _DARS architecture and performance benchmarks: https://clearlinux.org/stacks/data-analytics-stack-v1
-
-.. _DARS Terms of Use: https://clearlinux.org/stacks/data-analytics/terms-of-use
-
-.. _PySpark getting started guide: https://towardsdatascience.com/how-to-get-started-with-pyspark-1adc142456ec
-
-.. _Apache Spark documentation: https://spark.apache.org/docs/latest/
-
-.. _stacks-usecase: https://github.com/intel/stacks-usecase
-
-.. _Github Issue Classification: https://github.com/intel/stacks-usecase/tree/master/github-issue-classification
-
-.. _running-on-kubernetes: https://spark.apache.org/docs/latest/running-on-kubernetes.html#configuration

source/guides/stacks/dbrs.rst

@@ -1,712 +0,0 @@
-.. _dbrs:
-
-Database Reference Stack
-########################
-
-This guide describes the hardware and installation requirements for using the
-:abbr:`DBRS (Database Reference Stack)`, along with getting started configuration examples, using |CL-ATTR| as the host system.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Overview
-********
-
-The Database Reference Stack is integrated, highly-performant, open source,
-and optimized for 2nd generation Intel® Xeon® Scalable Processors and Intel®
-Optane™ persistent memory. This open source community release is part of
-an effort to ensure developers have easy access to the features and
-functionality of Intel Platforms.
-
-Stack Features
-==============
-
-Current supported  database applications are Apache Cassandra* and Redis*, which
-have been enabled for `Intel Optane PMM`_.
-
-DBRS with Apache Cassandra can be deployed as a standalone container or inside a
-Kubernetes* cluster.
-
-The Redis stack application is enabled for a multinode Kubernetes
-environment, using AEP PMem DIMM in fsdax mode for storage.
-
-Releases
-********
-
-Refer to the `Database Reference Stack website`_ for information and download links for the different versions and offerings of the stack.
-
-The release announcement for each release provides more detail about the stack features, as well as benchmark results.
-
-* `DBRS V2.0`_ release announcement.
-* `DBRS V1.0`_ release announcement.
-
-
-.. note::
-
-   The Database Reference Stack is a collective work, and each piece
-   of software within the work has its own license.  Please see the
-   `DBRS Terms of Use`_ for more details about licensing and usage of the Database Reference Stack.
-
-
-Hardware Requirements
-*********************
-
-* Intel® Xeon Scalable Platform with Intel® C620 chipset series
-* 2nd Gen Intel® Xeon Scalable processor CPU (Intel® Optane™ PMM-enabled stepping) Provides cache & memory control.  Intel® Optane™  PMem works only on systems powered by 2nd Generation Intel® Xeon® Platinum or Gold processors.
-* BIOS with Reference Code
-* Intel®Optane™ PMem
-
-Hardware configuration used in stacks development
-=================================================
-
-* Intel® Server System R2208WFTZSR
-* BIOS with Reference Code
-  * BIOS ID: SE5C620.86B.0D.01.0438.032620191658
-  * BMC Firmware: 1.94.6b42b91d
-  * Intel® Optane™ PMemFirmware: 1.2.0.5310
-* 2x Intel® Xeon Platinum 8268 Processor
-* Intel® SSD DC S5600 Series 960GB 2.5in SATA Drive
-* 64 GB RAM - Distributed in 4x 16 GB DDR4 DIMM's
-* 2x Intel® Optane™ PMem 256GB Module
-* 1-1-1 Layout 8 Optane™ : 1 RAM ratio
-
-
-.. list-table:: **Table 1. IMC**
-   :widths: 16,16,16,16,16,16
-   :header-rows: 1
-
-   * - Channel 2
-     - Channel 2
-     - Channel 1
-     - Channel 1
-     - Channel 0
-     - Channel 0
-
-   * - Slot 1
-     - Slot 0
-     - Slot 1
-     - Slot 0
-     - Slot 1
-     - Slot 0
-
-   * -
-     - 256 GB DCPMM
-     -
-     - 16 GB DRAM
-     -
-     - 16 GB DRAM
-
-Firmware configuration
-**********************
-
-.. important::
-
-   When updating DCPMM Firmware, all DCPMM parts must be in the same mode (you cannot mix 1LM and 2LM parts).
-
-The latest firmware download for the Intel® Server System S2600WF Family is available at the `Intel Download Center`_
-
-Firmware Update Steps
-=====================
-
-#. Unzip the contents of the update package and copy all files to the root directory of a removable media (USB flash drive).
-#. Insert the USB flash drive to any available USB port on the system to be updated.
-#. Boot to EFI shell.
-#. Input "fsx(x:0,1,...):" to enter into your usb disk
-#. Run "startup.nsh"
-#. After update BMC firmware, system BIOS, ME firmware,FD, FRUSDR, system will reboot automatically.
-
-
-If Intel® Optane™ PMem is installed, run startup.nsh a second time after the first reboot to upgrade Intel® Optane™ PMem Firmware:
-
-* Boot to EFI shell.
-* Input "fsx(x:0,1,...):" to enter into your usb disk
-* Run "startup.nsh" again to update the corresponding AEP FW.
-
-.. _dbrs-hardware-configuration:
-
-Hardware Configuration
-**********************
-
-
-Online Resources
-================
-
-Before going through the configuration steps, we strongly recommend visiting the following resources and wikis to have a broader understanding of what is being done
-
-* `Quick Start Guide`_ Configure Intel® Optane™ PMem Modules on Linux
-* `Managing NVDIMMs`_
-* `Configure, Manage, and Profile`_ Intel® Optane™ PMem Modules
-
-Optane™ DIMM Configuration
-==========================
-
-The PMem DIMMs can be configured in devdax or fsdax mode. The use case to enable database stack on a kubernetes environment currently only support fsdax mode.
-
-Configuration Steps
-===================
-
-.. important::
-
-   Run the following steps with root privileges (sudo) as shown in the examples
-
-
-#. To configure Optane™ DIMMs for App direct mode run this command
-
-   .. code-block:: bash
-
-      sudo ipmctl create -goal PersistentMemoryType=AppDirect
-
-#. Verify the Optane™ Configuration by showing the defined region, then reboot the system for your changes to take effect
-
-   .. code-block:: bash
-
-      sudo ipmctl show -region
-
-
-#. Next, list the defined namespaces for the pmem devices in the system. If they are not defined, create them as shown in the following step.
-
-
-   .. code-block:: bash
-
-      sudo ndctl list -N
-
-
-#. Create namespaces based on the regions and set mode as fsdax  -- use the names of the regions listed in previous step as the –-region parameter (default is region0 and region1; one for each CPU socket)
-
-   .. code-block:: bash
-
-      sudo ndctl create-namespace --region=region0 --mode=fsdax
-      sudo ndctl create-namespace --region=region1 --mode=fsdax
-
-
-#. Create the filesystem and mount it. We are using /mnt/dax{#} as a convention in this guide to mount our devices
-
-   .. code-block:: bash
-
-      sudo mkfs.ext4 /dev/pmem0
-      sudo mount -o dax /dev/pmem0 /mnt/dax0
-      sudo mkfs.ext4 /dev/pmem1
-      sudo mount -o dax /dev/pmem1 /mnt/dax1
-
-
-Running DBRS with Apache Cassandra*
-***********************************
-
-DBRS with Apache Cassandra can be deployed as a standalone container or inside
-Kubernetes\*. Instructions for both cases is included here. Note that you can
-use the released `Docker image with Apache Cassandra`_ (Docker\* examples below).
-These instructions provide a baseline for creating your own container image.
-If you are using the released image, skip this section.
-
-.. important::
-
-   At the initial release of DBRS, Apache Cassandra is considered to be Engineering Preview release quality and may not be suitable for production release.  Please take this into consideration when planning your project.
-
-
-
-Build the DBRS with Apache Cassandra container
-==============================================
-
-To build the container with Apache Cassandra, you must build cassandra-pmem, and then build the container using the :command:`docker build` command. We are using |CL| as our container host as well as the OS in the container.
-
-Build cassandra-pmem
-====================
-
-.. important::
-
-   At the initial release of DBRS, the pmem-csi driver is considered to be Engineering Preview release quality and may not be suitable for production release.  Please take this into consideration when planning your project.
-
-
-In the `DBRS github repository`_, there is a file called `build-cassandra-pmem.sh`_, which handles all the requirements for compiling cassandra-pmem for Dockerfile usage. The dependencies for this build can be installed with :command:`swupd`.
-
-.. code-block:: bash
-
-   sudo swupd bundle-add c-basic java-basic devpkg-pmdk pmdk
-
-
-Once installed, we run the script
-
-.. code-block:: bash
-
-   ./build-cassandra-pmem.sh
-
-
-At the completion of the build you will have a file called :file:`cassandra-pmem-build.tar.gz`. Place this file in the same directory with the Dockerfile  to build the Docker image.
-
-Build the Docker container
-==========================
-
-To build the Docker image, run the Dockerfile in the same directory with the :file:`cassandra-pmem-build.tar.gz`
-
-.. code-block:: bash
-
-   docker build --force-rm --no-cache -f Dockerfile -t $build_image_name .
-
-
-Once it completes, the Docker image is ready to be used.
-
-Deploy Apache Cassandra PMEM as a standalone container
-======================================================
-
-Requirements
-------------
-
-To deploy Apache Cassandra PMEM, you must meet the following requirements
-
-* PMEM memory must be configured in `devdax` or `fsdax`    mode. The container image is able to handle both modes and depending on the PMEM mode, the mount points inside the container must be different.
-* In order to make available `devdax` pmem devices inside the container you must use the `--device` directive. Internally the container always uses :command:`/dev/dax0.0`, so the mapping should be: :command:`--device=/dev/<host-device>:/dev/dax0.0`
-* In a similar fashion for `fsdax` we need the device to be mapped to :command:`/mnt/pmem` inside the container: :command:`--mount type=bind,source=<source-mount-point>,target=/mnt/pmem`
-
-
-Preparing PMEM for container use
---------------------------------
-
-The cassandra-pmem image is capable of using both `fsdax`   and `devdax`, the necessary steps to configure the PMEM to work with cassandra are documented here.
-
-.. tabs::
-
-   .. group-tab:: devdax
-
-      We need to verify the device we want to use is in `devdax` mode
-
-      .. code-block:: bash
-
-         sudo ndctl create-namespace -fe namespace0.0  --mode=devdax
-
-      .. code-block:: console
-
-         {
-           "dev":"namespace0.0",
-           "mode":"devdax",
-           "map":"dev",
-           "size":"3.94 GiB (4.23 GB)",
-           "uuid":"cb738cc7-711d-4578-bebf-1f7ba02ca169",
-           "daxregion":{
-           "id":0,
-           "size":"3.94 GiB (4.23 GB)",
-           "align":2097152,
-           "devices":[
-             {
-               "chardev":"dax0.0",
-               "size":"3.94 GiB (4.23 GB)"
-             }
-           ]
-          },
-          "align":2097152
-         }
-
-
-      If needed, we can reconfigure it using :command:`ndctl create-namespace -fe <namespace-name>  --mode=devdax`.
-
-      Before using a `devdax` device we need to clear the device:
-
-      .. code-block:: bash
-
-         sudo pmempool rm -vaf /dev/dax0.0
-
-
-      The `jvm.options` configuration for Apache Cassandra should look like the following:
-
-      .. code-block:: console
-
-         -Dpmem_path=/dev/dax0.0
-         -Dpool_size=0
-
-      Where
-      * pmem_path is the `devdax` device.
-      * pool_size=0 indicates to use the entire `devdax` device.
-
-      When using the `Docker image with Apache Cassandra`_, the file `jvm.options` is automatically populated.
-
-   .. group-tab:: fsdax
-
-      Verify that the PMEM is in `fsdax` mode
-
-      .. code-block:: bash
-
-         sudo ndctl list -u
-
-      .. code-block:: console
-
-        {
-          "dev":"namespace0.0",
-          "mode":"fsdax",
-          "map":"mem",
-          "size":"4.00 GiB (4.29 GB)",
-          "sector_size":512,
-          "blockdev":"pmem0"
-        }
-
-
-      If for some reason the device is not in `fsdax` mode you can reconfigure the namespace as follows:
-
-      .. code-block:: bash
-
-         sudo `ndctl create-namespace -fe <namespace-name>  --mode=fsdax`
-
-
-      Once the PMEM namespace is configured, you will see a device named :file:`/dev/pmem{0-9}`. We will create a filesystem on that device. The filesystem could be `ext4` or `xfs`, for this example we are going to use `ext4`.
-
-      .. code-block:: bash
-
-         sudo mkfs.ext4 /dev/pmem0
-
-      .. code-block:: console
-
-         mke2fs 1.45.2 (27-May-2019)
-         Creating filesystem with 1031680 4k blocks and 258048 inodes
-         Filesystem UUID: 303c03f5-ac4e-4462-8bf9-bc6b0fae53fe
-         Superblock backups stored on blocks:
-      	   32768, 98304, 163840, 229376, 294912, 819200, 884736
-
-         Allocating group tables: done
-         Writing inode tables: done
-         Creating journal (16384 blocks): done
-         Writing superblocks and filesystem accounting information: done
-
-
-      Once the filesystem is created, we mount it with the dax option
-
-      .. code-block:: bash
-
-         sudo mount /dev/pmem0 /mnt/pmem -o dax
-
-
-      When using `fsdax` mode cassandra-pmem creates a pool file on the pmem mountpoint, so the `jvm.options` configuration should look like the output below:
-
-      .. code-block:: console
-
-         -Dpmem_path=/mnt/pmem/cassandra_pool
-         -Dpool_size=3221225472
-
-
-
-      Where
-      * `pmem_path` is the path to the pool file, which should include the path itself and the file name
-      * `pool_size` is the size of the pool file in bytes. If you are using the `Docker image with Apache Cassandra`_ you can pass this value as an environment variable to the container runtime in Gb and the calculation is done automatically.
-
-      Is important to note that when creating the filesystem in the pmem device certain amount of space of the device is used by the filesystem metadata so the pool_size should be smaller than the total pmem namespace size.
-
-      When using the `Docker image with Apache Cassandra`_, the file `jvm.options` is automatically populated with the environment variables `CASSANDRA_PMEM_POOL_NAME` and `CASSANDRA_FSDAX_POOL_SIZE_GB`.
-
-
-
-Run the DBRS Container
-======================
-
-Replace `<image-id>` in the following commands with the name of the image you are using.
-
-.. tabs::
-
-   .. group-tab:: devdax
-
-      In `devdax` mode:
-
-      .. code-block:: bash
-
-         docker run --device=/<devdax-device>:/dev/dax0.0 --ulimit nofile=262144:262144 -p 9042:9042 -p 7000:7000 -it --name cassandra-test <image-id>
-
-   .. group-tab:: fsdax
-
-      In `fsdax` mode:
-
-      .. code-block:: bash
-
-         docker run --mount type=bind,source=/<fsdax-mountpoint>,target=/mnt/pmem  --ulimit nofile=262144:262144 -p 9042:9042 -p 7000:7000 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=<fsdax-pool-size-in-gb>' --name cassandra-test <image-id>
-
-
-Container Configuration
-=======================
-
-Using environment variables
----------------------------
-
-The container listens on the primary container IP address, but if required, some parameters can be provided as environment variables using `--env`.
-
-* `CASSANDRA_CLUSTER_NAME`  Cassandra cluster name, by default `Cassandra Cluster`
-* `CASSANDRA_LISTEN_ADDRESS`  Cassandra listen address
-* `CASSANDRA_RPC_ADDRESS`  Cassandra RPC address
-* `CASSANDRA_SEED_ADDRESSES`  A comma separated list of hosts in the cluster, if not provided, cassandra is going to run as a single node.
-* `CASSANDRA_SNITCH`  The snitch type for the cluster, by default it is `SimpleSnitch`, for more complex snitches you can mount your own `cassandra-rackdc.properties` file.
-* `LOCAL_JMX`  If set to `no` the JMX service will listen on all IP addresses, the default is `yes` and listens just on localhost 127.0.0.1
-* `JVM_OPTS` When set you can pass additional arguments to the JVM for cassandra execution, for example for specifying memory heap sizes `JVM_OPTS=-Xms16G -Xmx16G -Xmn12G`
-
-When using PMEM in `fsdax` mode, there are some parameters to control the allocation of memory:
-
-
-* `CASSANDRA_FSDAX_POOL_SIZE_GB`  The size of the fsdax pool in GB, if it is not specified the pool size is `1`
-* `CASSANDRA_PMEM_POOL_NAME`  The filename of the pool created in PMEM, by default `cassandra_pool`
-
-Using custom files
-------------------
-
-For more complex deployments it is also possible to provide custom `cassandra.yaml` and `jvm.options` files as shown below:
-
-.. code-block:: bash
-
-   docker run --mount type=bind,source=/<fsdax-mountpoint>,target=/mnt/pmem -it  --ulimit nofile=262144:262144 --mount type=bind,source=/<path-to-file>/cassandra.yaml,target=/workspace/cassandra/conf/cassandra.yaml --mount type=bind,source=/path-to-file>/jvm.options,target=/workspace/cassandra/conf/jvm.options --name cassandra-custom-files
-
-
-Clustering
-==========
-
-For a simple two node cluster using PMEM in `fsdax` mode on both containers:
-
-Node 1
-------
-
-* IP: 172.17.0.2
-* PMEM mountpoint: /mnt/pmem1
-
-.. code-block:: bash
-
-   docker run --mount type=bind,source=/mnt/pmem1,target=/mnt/pmem  --ulimit nofile=262144:262144 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=2' -e 'CASSANDRA_SEED_ADDRESSES=172.17.0.2:7000,172.17.0.3:7000'  --name cassandra-node1 <image-id>
-
-
-Node 2
-------
-
-* IP: 172.17.0.3
-* PMEM mountpoint: /mnt/pmem2
-
-.. code-block:: bash
-
-   docker run --mount type=bind,source=/mnt/pmem2,target=/mnt/pmem  --ulimit nofile=262144:262144 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=2' -e 'CASSANDRA_SEED_ADDRESSES=172.17.0.2:7000,172.17.0.3:7000'  --name cassandra-node2 <image-id>
-
-
-Once both nodes are running, eventually the gossip is settled and we can use `nodetool` on either container to check cluster status.
-
-.. code-block:: bash
-
-   docker exec -it <container-id> bash /workspace/cassandra/bin/nodetool status
-
-
-The output should look similar to this:
-
-.. code-block:: console
-
-
-   Datacenter: datacenter1
-   =======================
-   Status=Up/Down
-   |/ State=Normal/Leaving/Joining/Moving
-   --  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
-   UN  172.17.0.3  0 bytes    256          100.0%            22387159-8192-41cf-8b6c-8bf0e1049eb7  rack1
-   UN  172.17.0.2  0 bytes    256          100.0%            219b56ba-c07c-400b-a018-a5dc20edeb09  rack1
-
-
-
-Persistence
-===========
-
-By default you can access the data written to Apache Cassandra  as long as the container exists. In order to persist the data past that, you can mount volumes or bind mounts on :file:`/workspace/cassandra/data` and :file:`/workspace/cassandra/logs` and in this way the data can still be accessed once the container is deleted.
-
-Deploy An Apache Cassandra-PMEM cluster on Kubernetes*
-******************************************************
-
-Many containerized workloads are deployed in clusters and orchestration software like Kubernetes can be useful. We will use the `cassandra-pmem-helm`_ Helm* chart in this example.
-
-Requirements
-============
-
-* Kubectl* must be configured to access the Kubernetes Cluster
-
-* A Kubernetes cluster with `pmem-csi`_ enabled
-
-* The Kubernetes cluster must have `helm`_ and tiller installed
-
-* PMEM hardware
-
-.. important::
-
-   When selecting the `fsdax` pool file size, it is important to consider that when requesting a volume, certain amount of space is used by the filesystem metadata on that volume and the available space turns out to be less than total amount specified. Taking this into consideration the size of the fsdax pool file should be ~2G less than the total volume size requested.
-
-
-Configuration
-=============
-
-In order to configure the Apache Cassandra PMEM cluster some variables and values are provided. These values are set in :file:`test/cassandra-pmem-helm/values.yaml`, and can be modified according to your specific needs. A summary of those parameters is shown below:
-
-
-* clusterName:  The cluster Name set across all deployed nodes
-* replicaCount:  The number of nodes in the cluster to be deployed
-* image.repository:  The address of the container registry where the cassandra-pmem image should be pulled
-* image.tag:  The tag of the image to be pulled during deployment
-* image.name:  The name of the image to be pulled during deployment
-* pmem.containerPmemAllocation:  The size of the persistent volume claim to be used as heap, it uses the storage class `pmem-csi-sc-ext4` from pmem-csi  The size of the fsdax pool to be created inside the persistent volume claim, in practice it should be `1G` less than pmem.containerPmemAllocation
-* pmem.fsdaxPoolSizeInGB: The size of the fsdax pool to be created inside the persistent volume claim, in practice it should be 1G less than pmem.containerPmemAllocation
-* enablePersistence: If set to `true`, K8s persistent volumes are deployed to store data and logs
-* persistentVolumes.logsVolumeSize:  The size of the persistent volume used for storing logs on each node, the default is `4G`
-* persistentVolumes.dataVolumeSize:  The size of the persistent volume used for storing data on each node, the default is `4G`
-* persistentVolumes.logsStorageClass:  Storage class used by  the logs pvc, by default it uses `pmem-csi-sc-ext4`
-* persistentVolumes.dataStorageClass:  Storage class used by  the data pvc, by default it uses `pmem-csi-sc-ext4`
-* provideCustomConfig:  If set to `true`, it mounts all the files located on `<helm-chart-dir>/files/conf` on `/workspace/cassandra/conf` inside each container in order to provide a way to customize the deployment beyond the options provided here
-* exposeJmxPort:  When set to `true` it exposes the JMX port as part of the Kubernetes headless service. It should be used together with `enableAdditionalFilesConfigMap` in order to provide authentication files needed for JMX when the remote connections are allowed. When set to `false` only local access through 127.0.0.1 is granted and no additional authentication is needed.
-* enableClientToolsPod:  If set to `true`, an additional pod independent from the cluster is deployed, this pod contains various Cassandra client tools and mounts test profiles located under `<helm-chart-dir>/files/testProfiles` to `/testProfiles` inside the pod. This pod is useful to test and launch benchmarks
-* enableAdditionalFilesConfigMap:  When set to true, it takes the files located in `<helm-chart-dir>/files/additionalFiles` and mount them in `/etc/cassandra` inside the pods, some additional files for cassandra can be stored here, such as JMX auth files
-* jvmOpts.enabled:  If set to `true` the environment variable `JVM_OPTS` is overridden with the value provided on jvmOpts.value
-* jvmOpts.value: Sets the value of the environment variable `JVM_OPTS`, in this way some java runtime configurations can be provided such as RAM heap usage
-* resources.enabled:  if set to `true`, the resource constraints are set on each pod using the values under resources.requests and resources.limits
-* resources.requests.memory: Initial resource allocation for each pod in the cluster
-* resources.request.cpu: Initial resource allocation for each pod in the cluster
-* resources.limits.memory:  Limits for memory allocation for each pod in the cluster
-* resources.limits.cpu: Limits for cpu allocation for each pod in the cluster
-
-Installation
-============
-
-Once all the configurations are set, to install the chart inside a given Kubernetes cluster you must run:
-
-.. code-block:: bash
-
-   helm install ./cassandra-pmem-helm
-
-
-Eventually all the given nodes will be shown as running using :command:`kubectl get pods`.
-
-
-Running DBRS with Redis
-***********************
-
-The Redis stack application is enabled for a multinode Kubernetes environment using Intel® Optane™ DCPMM PMem DIMMs in fsdax mode for storage.
-
-The source code used for this application can be found in the `Github repository`_
-
-The following examples will use the `Docker image with Redis`_.  You can also build your own image with Docker by using the :file:`Dockerfile` and running with this command
-
-.. code-block:: bash
-
-   docker build --force-rm --no-cache -f Dockerfile -t ${DOCKER_IMAGE} .
-
-
-
-Single node
-===========
-
-Prior to starting the container, you will need to have the Intel® Optane™ DCPMM module in fsdax with a file system and mounted in `/mnt/dax0` as shown above.
-
-Use the following to start the container, replacing ${DOCKER_IMAGE} with the name of the image you are using.
-
-.. code-block:: bash
-
-   docker run --mount type=bind,source=/mnt/dax0,target=/mnt/pmem0 -i -d --name pmem-redis ${DOCKER_IMAGE} --nvm-maxcapacity 200 --nvm-dir /mnt/pmem0 --nvm-threshold 64 --protected-mode no
-
-
-
-
-Redis Operator in a Kubernetes cluster
-======================================
-
-After setting up :ref:`kubernetes` in |CL|, you will need to enable it to support DCPMM using the pmem-cls driver.  To install the driver follow the instructions in the `pmem-csi`_ repository.
-
-We are using source code from the `Redis operator`_ .
-
-.. note::
-
-   If you already have a redis-operator, you will need to delete it before installing a new one.
-
-
-
-
-After installing the operator you are ready to deploy redisfailover instances using a yaml file, like this `example for persistent memory`_. You can download it and change the source of the image to reflect your environment. We have named our yaml `redis-failover.yml`
-
-To start a redisfailover instance in Kubernetes run the following
-
-.. code-block:: bash
-
-   kubectl create -f redis-failover.yml
-
-
-.. important::
-
-   There is a `known issue`_ in which the sentinels do not have enough memory to create the InitContainer. The current workaround is to build the image increasing the limits for the InitContainer memory to 32Mb
-
-Running DBRS with Memcached
-***************************
-
-With DBRS V2.0 you can use the DBRS stack with `Memcached`_, a free and open source, high performance, distributed meory object caching system. This stack is ready to use DCPMM in fsdax for storage.   The source for this application can be found in the `Memcached`_ repository.
-
-.. note::
-
-   The DBRS v2.0 release does not support Redis or Cassandra.
-
-
-
-Build the DBRS Memcached image
-==============================
-
-To build the Memcached enabled image, use the Dockerfile with this command:
-
-.. code-block:: bash
-
-   docker build --force-rm --no-cache -f Dockerfile -t ${DOCKER_IMAGE} .
-
-
-Run DBRS with Memcached as a standalone container
-=================================================
-
-Prior to launching the container, you will need to configure the DCPMM in fsdax mode with a file system, and have it mounted in :file:`/mnt/dax0`. Instructions for configuration can be found in :ref:`dbrs-hardware-configuration`.
-
-To launch the container run this command:
-
-.. code-block:: bash
-
-   docker run --mount type=bind,source=/mnt/dax0,target=/mnt/pmem0 -i -d --name pmem-memchached ${DOCKER_IMAGE} -e /mnt/pmem0/memcached.file -m 64 -c 1024 -p 11211
-
-where:
-
-:command:`-m` is the maximum memory limit to use in megabytes
-:command:`-e` is the mmap path for external memory (DCPMM storage).  For this container the DCPMM sould be mounted inside the container on :file:`/mnt/pmem0`
-:command:`-c` is the number of concurrent connections
-:command:`-p` is the TCP connection port.
-
-For more information please refer to this `blog post`_ from `Memcached`_
-
-
-
-.. _Intel Download Center: https://downloadcenter.intel.com/download/28695/Intel-Server-Board-S2600WF-Family-BIOS-and-Firmware-Update-Package-for-UEFI
-
-.. _Quick Start Guide: https://software.intel.com/en-us/articles/quick-start-guide-configure-intel-optane-dc-persistent-memory-on-linux
-
-.. _Managing NVDIMMs: https://docs.pmem.io/ndctl-user-guide/managing-nvdimms
-
-.. _Configure, Manage, and Profile: https://software.intel.com/en-us/articles/configure-manage-and-profile-intel-optane-dc-persistent-memory-modules
-
-.. _DBRS github repository: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs
-
-.. _build-cassandra-pmem.sh: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs/cassandra/scripts/
-
-.. _cassandra-pmem-helm: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs/cassandra/cassandra-pmem-helm
-
-.. _helm: https://helm.sh/
-
-.. _Github repository: https://github.com/pmem/pmem-redis
-
-.. _Redis operator: https://github.com/spotahome/redis-operator
-
-.. _example for persistent memory: https://github.com/spotahome/redis-operator/blob/master/example/redisfailover/pmem.yaml
-
-.. _known issue: https://github.com/spotahome/redis-operator/issues/176
-
-.. _Docker image with Apache Cassandra: https://hub.docker.com/r/clearlinux/stacks-dbrs-cassandra
-
-.. _Docker image with Redis: https://hub.docker.com/r/clearlinux/stacks-dbrs-redis
-
-.. _Intel Optane PMM: https://www.intel.com/content/www/us/en/architecture-and-technology/optane-technology/optane-for-data-centers.html
-
-.. _pmem-csi: https://github.com/intel/pmem-csi/blob/release-0.6/README.md
-
-.. _DBRS Terms of Use: https://clearlinux.org/stacks/database/terms-of-use
-
-
-
-.. _Database Reference Stack website: https://clearlinux.org/stacks/database-reference
-
-.. _DBRS V1.0: https://clearlinux.org/news-blogs/database-reference-stack-dbrs-v10-now-available
-
-.. _DBRS V2.0: https://clearlinux.org/blogs-news/database-reference-stack-dbrs-v2-now-available
-
-.. _Memcached: https://memcached.org
-
-.. _blog post: https://memcached.org/blog/persistent-memory/

source/guides/stacks/greengrass.rst

@@ -1,425 +0,0 @@
-.. _greengrass:
-
-Enable AWS Greengrass\* and OpenVINO™ toolkit
-#############################################
-
-This guide explains how to enable AWS Greengrass\* and OpenVINO™ toolkit.
-Specifically, the guide demonstrates how to:
-
-* Set up the Intel® edge device with |CL-ATTR|
-* Install the OpenVINO™ toolkit and Amazon Web Services\* (AWS\*)
-  Greengrass\* software stacks
-* Use AWS Greengrass\* and AWS Lambda\* to deploy the FaaS samples from
-  the cloud
-
-.. contents::
-   :local:
-   :depth: 1
-
-Overview
-********
-
-Hardware accelerated Function-as-a-Service (FaaS) enables cloud developers to
-deploy inference functionalities [1] on Intel® IoT edge devices with
-accelerators (CPU, Integrated GPU, Intel® FPGA, and Intel® Movidius™
-technology). These functions provide a great developer experience and
-seamless migration of visual analytics from cloud to edge in a secure manner
-using a containerized environment. Hardware-accelerated FaaS provides the
-best-in-class performance by accessing optimized deep learning libraries on
-Intel® IoT edge devices with accelerators.
-
-Supported platforms
-*******************
-
-*	Operating System: |CL| latest release
-*	Hardware:	Intel® core platforms (that support inference on CPU only)
-
-Sample description
-==================
-
-The AWS Greengrass samples are located at `Edge-Analytics-FaaS`_. This
-guide uses the 1.0 version of the source code.
-
-|CL| provides the following AWS Greengrass samples:
-
-* `greengrass_classification_sample.py`_
-
-  This AWS Greengrass sample classifies a video stream using classification
-  networks such as AlexNet and GoogLeNet and publishes top-10 results on AWS\*
-  IoT Cloud every second.
-
-* `greengrass_object_detection_sample_ssd.py`_
-
-  This AWS Greengrass sample detects objects in a video stream and
-  classifies them using single-shot multi-box detection (SSD) networks such
-  as SSD Squeezenet, SSD Mobilenet, and SSD300. This sample publishes
-  detection outputs such as class label, class confidence, and bounding box
-  coordinates on AWS IoT Cloud every second.
-
-
-Install the OS on the edge device
-*********************************
-
-Start with a clean installation of |CL| on a new system, using the
-:ref:`bare-metal-install-desktop`, found in :ref:`get-started`.
-
-Create user accounts
-====================
-
-After |CL| is installed, create two user accounts. Create an administrative
-user in |CL| and create a user account for the Greengrass services to use (
-see Greengrass user below).
-
-#. Create a new user and set a password for that user. Enter the following
-   commands as ``root``:
-
-   .. code-block:: bash
-
-      useradd <userid>
-      passwd <userid>
-
-#. Next, enable the :command:`sudo` command for your new <userid>. Add
-   <userid> to the `wheel` group:
-
-   .. code-block:: bash
-
-      usermod -G wheel -a <userid>
-
-#. Create a :file:`/etc/fstab` file.
-
-   .. code-block:: bash
-
-      touch /etc/fstab
-
-   .. note::
-
-      By default, |CL| does not create an :file:`/etc/fstab` file.
-      You must create this file before the Greengrass service runs.
-
-Add required bundles
-====================
-
-Use the :command:`swupd` software updater utility to add the prerequisite bundles
-for the OpenVINO software stack:
-
-.. code-block:: bash
-
-   swupd bundle-add os-clr-on-clr desktop-autostart computer-vision-basic
-
-.. note::
-
-   Learn more about how to :ref:`swupd-guide`.
-
-The :command:`computer-vision-basic` bundle installs the OpenVINO™ toolkit,
-and the sample models optimized for Intel® edge platforms.
-
-.. _convert-dl-models:
-
-Convert deep learning models
-============================
-
-Locate sample models
---------------------
-
-There are two types of provided models that can be used in conjunction with
-AWS Greengrass for this guide: classification or object detection.
-
-To complete this guide using an image classification model,
-download the BVLC AlexNet model files `bvlc_alexnet.caffemodel`_ and
-`deploy.prototxt`_ to the default model_location at
-:file:`/usr/share/openvino/models`. Any custom pre-trained classification models
-can be used with the classification sample.
-
-For object detection, the sample models optimized for Intel® edge platforms
-are included with the computer-vision-basic bundle installation at
-:file:`/usr/share/openvino/models`. These models are provided as an example;
-you may also use a custom SSD model with the Greengrass object detection sample.
-
-Run model optimizer
--------------------
-
-Follow the instructions in the `Model Optimizer Developer Guide`_ for converting
-deep learning models to Intermediate Representation using Model Optimizer. To
-optimize either of the sample models described above, run one of the following commands.
-
-For classification using BVLC AlexNet model:
-
-.. code-block:: bash
-
-   python3 mo.py --framework caffe --input_model
-   <model_location>/bvlc_alexnet.caffemodel --input_proto
-   <model_location>/deploy.prototxt --data_type <data_type> --output_dir
-   <output_dir> --input_shape [1,3,227,227]
-
-For object detection using SqueezeNetSSD-5Class model:
-
-.. code-block:: bash
-
-   python3 mo.py --framework caffe --input_model
-   <model_location>/'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.caffemodel
-   --input_proto <model_location>/'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.prototxt
-   --data_type <data_type> --output_dir <output_dir>
-
-In these examples:
-
-* `<model_location>` is :file:`/usr/share/openvino/models`.
-
-* `<data_type>` is FP32 or FP16, depending on target device.
-
-* `<output_dir>` is the directory where the Intermediate Representation
-  (IR) is stored. IR contains .xml format corresponding to the network
-  structure and .bin format corresponding to weights. This .xml file should be
-  passed to :command:`<PARAM_MODEL_XML>`.
-
-* In the BVLC AlexNet model, the prototxt defines the input shape with
-  batch size 10 by default. In order to use any other batch size, the
-  entire input shape must be provided as an argument to the model
-  optimizer. For example, to use batch size 1, you must provide:
-  `--input_shape [1,3,227,227]`
-
-
-Configure AWS Greengrass group
-******************************
-
-For each Intel® edge platform, you must create a new AWS Greengrass group
-and install AWS Greengrass core software to establish the connection between
-cloud and edge.
-
-#. To create an AWS Greengrass group, follow the instructions in
-   `Configure AWS IoT Greengrass on AWS IoT`_.
-
-#. To install and configure AWS Greengrass core on edge platform, follow
-   the instructions in `Start AWS Greengrass on the Core Device`_. In
-   step 8(b), download the x86_64 Ubuntu\* configuration of the AWS Greengrass
-   core software.
-
-   .. note::
-
-      You do not need to run the :file:`cgroupfs-mount.sh` script in step #6
-      of Module 1 of the `AWS Greengrass Developer Guide`_ because this is
-      enabled already in |CL|.
-
-#. Be sure to download both the security resources and the AWS Greengrass
-   core software.
-
-   .. note::
-
-      Security certificates are linked to your AWS account.
-
-
-Create and package Lambda function
-**********************************
-
-#. Complete steps 1-4 of the AWS Greengrass guide at
-   `Create and Package a Lambda Function`_.
-
-   .. note::
-
-      This creates the tarball needed to create the AWS Greengrass
-      environment on the edge device.
-
-
-#. In step 5, replace :file:`greengrassHelloWorld.py` with the classification or
-   object detection Greengrass sample from `Edge-Analytics-Faas`_:
-
-   * Classification: `greengrass_classification_sample.py`_
-
-   * Object Detection: `greengrass_object_detection_sample_ssd.py`_
-
-#. Zip the selected Greengrass sample with the extracted Greengrass SDK folders
-   from the previous step into :file:`greengrass_sample_python_lambda.zip`.
-
-   The zip should contain:
-
-   * greengrasssdk
-
-   * greengrass classification or object detection sample
-
-   For example:
-
-   .. code-block:: bash
-
-      zip -r greengrass_lambda.zip greengrasssdk
-      greengrass_object_detection_sample_ssd.py
-
-#. Return to the AWS documentation section called
-   `Create and Package a Lambda Function`_ and complete the procedure.
-
-   .. note::
-
-      In step 9(a) of the AWS documentation, while uploading the zip file,
-      make sure to name the handler to one of the following, depending on the
-      AWS Greengrass sample you are using:
-
-      * greengrass_object_detection_sample_ssd.function_handler
-      * greengrass_classification_sample.function_handler
-
-
-Configure Lambda function
-*************************
-
-After creating the Greengrass group and the Lambda function, start
-configuring the Lambda function for AWS Greengrass.
-
-#. Follow steps 1-8 in `Configure the Lambda Function for AWS IoT Greengrass`_
-   in the AWS documentation.
-
-#. In addition to the details mentioned in step 8, change the Memory limit
-   to 2048 MB to accommodate large input video streams.
-
-#. Add the following environment variables as key-value pairs when editing
-   the Lambda configuration and click on update:
-
-   .. list-table:: **Table 1.  Environment variables: Lambda configuration**
-      :widths: 20 80
-      :header-rows: 1
-
-      * - Key
-        - Value
-      * - PARAM_MODEL_XML
-        - <MODEL_DIR>/<IR.xml>, where <MODEL_DIR> is user specified and
-          contains IR.xml, the Intermediate Representation file from Intel® Model Optimizer.
-          For this guide, <MODEL_DIR> should be set to '/usr/share/openvino/models'
-          or one of its subdirectories.
-      * - PARAM_INPUT_SOURCE
-        - <DATA_DIR>/input.webm to be specified by user. Holds both input and
-           output data. For webcam, set PARAM_INPUT_SOURCE to ‘/dev/video0’
-      * - PARAM_DEVICE
-        - "CPU"
-      * - PARAM_CPU_EXTENSION_PATH
-        - /usr/lib64/libcpu_extension.so
-      * - PARAM_OUTPUT_DIRECTORY
-        - <DATA_DIR> to be specified by user. Holds both input and output
-          data
-      * - PARAM_NUM_TOP_RESULTS
-        - User specified for classification sample.
-          (e.g. 1 for top-1 result, 5 for top-5 results)
-
-#. Add subscription to subscribe, or publish messages from AWS Greengrass
-   Lambda function by completing the procedure in `Configure the Lambda Function for AWS IoT Greengrass`_.
-
-   .. note::
-
-      The optional topic filter field is the topic mentioned inside the Lambda function. In this guide, sample topics include the following:
-      :command:`openvino/ssd` or :command:`openvino/classification`
-
-Add local resources
-===================
-
-Refer to the AWS documentation `Access Local Resources with Lambda Functions and Connectors`_
-for details about local resources and access privileges.
-
-The following table describes the local resources needed for the CPU:
-
-.. list-table:: **Local resources**
-    :widths: 20, 20, 20, 20
-    :header-rows: 1
-
-    * - Name
-      - Resource type
-      - Local path
-      - Access
-
-    * - ModelDir
-      - Volume
-      - <MODEL_DIR> to be specified by user
-      - Read-Only
-
-    * - Webcam
-      - Device
-      - /dev/video0
-      - Read-Only
-
-    * - DataDir
-      - Volume
-      - <DATA_DIR> to be specified by user. Holds both input and output
-        data.
-      - Read and Write
-
-Deploy Lambda function
-**********************
-
-Refer to the AWS documentation `Deploy Cloud Configurations to an AWS IoT Greengrass Core Device`_ for instructions on how to deploy the lambda function to AWS
-Greengrass core device. Select *Deployments* on the group page and follow the instructions.
-
-Output consumption
-==================
-
-There are four options available for output consumption. These options are
-used to report, stream, upload, or store inference output at an interval
-defined by the variable :command:`reporting_interval` in the AWS Greengrass samples.
-
-#. IoT cloud output:
-
-   This option is enabled by default in the AWS Greengrass samples using the 
-   :command:`enable_iot_cloud_output` variable. You can use it to verify the lambda
-   running on the edge device. It enables publishing messages to IoT cloud
-   using the subscription topic specified in the lambda. (For example, topics
-   may include :command:`openvino/classification` for classification and :command:`openvino/ssd`
-   for object detection samples.) For classification, top-1 result with class
-   label are published to IoT cloud. For SSD object detection, detection
-   results such as bounding box coordinates of objects, class label, and
-   class confidence are published.
-
-   Refer to the AWS documentation
-   `Verify the Lambda Function Is Running on the Device`_ for instructions on
-   how to view the output on IoT cloud.
-
-#. Kinesis streaming:
-
-   This option enables inference output to be streamed from the edge device
-   to cloud using Kinesis [3] streams when :command:`enable_kinesis_output` is set
-   to True. The edge devices act as data producers and continually push
-   processed data to the cloud. You must set up and specify
-   Kinesis stream name, Kinesis shard, and AWS region in the AWS Greengrass
-   samples.
-
-#. Cloud storage using AWS S3 bucket:
-
-   When the :command:`enable_s3_jpeg_output` variable is set to True, it enables
-   uploading and storing processed frames (in jpeg format) in an AWS S3
-   bucket. You must set up and specify the S3 bucket name in the AWS
-   Greengrass samples to store the JPEG images. The images are named using the
-   timestamp and uploaded to S3.
-
-#. Local storage:
-
-   When the :command:`enable_s3_jpeg_output` variable is set to True, it enables
-   storing processed frames (in jpeg format) on the edge device. The images
-   are named using the timestamp and stored in a directory specified by
-   :command:`PARAM_OUTPUT_DIRECTORY`.
-
-References
-**********
-
-#. AWS Greengrass: https://aws.amazon.com/greengrass/
-#. AWS Lambda: https://aws.amazon.com/lambda/
-#. AWS Kinesis: https://aws.amazon.com/kinesis/
-
-.. _Edge-Analytics-FaaS: https://github.com/intel/Edge-Analytics-FaaS/tree/v1.0/AWS%20Greengrass
-
-.. _bvlc_alexnet.caffemodel: http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel
-
-.. _deploy.prototxt: https://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/deploy.prototxt
-
-.. _greengrass_classification_sample.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_classification_sample.py
-
-.. _greengrass_object_detection_sample_ssd.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_object_detection_sample_ssd.py
-
-.. _Model Optimizer Developer Guide: https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer
-
-.. _AWS Greengrass Developer Guide: https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html
-
-.. _Configure AWS IoT Greengrass on AWS IoT: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html
-
-.. _Start AWS Greengrass on the Core Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-device-start.html
-
-.. _Configure the Lambda Function for AWS IoT Greengrass: https://docs.aws.amazon.com/greengrass/latest/developerguide/config-lambda.html
-
-.. _Access Local Resources with Lambda Functions and Connectors: https://docs.aws.amazon.com/greengrass/latest/developerguide/access-local-resources.html
-
-.. _Deploy Cloud Configurations to an AWS IoT Greengrass Core Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/configs-core.html
-
-.. _Verify the Lambda Function Is Running on the Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-check.html
-
-.. _Create and Package a Lambda Function: https://docs.aws.amazon.com/greengrass/latest/developerguide/create-lambda.html

source/guides/stacks/mers.rst

@@ -1,420 +0,0 @@
-.. _mers:
-
-Media Reference Stack
-#####################
-
-The Media Reference Stack (MeRS) is a highly optimized software stack for
-Intel® architecture to enable media prioritized workloads, such as transcoding and analytics.
-
-This guide explains how to use the pre-built |MERS| container image, build
-your own |MERS| container image, and use the reference stack.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Overview
-********
-
-Finding the balance between quality and performance, understanding all of the
-complex standard-compliant encoders, and optimizing across the
-hardware-software stack for efficiency are all engineering and time
-investments for developers.
-
-The Media Reference Stack (MeRS) offers a highly optimized software stack for
-Intel Architecture to enable media prioritized workloads, such as transcoding
-and analytics. |MERS| abstracts away the complexity of integrating multiple
-software components and specifically tunes them for Intel platforms. |MERS|
-allows media and visual cloud developers to deliver experiences using a simple
-containerized solution. 
-
-Prerequisites
-=============
-
-|MERS| can run on any host system that supports Docker\*.
-
-The steps in this guide use |CL-ATTR| as the host system.
-
-- To install |CL| on a host system, see how to 
-  :ref:`install Clear Linux* OS from the live desktop
-  <bare-metal-install-desktop>`. 
-
-- To install Docker* on a |CL| host system, see
-  the :ref:`instructions for installing Docker* <docker>`.
-
-.. important:: 
-
-   For optimal performance, a processor with Vector Neural Network
-   Instructions (VNNI) should be used. VNNI is an extension of Intel® 
-   Advanced Vector Extensions 512 (Intel® AVX-512) and is available starting 
-   with the 2nd generation of Intel® Xeon® Scalable Platform, providing AI 
-   inference acceleration.
-
-Stack Features
-==============
-
-The |MERS| provides a `pre-built Docker image available on DockerHub
-<https://hub.docker.com/r/clearlinux/stacks-mers>`_, which includes
-instructions on build the image from source. |MERS| is open-sourced to ensure
-developers have easy access to the source code and are able to customize it.
-|MERS| is built using the *clearlinux:latest* Docker image and aims to support
-the latest |CL| version.
-
-|MERS| provides the following libraries:
-
-.. list-table::
-   :widths: auto
-
-   * - SVT-HEVC
-     - Scalable Video Technology for HEVC encoding, also known as H.265
-   * - SVT-AV1
-     - Scalable Video Technology for AV1 encoding
-   * - x264
-     - x264 for H.264/MPEG-4 AVC encoding
-   * - MKL-DNN
-     - `Intel® Math Kernel Library for Deep Neural Networks <https://01.org/mkl-dnn>`_
-
-Components of the |MERS| include:
-
-* |CL| as a base for performance and security.
-
-* `OpenVINO™ toolkit
-  <https://01.org/openvinotoolkit>`_ for inference.
-
-* `FFmpeg* <https://www.ffmpeg.org>`_ with `Scalable Video Technology (SVT)
-  <https://01.org/svt>`_ plugins for encoding, decoding, and transcoding.
-
-* `GStreamer* <https://gstreamer.freedesktop.org/>`_  with `Scalable Video
-  Technology (SVT) <https://01.org/svt>`_ and `OpenVINO™ toolkit
-  <https://01.org/openvinotoolkit>`_ plugins for analytics.
-
-
-.. note::
-
-   The pre-built |MERS| container image configures :command:`FFmpeg` without
-   certain elements (specific encoder, decoder, muxer, etc.) that you may
-   require. If you require changes to :command:`FFmpeg` we suggest starting at
-   :ref:`building-the-mers-container-image`.
-
-.. note::
-
-   The Media Reference Stack is a collective work, and each piece of software
-   within the work has its own license. Please see the `MeRS Terms of Use
-   <https://clearlinux.org/stacks/media/terms-of-use>`_ for more details about
-   licensing and usage of the Media Reference Stack.
-
-
-Getting the pre-built |MERS| container image
-********************************************
-
-Pre-built |MERS| Docker images are available on DockerHub at
-https://hub.docker.com/r/clearlinux/stacks-mers
-
-
-To use the |MERS|:
-
-#. Pull the image directly from `Docker Hub
-   <https://hub.docker.com/r/clearlinux/stacks-mers>`_. 
-
-   .. code-block:: bash
-
-      docker pull clearlinux/stacks-mers
-
-   .. note ::
-
-      The |MERS| docker image is large in size and will take some time to
-      download depending on your Internet connection.
-
-      If you are on a network with outbound proxies, be sure to configure
-      Docker allow access. See the `Docker service proxy
-      <https://docs.docker.com/config/daemon/systemd/#httphttps-proxy>`_ and
-      `Docker client proxy
-      <https://docs.docker.com/network/proxy/#configure-the-docker-client>`_
-      documentation for more details.
-      
-#. Once you have downloaded the image, run it with:
-
-   .. code-block:: bash
-
-      docker run -it clearlinux/stacks-mers
-
-   This will launch the image and drop you into a bash shell inside the
-   container. :command:`GStreamer` and :command:`FFmpeg` programs are
-   installed in the container image and accessible in the default $PATH. These
-   programs can be used as you would normally outside of |MERS|.
-
-   Paths to media files and video devices, such as cameras, can be shared from
-   the host to the container with the :command:`--volume` switch `using Docker
-   volumes <https://docs.docker.com/storage/volumes/>`_.
-
-.. _building-the-mers-container-image:
-
-Building the |MERS| container image from source
-***********************************************
-
-If you choose to build your own MeRS container image, you can optionally add
-customizations as needed. The :file:`Dockerfile` for the MeRS is available on
-`GitHub <https://github.com/intel/stacks/tree/master/mers>`_ and
-can be used for reference.
-
-#. The |MERS| image is part of the dockerfiles repository inside the |CL|
-   organization on GitHub. Clone the :file:`stacks` repository.
-
-   .. code-block:: bash
-
-      git clone https://github.com/intel/stacks.git
-
-#. Navigate to the :file:`stacks/mers/clearlinux` directory which contains 
-   the Dockerfile for the |MERS|.
-   
-   .. code-block:: bash
-
-      cd ./stacks/mers/clearlinux
-       
-#. Use the :command:`docker build` command with the :file:`Dockerfile` to the
-   MeRS container image.
-
-   .. code-block:: bash
-
-      docker build --no-cache -t clearlinux/stacks-mers .
-
-Using the |MERS| container image
-********************************
-
-Below are some examples of how the |MERS| container image can be used to
-process media files.
-
-The models and video source can be substituted from your use-case. Some
-publicly licensed sample videos are available at `sample-videos repsoitory
-<https://github.com/intel-iot-devkit/sample-videos>`_ for testing.
-
-
-Example 1: Transcoding
-======================
-
-This example shows how to perform transcoding with :command:`FFmpeg`.
-
-#. On the host system, setup a workspace for data and models:
-
-   .. code:: bash
-
-      mkdir ~/ffmpeg
-      mkdir ~/ffmpeg/input
-      mkdir ~/ffmpeg/output
-
-#. Copy a video file to :file:`~/ffmpeg/input`. 
-
-   .. code:: bash
-
-      cp </path/to/video> ~/ffmpeg/input
-
-#. Run the *clearlinux/stack-mers* docker image, allowing shared access to the
-   workspace on the host:
-
-
-   .. code:: bash
-
-      docker run -it \
-      -v ~/ffmpeg:/home/mers-user:ro \
-      clearlinux/stacks-mers:latest
-
-   After running the :command:`docker run` command, you enter a bash shell
-   inside the container. 
-
-#. From the container shell, you can run :command:`FFmpeg` against the videos
-   in :file:`/home/mers-user/input` as you would normally outside of |MERS|.
-
-   For example, to transcode raw yuv420 content to SVT-HEVC and mp4:
-
-   .. code:: bash
-
-      ffmpeg -f rawvideo -vcodec rawvideo -s 320x240 -r 30 -pix_fmt yuv420p -i </home/mers-user/input/test.yuv> -c:v libsvt_hevc -y </home/mers-user/output/test.mp4>
-      
-   Some more generic examples of :command:`FFmpeg` commands can be found in
-   the `OpenVisualCloud repository
-   <https://github.com/OpenVisualCloud/Dockerfiles/blob/master/doc/ffmpeg.md>`_ and used for reference with |MERS|.
-
-   For more information on using :command:`FFmpeg`, refer to the `FFmpeg
-   documentation <https://ffmpeg.org/documentation.html>`_.
-
-Example 2: Analytics
-====================
-
-This example shows how to perform analytics and inferences with
-:command:`GStreamer`.
-
-The steps here are referenced from the `gst-video-analytics Getting Started
-Guide <https://github.com/opencv/gst-video-analytics/wiki>`_ except simply
-substituting the *gst-video-analytics* docker image for the
-*clearlinux/stacks-mers* image.
-
-The example below shows how to use the |MERS| container image to perform video
-with object detection and attributes recognition of a video using GStreamer
-using pre-trained models and sample video files.
-
-#. On the host system, setup a workspace for data and models:
-
-   .. code:: bash
-
-      mkdir ~/gva
-      mkdir ~/gva/data
-      mkdir ~/gva/data/models
-      mkdir ~/gva/data/models/intel
-      mkdir ~/gva/data/models/common
-      mkdir ~/gva/data/video
-
-#. Clone the opencv/gst-video-analytics repository into the workspace:
-
-   .. code:: bash
-
-      git clone https://github.com/opencv/gst-video-analytics ~/gva/gst-video-analytics
-      cd ~/gva/gst-video-analytics
-      git submodule init
-      git submodule update
-
-#. Clone the Open Model Zoo repository into the workspace:
-
-   .. code:: bash
-
-      git clone https://github.com/opencv/open_model_zoo.git ~/gva/open_model_zoo
-      
-#. Use the Model Downloader tool of Open Model Zoo to download ready to use
-   pre-trained models in IR format.
-
-   .. note::
-      
-      If you are on a network with outbound proxies, you will need to
-      configure set environment variables with the proxy server. 
-      Refer to the documentation on :ref:`proxy` for detailed steps.
-
-      On |CL| systems you will need the *python-extras* bundle. 
-      Use :command:`sudo swupd bundle-add python-extras` for the downloader script to work.
-
-   .. code:: bash
-
-      cd ~/gva/open_model_zoo/tools/downloader 
-      python3 downloader.py --list ~/gva/gst-video-analytics/samples/model_downloader_configs/intel_models_for_samples.LST -o ~/gva/data/models/intel
-  
-  
-#. Copy a video file in h264 or mp4 format to :file:`~/gva/data/video`. Any
-   video with cars, pedestrians, human bodies, and/or human faces can be used.
-
-   .. code:: bash
-
-      git clone https://github.com/intel-iot-devkit/sample-videos.git ~/gva/data/video
-
-   This example simply clones all the video files from the `sample-videos
-   repsoitory <https://github.com/intel-iot-devkit/sample-videos>`_.
-   
-#. From a desktop terminal, allow local access to the X host display. 
-
-   .. code:: bash
-
-      xhost local:root
-
-      export DATA_PATH=~/gva/data
-      export GVA_PATH=~/gva/gst-video-analytics
-      export MODELS_PATH=~/gva/data/models
-      export INTEL_MODELS_PATH=~/gva/data/models/intel
-      export VIDEO_EXAMPLES_PATH=~/gva/data/video
-
-#. Run the *clearlinux/stack-mers* docker image, allowing shared access to 
-   the X server and workspace on the host:
-
-   .. code:: bash
-
-      docker run -it --runtime=runc --net=host \
-      -v ~/.Xauthority:/root/.Xauthority \
-      -v /tmp/.X11-unix:/tmp/.X11-unix \
-      -e DISPLAY=$DISPLAY \
-      -e HTTP_PROXY=$HTTP_PROXY \
-      -e HTTPS_PROXY=$HTTPS_PROXY \
-      -e http_proxy=$http_proxy \
-      -e https_proxy=$https_proxy \
-      -v $GVA_PATH:/home/mers-user/gst-video-analytics \      
-      -v $INTEL_MODELS_PATH:/home/mers-user/intel_models \
-      -v $MODELS_PATH:/home/mers-user/models \
-      -v $VIDEO_EXAMPLES_PATH:/home/mers-user/video-examples \
-      -e MODELS_PATH=/home/mers-user/intel_models:/home/mers-user/models \      
-      -e VIDEO_EXAMPLES_DIR=/home/mers-user/video-examples \
-      clearlinux/stacks-mers:latest
-
-   .. note:: 
-
-      In the :command:`docker run` command above:
-
-      - :command:`--runtime=runc` specifies the container runtime to be
-        *runc* for this container. It is needed for correct interaction with X
-        server.
-
-      - :command:`--net=host` provides host network access to container. It is
-        needed for correct interaction with X server.
-      
-      - Files :file:`~/.Xauthority` and :file:`/tmp/.X11-unix` mapped to the
-        container are needed to ensure smooth authentication with X server.
-      
-      - :command:`-v` instances are needed to map host system directories
-        inside Docker container.
-      
-      - :command:`-e` instances set Docker container environment variables.
-        Samples need them some of them set correctly to operate. Proxy variables
-        are needed if host is behind firewall.
-      
-
-   After running the :command:`docker run` command, it will drop you into a
-   bash shell inside the container. 
-
-#. From the container shell, run a sample analytics program in 
-   :file:`~/gva/gst-video-analytics/samples` against your video source.
-
-   Below are sample analytics that can be run against the sample videos.
-   Choose one to run:
-
-   - Samples with *face detection and classification*:
-
-     .. code:: bash
-
-        ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/face-demographics-walking-and-pause.mp4
-        ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/face-demographics-walking.mp4
-        ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-female-and-male.mp4
-        ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-male.mp4
-        ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-female.mp4
-      
-     When running, a video with object detection and attributes recognition
-     (bounding boxes around faces with recognized attributes) should be
-     played.
-     
-     .. figure:: /_figures/stacks/mers-fig-1.png
-        :scale: 60%
-        :align: center
-        :alt: Face detection with the Clear Linux* OS Media Reference Stack
-
-        Figure 1: Screenshot of |MERS| running face detection with GSTreamer
-        and OpenVINO.
-
-   - Sample with  *vehicle detection*:
-
-     .. code:: bash
-
-        ./gst-video-analytics/samples/shell/vehicle_detection_2sources_cpu.sh $VIDEO_EXAMPLES_DIR/car-detection.mp4
-   
-     When running, a video with object detection and attributes recognition
-     (bounding boxes around vehicles with recognized attributes) should be
-     played.
-
-     .. figure:: /_figures/stacks/mers-fig-2.png
-        :scale: 60%
-        :align: center
-        :alt: Vehicle detection with the Clear Linux* OS Media Reference Stack
-        
-        Figure 2: Screenshot of |MERS| running vehicle detection with
-        GSTreamer and OpenVINO.
-
-   - Sample with *FPS measurement*:
-
-     .. code:: bash
-
-       ./gst-video-analytics/samples/shell/console_measure_fps_cpu.sh $VIDEO_EXAMPLES_DIR/bolt-detection.mp4
-
-
-**OpenVINO is a trademark of Intel Corporation or its subsidiaries.**

source/index.rst

@@ -15,11 +15,6 @@
          **autospec** is a tool to assist in the automated creation and
          maintenance of RPM packaging in Clear Linux OS. 
 
-      :ref:`dlrs`
-         This tutorial shows you how to run benchmarking workloads in Clear
-         Linux OS using TensorFlow\* or PyTorch\* with the Deep Learning
-         Reference Stack.
-
       :ref:`docker`
          Clear Linux OS supports multiple containerization platforms,
          including a Docker solution.
@@ -57,7 +52,7 @@
 
       | `Ask the Clear Linux experts <https://clearlinux.org/community/mailing-list>`_
       | `Clear Linux Forum <https://community.clearlinux.org/>`_
-      | `Freenode IRC: #clearlinux <https://webchat.freenode.net/#clearlinux>`_
+      | `irc.libera.chat IRC: #clearlinux <https://kiwiirc.com/client/irc.libera.chat/?nick=web_guest|?#clearlinux>`_
 
 
 .. container:: video
@@ -75,4 +70,4 @@
    tutorials/index
    reference/index
    FAQ/index
-   collaboration/collaboration
+   collaboration/collaboration

source/reference/compatible-hardware.rst

@@ -11,103 +11,104 @@ grow.
     :widths: 20, 20
     :header-rows: 1
 
-    * - Processor SKU
+    * - Processor
       - Platform
 
-    * - Intel® Core™ i5-6260U
+    * - Intel® Core™ i5-6260U processor
       -
 
-    * - Intel® Core™ i5-6560U
+    * - Intel® Core™ i5-6560U processor
       - Dell XPS\* 13 9350
 
-    * - Intel® Celeron® J3455
+    * - Intel® Celeron® J3455 processor
       - NUC6CAYS
 
-    * - Intel® Core™ i5-4250U
+    * - Intel® Core™ i5-4250U processor
       -
 
-    * - Intel® Core™ i7-5557U
+    * - Intel® Core™ i7-5557U processor
       -
 
-    * - Intel® Core™ i9-7900X
+    * - Intel® Core™ i9-7900X X-series processor
       - Gigabyte\* X299
 
-    * - Intel® Core™ i3-4130
+    * - Intel® Core™ i3-4130 processor
       - Lenovo Thinkserver\* TS140
 
-    * - Intel® Core™ i7-7567U
+    * - Intel® Core™ i7-7567U processor
       - NUC7i7BNH
 
-    * - Intel® Core™ i7-8809G
+    * - Intel® Core™ i7-8809G processor
       - NUC8i7HVK
 
-    * - Intel® Core™ i5-7260U
+    * - Intel® Core™ i5-7260U processor
       - NUC7i5BNH
 
-    * - Intel® Core™ i7-8650U
+    * - Intel® Core™ i7-8650U processor
       - NUC7i7DNKE
 
-    * - Intel® Core™ i5-7300U
+    * - Intel® Core™ i5-7300U processor
       - NUC7i5DNHE
 
-    * - Intel® Xeon® Gold 6138
+    * - Intel® Xeon® Gold 6138 processor
       -
 
-    * - Intel® Xeon® E5-2699A v4
+    * - Intel® Xeon® E5-2699A v4 processor
       - Dell PowerEdge\* R630
 
-    * - Intel® Xeon® E5-2620 v3
+    * - Intel® Xeon® E5-2620 v3 processor
       -
 
-    * - Intel® Core™ i5-6600
+    * - Intel® Core™ i5-6600 processor
       - Gigabyte\* Z170X-UD5
 
-    * - Intel® Core™ i5-4250U
+    * - Intel® Core™ i5-4250U processor
       - D54250WYK
 
-    * - Intel® Xeon® E5-2699 v3
+    * - Intel® Xeon® E5-2699 v3 processor
       - S2600WT2
 
-    * - Intel® Atom™ J3455
+    * - Intel Atom™ J3455 processor
       - NUC6CAYB
 
-    * - Intel® Xeon® Bronze 3104
+    * - Intel® Xeon® Bronze 3104 processor
       - 0W23H8
 
-    * - Intel® Atom™ C2750
+    * - Intel Atom™ C2750 processor
       - SuperMicro\* A1SAi
 
-    * - Intel® Atom™ E3825
+    * - Intel Atom™ E3825 processor
       - CircuitCo MinnowBoard MAX\*
 
-    * - Intel® Core™ i7-8700
+    * - Intel® Core™ i7-8700 processor
       - Gigabyte\* H370 WIFI
 
-    * - Intel® Core™ i7-3667U
+    * - Intel® Core™ i7-3667U processor
       - Lenovo ThinkPad\* X1 Carbon laptop
 
-    * - Intel® Core™ i5-4210U
+    * - Intel® Core™ i5-4210U processor
       - Dell XPS\* 13 laptop
 
-    * - Intel® Celeron® J3455
+    * - Intel® Celeron® J3455 processor
       - NUC6CAYB
 
-    * - Intel® Core™ i7-4790
+    * - Intel® Core™ i7-4790 processor
       - Gigabyte\* desktop
 
-    * - Intel® Core™ i5-6260U
+    * - Intel® Core™ i5-6260U processor
       - NUC6I6SYH
 
-    * - Intel® Core™ i7-5557U
+    * - Intel® Core™ i7-5557U processor
       - NUC5I7RYH
 
-    * - Intel® Core™ i7-4700MQ
+    * - Intel® Core™ i7-4700MQ processor
       - Lenovo ThinkPad\* T540p
 
-    * - Intel® Core™ i7-5557U
+    * - Intel® Core™ i7-5557U processor
       - NUC5I7RYB
 
-    * - Intel® Core™ i5-6260U
+    * - Intel® Core™ i5-6260U processor
       - NUC6I5SYH
 
-\* Other names and brands may be claimed as the property of others.
+*Intel, Celeron, Xeon, Intel Atom, and Intel Core are trademarks of Intel
+Corporation or its subsidiaries.*

source/reference/image-types.rst

@@ -51,17 +51,6 @@ Table 2 lists the currently available images that are platform specific.
    * - aws.img
      - Image suitable for use with Amazon\* AWS\*.
 
-   * - azure.vhd
-     - Virtual Hard Disk for use on Microsoft\* Azure\* cloud platform.
-
-   * - azure-docker.vhd
-     - Virtual Hard Disk for use on Microsoft Azure cloud platform with
-       Docker\* pre-installed.
-
-   * - azure-machine-learning.vhd
-     - Virtual Hard Disk for use on Microsoft Azure cloud platform with the
-       `machine-learning-basic` bundle installed.
-
    * - cloudguest.img
      - Image with generic cloud guest virtual machine (VM) requirements
        installed.
@@ -69,8 +58,8 @@ Table 2 lists the currently available images that are platform specific.
    * - gce.tar
      - Image with the Google Compute Engine (GCE) specific kernel.
 
-   * - hyperv.vhdx
-     - Virtual Hard Disk for use with Microsoft Hyper-V\* hypervisor. Includes
+   * - azure-hyperv.vhd
+     - Image for Microsoft* Azure and Hyper-V generation 1 VMs. Includes
        :ref:`optimized kernel <vm-kernels>` for Hyper-V.
 
    * - kvm.img

source/reference/index.rst

@@ -13,3 +13,5 @@ features.
    bundles/bundles
    system-requirements
    image-types
+   man-pages
+   tutorial-ratings

source/reference/system-requirements.rst

@@ -4,11 +4,19 @@ Recommended minimum system requirements
 #######################################
 
 |CL-ATTR| can run on most modern hardware and is capable of running with
-modest hardware resources. Out of the box, |CL| can run on a single CPU core, 1 GB RAM, and minimum of disk space of: 
+modest hardware resources. Out of the box, |CL| can run on a single CPU core,
+1 GB RAM, and minimum of disk space of:
 
 * 4 GB for the *live server*
 * 20 GB for the *live desktop*
 
+.. caution::
+
+   Advanced users who wish to install on a disk using less than the recommended
+   space requirements may use the flag ``--skip-validation-size``. Use of this
+   flag may cause the installation to fail due to inadequate disk space.
+
+
 For use cases requiring minimal resources, |CL| :ref:`about <about>` can
 be used to create a highly customized installation that can even run on a
 system with a 128MB of memory and 600MB of disk space, for example.
@@ -31,7 +39,9 @@ System requirements
 *******************
 
 |CL| requires an x86 64-bit processor which supports Intel® Streaming SIMD
-Extensions 4.2 (Intel® SSE 4.2), and it requires a system that supports UEFI.
+Extensions 4.2 (Intel® SSE 4.2). 
+
+For information on the boot loader, see the `clr-boot-manager readme`_ .
 
 The |CL| installer performs a system compatibility check upon booting. To
 manually verify system compatibility with |CL|, run the :ref:`compatibility
@@ -44,19 +54,19 @@ these features:
 
 * Instruction Set Extensions:
 
-  - Supplemental Streaming SIMD Extension 3 (Intel® SSSE3)
+  - Supplemental Streaming SIMD Extension 3 (SSSE3)
   - Intel® Streaming SIMD Extensions 4.1 (Intel® SSE 4.1)
   - Intel® Streaming SIMD Extensions 4.2 (Intel® SSE 4.2)
   - Carry-less Multiplication (PCLMUL)
 
 The following processor families have been verified to run |CL|:
 
-* Intel® Core™ Processor Family (2nd generation or greater)
+* Intel® Core™ processor family (2nd generation or greater)
 * Intel® Xeon® E3-xxxx processor
 * Intel® Xeon® E5-xxxx processor
 * Intel® Xeon® E7-xxxx processor
-* Intel® Atom® processor C Series
-* Intel® Atom® processor E Series
+* Intel Atom® processor C Series
+* Intel Atom® processor E Series
 
 
 Recommended configurations
@@ -78,3 +88,8 @@ Graphics     Device with openGL support (e.g. Intel HD/UHD Graphics)
 Network      Active Internet connection
 =========    ===============================
 
+
+*Intel, Intel Core, Xeon, Intel Atom, and the Intel logo are trademarks of
+Intel Corporation or its subsidiaries.*
+
+.. _clr-boot-manager readme: https://github.com/clearlinux/clr-boot-manager

source/reference/tutorial-ratings.rst

@@ -0,0 +1,52 @@
+.. _tutorial-ratings:
+
+Tutorial difficulty ratings
+###########################
+
+Tutorial difficulty ratings provide a simple way to start using and developing with |CL-ATTR|. If you're new to the distro, we suggest starting with ``Easy`` tutorials and  working towards the more ``Difficult``. Ratings not only expose learning paths but also provide a starting point from which to advance or improve use cases, so be sure to :ref:`share your insights <collaboration>`. Three main metrics help us to determine how to rate a tutorial: 
+
+.. contents::
+   :local:
+   :depth: 1
+
+The sum total of these metrics, the rating shown in Figure 1, represents the
+ability to successfully complete a tutorial based on skill level, balanced against the risk of failure.   
+
+.. figure:: /_figures/reference/tutorial-ratings-01.svg
+   :scale: 100%
+   :alt: Tutorial difficulty ratings
+
+   Figure 1: Tutorial difficulty ratings
+
+Time and complexity
+*******************
+
+Are there about 8 or more *complex* steps? *Complex steps* are those that: 
+
+* Require more than one action 
+* Require external reading/review
+* Include explanation or context
+* Give alternative(s) 
+
+This metric factors in the cognitive load and its impact on a user. 
+
+User experience level
+*********************
+
+Our tutorials primarily target two types of Linux users.
+
+**Experienced** A Linux\* user who is familiar with common topics like userspace, networking, sudo privileges, services, and more. 
+
+**Advanced** A Linux user who is beyond Experienced and is familiar with most sysadmin and programming topics. 
+
+This metric establishes a starting point for skills, based on user
+experience. 
+
+Impact of failure
+******************
+
+The impact of failure calculates the risk of failing to complete a tutorial as a result of entering incorrect data or configuration, or failing to follow the steps in the given order. We estimate the potential state of a system, given these failure scenarios and their severity. This metric also factors in the ability to troubleshoot and recover when faced with errors. Therefore, the final impact incorporates the previous two metrics while it helps to predict an appropriate difficulty rating. 
+
+* Will impact of errors be inconsequential? ``Easy``
+* Will impact of errors cause inconvenience (but system still works)? ``Moderate``
+* Will impact of errors cause system failure (difficult to recover)? ``Difficult``

source/tutorials/apache-spark.rst

@@ -1,142 +0,0 @@
-  .. _spark:
-
-Apache\* Spark\*
-################
-
-This tutorial describes how to install, configure, and run Apache Spark on
-|CL-ATTR| on a single machine running the master daemon and a worker daemon.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Description
-***********
-
-Apache Spark is a fast, general-purpose cluster computing system with
-the following features:
-
-*  Provides high-level APIs in Java\*, Scala\*, Python\*, and R\*.
-*  Includes an optimized engine that supports general execution graphs.
-*  Supports high-level tools including Spark SQL, MLlib, GraphX, and Spark
-   Streaming.
-
-
-Prerequisites
-*************
-
-* |CL| installed on your host system.
-
-  For detailed instructions on installing |CL| on a bare metal system, visit
-  the :ref:`bare metal installation guide <bare-metal-install-desktop>`.
-
-* Before installing any new packages, update |CL| with the following command:
-
-  .. code-block:: bash
-
-     sudo swupd update
-
-Install Apache Spark
-********************
-
-Apache Spark is included in the :command:`big-data-basic` bundle. To install the
-framework, run the following command:
-
-.. code-block:: bash
-
-   sudo swupd bundle-add big-data-basic
-
-Configure Apache Spark
-**********************
-
-#. Create the configuration directory:
-
-   .. code-block:: bash
-
-      sudo mkdir /etc/spark
-
-#. Copy the default templates from :file:`/usr/share/defaults/spark` to
-   :file:`/etc/spark`:
-
-   .. code-block:: bash
-
-      sudo cp /usr/share/defaults/spark/* /etc/spark
-
-   .. note:: Since |CL| is a stateless system, you should never modify the
-      files under the :file:`/usr/share/defaults` directory. The software
-      updater overwrites those files.
-
-#. Copy the template files shown below to create custom configuration files:
-
-   .. code-block:: bash
-
-      sudo cp /etc/spark/spark-defaults.conf.template /etc/spark/spark-defaults.conf
-      sudo cp /etc/spark/spark-env.sh.template /etc/spark/spark-env.sh
-      sudo cp /etc/spark/log4j.properties.template /etc/spark/log4j.properties
-
-#. Edit the :file:`/etc/spark/spark-env.sh` file and add the
-   :envvar:`SPARK_MASTER_HOST` variable. Replace the example address below
-   with your localhost IP address. View your IP address using the
-   :command:`hostname -I` command.
-
-   .. code-block:: bash
-
-      SPARK_MASTER_HOST="10.300.200.100"
-
-   .. note:: This optional step enables the master's web user interface to
-      view information needed later in this tutorial.
-
-#. Edit the :file:`/etc/spark/spark-defaults.conf` file and update the
-   :envvar:`spark.master` variable with the `SPARK_MASTER_HOST` address and port
-   `7077`.
-
-   .. code-block:: bash
-
-      spark.master    spark://10.300.200.100:7077
-
-Start the master server and a worker daemon
-*******************************************
-
-#. Start the master server:
-
-   .. code-block:: bash
-
-      sudo /usr/share/apache-spark/sbin/./start-master.sh
-
-#. Start one worker daemon and connect it to the master using the
-   :envvar:`spark.master` variable defined earlier:
-
-   .. code-block:: bash
-
-      sudo /usr/share/apache-spark/sbin/./start-slave.sh spark://10.300.200.100:7077
-
-#. Open an internet browser and view the worker daemon information using
-   the master's IP address and port `8080`:
-
-   .. code-block:: bash
-
-      http://10.300.200.100:8080
-
-Run the Spark wordcount example
-*******************************
-
-#. Run the wordcount example using a file on your local host and output the
-   results to a new file with the following command:
-
-   .. code-block:: bash
-
-      sudo spark-submit /usr/share/apache-spark/examples/src/main/python/wordcount.py ~/Documents/example_file > ~/Documents/results
-
-#. Open an internet browser and view the application information using
-   the master's IP address and port `8080`:
-
-   .. code-block:: bash
-
-      http://10.300.200.100:8080
-
-#. View the results of the wordcount application in the :file:`~/Documents/results` file.
-
-**Congratulations!**
-
-You have successfully installed and set up a standalone Apache Spark cluster,
-and ran a simple wordcount example.

source/tutorials/broadcom.rst

@@ -4,7 +4,7 @@ Broadcom\* Drivers
 ##################
 
 Broadcom manufactures wireless network interfaces, including devices that
-support WiFi and Bluetooth. 
+support WiFi and Bluetooth® technology.
 
 Broadcom wireless devices on Linux\* have a lot of different combinations of
 possible required software depending on the exact model of your device. These
@@ -96,7 +96,7 @@ and has to be built as an out-of-tree kernel module.
    It is recommended to use the :ref:`LTS kernel <compatible-kernels>` if you
    have to use this driver.
 
-#. See if your device is supported and download the **Linux® STA 64-bit
+#. See if your device is supported and download the **Linux\* STA 64-bit
    driver** from
    `Broadcom's download website
    <https://www.broadcom.com/support/download-search?pg=&pf=Wireless+LAN+Infrastructure>`_
@@ -236,3 +236,4 @@ Troubleshooting
   different path than expected. Check the output of :command:`sudo dmesg |
   grep -i firmware` for firmware loading issues.
 
+*The Bluetooth® word mark and logos are registered trademarks owned by Bluetooth SIG, Inc. and any use of such marks by Intel Corporation is under license.*

source/tutorials/docker.rst

@@ -143,8 +143,8 @@ typically located at :file:`/etc/docker/daemon.json`. |CL| features a
 
       sudo systemctl restart docker
 
-Pulling and running an image from Docker Hub
-********************************************
+Pulling and running an image from Docker Hub\*
+**********************************************
 
 `Docker Hub`_ is a publicly available container image repository which
 comes pre-configured with Docker. In the example below we will pull and run

source/tutorials/flatpak.rst

@@ -3,7 +3,7 @@
 Flatpak\*
 #########
 
-This tutorial shows how to install a `Flatpak`_ app on |CL| using Gnome\* Software
+This tutorial shows how to install a `Flatpak`_ app on |CL| using GNOME\* Software
 and the command line.
 
 .. contents::
@@ -37,21 +37,21 @@ Prerequisites
 
      sudo swupd bundle-add desktop-autostart
 
-Install a Flatpak app with Gnome Software
+Install a Flatpak app with GNOME Software
 *****************************************
 
-|CL| desktop comes with Gnome Software installed. Flatpak apps can be
-installed from within Gnome Software.
+|CL| desktop comes with GNOME Software installed. Flatpak apps can be
+installed from within GNOME Software.
 
-#. Launch Gnome Software from your desktop.
+#. Launch GNOME Software from your desktop.
 
 #. Search for the Flatpak app that you want to install, as shown in Figure 1.
 
    .. figure:: /_figures/flatpak/flatpak-01.png
       :scale: 50%
-      :alt: Searching for Filezilla app in Gnome Software
+      :alt: Searching for Filezilla app in GNOME Software
 
-      Figure 1: Searching for Filezilla app in Gnome Software
+      Figure 1: Searching for Filezilla\* app in GNOME Software
 
 #. When you find the app you want to install, click it to view application
    details.
@@ -61,12 +61,12 @@ installed from within Gnome Software.
 
    .. figure:: /_figures/flatpak/flatpak-02.png
       :scale: 50%
-      :alt: Filezilla Flatpak detail page in Gnome Software
+      :alt: Filezilla Flatpak detail page in GNOME Software
 
-      Figure 2: Filezilla Flatpak detail page in Gnome Software
+      Figure 2: Filezilla Flatpak detail page in GNOME Software
 
 #. After installation is complete, the new application will be in your
-   Gnome applications list, as shown in Figure 3.
+   GNOME applications list, as shown in Figure 3.
 
    .. figure:: /_figures/flatpak/flatpak-03.png
       :scale: 50%
@@ -81,7 +81,7 @@ Install a Flatpak with the command line
 
 Both Flathub and the Clear Linux Store provide the command line instructions
 for installing a Flatpak. Figure 4 shows the command line instructions to
-install Filezilla from the Clear Linux Store:
+install Filezilla\* from the Clear Linux Store:
 
 .. figure:: /_figures/flatpak/flatpak-04.png
    :scale: 50%

source/tutorials/fmv.rst

@@ -277,8 +277,10 @@ application code.
 
 You have successfully installed an FMV development environment on |CL|.
 Furthermore, you used cutting edge compiler technology to improve the
-performance of your application based on Intel® architecture technology and
+performance of your application based on Intel® architecture and
 profiling of the specific execution of your application.
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _GCC:  https://gcc.gnu.org
 .. _make-fmv-patch: https://github.com/clearlinux/make-fmv-patch

source/tutorials/hpc.rst

@@ -0,0 +1,736 @@
+.. _hpc:
+
+HPC Cluster 
+###########
+
+This tutorial demonstrates how to set a simple :abbr:`HPC (High
+Performance Computing)` cluster using `Slurm`_, `MUNGE`_, and 
+`pdsh`_.  For this tutorial, this cluster consists of a controller node 
+and four worker nodes, as shown in Figure 1.  For the sake of simplicity,
+each node resides on a separate host and their hostnames are hpc-controller,
+hpc-worker1, hpc-worker2, hpc-worker3, and hpc-worker4.  
+
+.. rst-class:: dropshadow
+
+.. figure:: ../_figures/hpc/hpc-01.png
+   :alt: Simple HPC cluster
+
+   Figure 1: Simple HPC cluster
+
+The configuration is intentionally kept simple, notably avoiding setting 
+up cgroups and accounting. These and many more additional configuration 
+options can be added later.
+
+.. note::
+
+   This tutorial assumes you start with a new installation of |CL| for all 
+   nodes.
+
+Prerequisites
+*************
+
+* Knowledge and experience with HPC 
+* Familiarity with Slurm, MUNGE, and pdsh
+* All nodes have synchronized clocks (typically by NTP)
+
+.. contents::
+    :local:
+    :depth: 1
+
+Set up controller node
+**********************
+
+In this step, install the cluster tools, configure and enable the MUNGE service,
+and enable the Slurm controller service.
+
+#. Install |CL| on the controller node, add a user with adminstrator 
+   privilege, and set its hostname to `hpc-controller`.  
+
+#. Boot it up and log in.
+
+#. Update |CL| to the latest version.
+
+   .. code-block:: bash
+
+      sudo swupd update
+
+#. Set the date and time to synchronize with an NTP server.
+
+   .. code-block:: bash
+
+      sudo timedatectl set-ntp true
+
+#. Install the `cluster-tools` bundle.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add cluster-tools
+
+#. Create a MUNGE key and start the MUNGE service.
+   
+   a. Create the MUNGE key.
+      
+      .. code-block:: bash
+         
+         sudo mkdir /etc/munge
+         dd if=/dev/urandom bs=1 count=1024 | sudo tee -a /etc/munge/munge.key
+
+   #. Set the ownership to `munge` and set the correct access permissions.
+     
+      .. code-block:: bash
+         
+         sudo chown munge: /etc/munge/munge.key
+         sudo chmod 400 /etc/munge/munge.key      
+
+   #. Start the MUNGE service and set it to start automatically on boot.
+
+      .. code-block:: bash
+               
+         sudo systemctl enable munge --now
+
+#. Test MUNGE.
+
+   a. Create a MUNGE credential.
+
+      .. code-block:: bash
+
+         munge -n
+
+      Example output:
+
+      .. code-block:: console
+
+         MUNGE:AwQFAAC8QZHhL/+Fqhalhi+ZJBD5LavtMa8RMles1aPq7yuIZq3LtMmrB7KQZcQjG0qkFmoIIvixaCACFe1stLmF4VIg4Bg/7tilxteXHS940cuZ/TxpIuqC6fUH8zLgUZUPwJ4=:
+
+
+   #. Validate a MUNGE credential.  
+      
+      .. code-block:: bash
+
+         munge -n | unmunge | grep STATUS
+
+      Example output:
+        
+      .. code-block:: console
+
+         STATUS:           Success (0)
+
+#. Start the Slurm controller service and enable it to start automatically 
+   on boot.
+
+   .. code-block:: bash
+
+      sudo systemctl enable slurmctld --now
+
+Set up worker nodes
+*******************
+
+For each worker node, perform these steps:
+
+#. Install |CL| on the worker node, add a user with adminstrator privilege,
+   and set its hostname to `hpc-worker` plus its number, i.e. hpc-worker1, 
+   hpc-worker2, etc.  
+
+   Ensure the username is the same as the one on the controller node.  This
+   is needed to simplify password-less-SSH-access setup, which is needed for
+   pdsh, in the next section.  
+
+#. Boot it up and log in.
+
+#. Update |CL| to the latest version.
+
+   .. code-block:: bash
+
+      sudo swupd update
+
+#. Set the date and time to synchronize with an NTP server.
+
+   .. code-block:: bash
+
+      sudo timedatectl set-ntp true
+
+#. Install the `cluster-tools` bundle.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add cluster-tools
+
+Set up password-less SSH access and pdsh on all nodes
+*****************************************************
+
+To efficiently manage a cluster, it is useful to have a tool
+that allows issuing the same command to multiple nodes at once.  
+And that tool is :abbr:`pdsh (parallel distributed shell)`, which is included
+with the `cluster-tools` bundle. pdsh is built with Slurm support, so it can 
+access hosts as defined in the Slurm partitions.  pdsh relies on password-less 
+SSH access in order for it to work properly. There are two ways to set up 
+pasword-less SSH authentication: key-based or host-based.  In this case, 
+the latter approach will be used.  The controller authenticates a user and 
+all worker nodes will trust that authentication and not ask the user to 
+enter a password again.
+
+#. Configure the controller node.
+
+   a. Log into the controller node.
+
+   #. Configure the SSH service for host-based authentication.
+
+      .. code-block:: bash
+
+         sudo tee -a /etc/ssh/ssh_config << EOF
+	 HostbasedAuthentication yes
+         EnableSSHKeysign yes
+	 EOF
+
+   #. Restart the SSH service.
+
+      .. code-block:: bash
+
+         sudo systemctl restart sshd
+
+#. Configure each worker node.
+
+   a. Configure SSH service for host-based authentication.
+
+      .. code-block:: bash
+
+	 sudo tee -a /etc/ssh/sshd_config << EOF 
+         HostbasedAuthentication yes
+         IgnoreRhosts no
+         UseDNS yes
+	 EOF
+
+   #. Create the :file:`/etc/hosts.equiv` file and add the controller's
+      :abbr:`FQDN (fully qualified domain name)`.  This tells the worker 
+      node to accept connection from the controller.  
+
+      For example:
+
+      .. code-block:: console
+
+         hpc-controller.my-domain.com
+
+   #. Set its permission to root access only.
+
+      .. code-block:: bash
+
+         sudo chmod 600 /etc/hosts.equiv
+
+   #. Add the controller's FQDN to :file:`/root/.shosts`.  This allows
+      host-based authentication for the root account so that 
+      actions requiring sudo privileges can be performed.
+
+      .. code-block:: bash
+
+         sudo cp -v /etc/hosts.equiv /root/.shosts
+
+   #. Using the controller's FQDN in :file:`/etc/hosts.equiv`, scan for its 
+      RSA public key and copy it to :file:`/etc/ssh/ssh_known_hosts`.
+      Verify the scanned RSA public key matches the controller's 
+      :file:`/etc/ssh/ssh_rsa_key.pub` file.
+
+      .. code-block:: bash
+
+        sudo ssh-keyscan -t rsa -f /etc/hosts.equiv > ~/ssh_known_hosts
+        sudo cp -v ~/ssh_known_hosts /etc/ssh
+	rm ~/ssh_known_hosts
+
+   #. Restart the SSH service.
+
+      .. code-block:: bash
+
+         sudo systemctl restart sshd
+
+#. On the controller node, SSH into each worker node without having to enter 
+   a password. At the first-time connection to each host, you'll be asked to 
+   add the unknown host to the :file:`$HOME/.ssh/known_hosts` file.  Accept
+   the request.  This is will make future SSH connections to each host be 
+   non-interactive.
+
+   .. code-block:: bash
+
+      ssh <worker-node>
+
+   .. note::
+
+      Setting up host-based authentication on 
+      :abbr:`CSP (Cloud Service Provider)` environments such as Microsoft Azure
+      and Amazon AWS may require some tweaking on the worker nodes' SSH 
+      configurations due to the CSP's virtual network setup. In general, 
+      cloud VMs have a public and private DNS name.  When SSH'ing from the 
+      controller to a worker node, the SSH client may send the controller's 
+      private DNS name, usually something with "internal" in the name,
+      as the `chost` instead of its public FQDN as expected in worker node's 
+      :file:`/etc/hosts.equiv`, :file:`/root/.shosts`, and 
+      :file:`/etc/ssh/ssh_known_hosts` files.  If the above configurations 
+      do not work, meaning you're asked to enter a password when
+      SSH'ing from the controller to a worker node, on a cloud VM, here are 
+      some suggestions for debugging the problem:
+
+      #. On the controller, try to identify the chost data sent by the SSH 
+         client using :command:`ssh -vvv <worker-node>`.  Look for `chost`
+         in the debug log.  If the chost value is different than the controller's
+	 FQDN listed in worker node's :file:`/etc/hosts.equiv`, 
+	 :file:`/root/.shosts`, and :file:`/etc/ssh/ssh_known_hosts` files, 
+	 then that is likely the cause of the problem.  In some cases, chost 
+	 data may not be shown.  If so, it's safe to assume that the SSH client 
+	 is using the controller's private DNS name as the chost.  Proceed to 
+	 steps 2 and 3 below to fix the problem.
+
+      #. Get the controller's private DNS name either by the above step or by
+         getting it from your system administrator.
+
+      #. On the worker node, make these changes:
+
+         #. Change the controller's FQDN in :file:`/etc/hosts.equiv`,
+            :file:`/root/.shosts`, and :file:`/etc/ssh/ssh_known_hosts`
+	    to its private DNS name.
+
+         #. Restart the SSH service on the worker node.
+
+         #. Retest the connection from the controller node to the worker node. 
+            If that still doesn't work, try the SSH directive 
+	    `HostbasedUsesNameFromPacketOnly yes` which tell the SSH service
+	    to accept the supplied host name as is and not try to resolve it.
+	    Also, set the directive `UseDNS` to `no` to disable host name lookup.  
+#. Verify you can issue a simple command over SSH without typing a password.
+
+   a. Issue the :command:`hostname` command.
+
+      .. code-block:: bash
+
+         ssh <worker-node> hostname
+
+   #. Issue the :command:`hostname` command with :command:`sudo`.
+
+      .. code-block:: bash
+
+         ssh <worker-node> sudo hostname
+
+   In both cases, you should get a response with the worker node's hostname.
+   If the `sudo` version requires additional permission, grant the user
+   `NOPASSWD` privilege.  For example:
+
+   #. Edit the sudoer file.
+
+      .. code-block:: bash
+
+         sudo visudo
+
+   #. Add the following:
+
+      .. code-block:: console
+
+         <user> ALL=(ALL) NOPASSWD: ALL
+
+Create :file:`slurm.conf` configuration file
+********************************************
+
+On the controller, create a new :file:`slurm.conf` configuration file 
+that contains general settings, each node's hardware resource information, 
+grouping of nodes into different partitions, and scheduling settings for 
+each partition.  This file will be copied to all worker nodes in the cluster.  
+
+#. Create a base :file:`slurm.conf` configuration file.
+
+   .. code-block:: bash
+
+      sudo mkdir -p /etc/slurm
+      sudo cp -v /usr/share/defaults/slurm/slurm.conf /etc/slurm
+
+#. Add the controller information.
+
+   a. :command:`sudoedit` the :file:`slurm.conf` file. Set the `ControlMachine`
+      value to the controller's resolvable hostname.
+
+      For example:
+
+      .. code-block:: console
+
+	 ControlMachine=hpc-controller
+
+      .. note:: 
+
+         Assuming the controller's FQDN is resolvable, specifying the 
+	 controller's IP address with the `ControlAddr` key is optional.  
+	 However, it maybe helpful to add it.  
+
+   #. Save and exit.
+
+#. Add the worker nodes information.
+   
+   a. Create a file containing a list of the worker nodes.
+
+      .. code-block:: bash
+
+         cat > worker-nodes-list << EOF
+	 hpc-worker1
+	 hpc-worker2
+	 hpc-worker3
+	 hpc-worker4
+	 EOF
+
+   #. Using pdsh, get the hardware configuration of each node.
+
+      .. code-block:: bash
+
+         pdsh -w ^worker-nodes-list slurmd -C
+
+      Example output:
+
+      .. code-block:: console
+
+         hpc-worker4: NodeName=hpc-worker4 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1915
+         hpc-worker4: UpTime=0-01:23:28
+         hpc-worker3: NodeName=hpc-worker3 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1663
+         hpc-worker3: UpTime=0-01:33:41
+         hpc-worker2: NodeName=hpc-worker2 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=721
+         hpc-worker2: UpTime=0-01:34:56
+         hpc-worker1: NodeName=hpc-worker1 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=721
+         hpc-worker1: UpTime=0-01:39:21
+
+   #. :command:`sudoedit` the :file:`slurm.conf` file.  Append each worker node 
+      information, but without the `UpTime`, under the `COMPUTE NODES` section.
+
+      .. tip::
+
+         It is strongly recommended to set the `RealMemory` value for each 
+	 worker node slightly below, say 90%, what was reported by 
+	 :command:`slurmd -C` 
+	 in case some memory gets use by some processes, which would 
+	 cause Slurm to make a node not available due to its memory 
+	 resource falling below the stated value in the configuration file.    
+
+      Here's an example with four worker nodes:
+
+      .. code-block:: console
+
+         # 
+         # COMPUTE NODES (mode detailed values reported by "slurmd -C" on each node)
+         NodeName=hpc-worker1 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=648
+         NodeName=hpc-worker2 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=648
+         NodeName=hpc-worker3 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1497
+         NodeName=hpc-worker4 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1723
+
+   #. Create partitions.
+
+      A Slurm partition is basically the grouping of worker nodes.  
+      Give each partition a name and decide which worker node(s) belong to 
+      it.
+
+      For example:
+   
+      .. code-block:: console
+      
+         PartitionName=workers Nodes=hpc-worker1, hpc-worker2, hpc-worker3, hpc-worker4 Default=YES MaxTime=INFINITE State=UP
+         PartitionName=debug Nodes=hpc-worker1, hpc-worker3 MaxTime=INFINITE State=UP
+
+   #. Save and exit.
+
+   #. Set the ownership of the :file:`slurm.conf` file to `slurm`.
+
+      .. code-block:: bash
+         
+	 sudo chown slurm: /etc/slurm/slurm.conf
+
+#. On the controller node, restart the Slurm controller service.
+
+   .. code-block:: bash
+
+      sudo systemctl restart slurmctld
+
+#. Verify the Slurm controller service restarted without any issues before 
+   proceeding.
+
+   .. code-block:: bash
+
+      sudo systemctl status slurmctld
+
+   
+
+Copy MUNGE key and :file:`slurm.conf` to all worker nodes
+*********************************************************
+
+On the controller node, using pdsh, in conjunction with the list of 
+defined nodes in the :file:`slurm.conf`, copy it and the MUNGE key to 
+all worker nodes.  
+
+#. On the controller node, copy the MUNGE key to all worker nodes and start the 
+   MUNGE service.
+          
+   a. Create the :file:`/etc/munge/` directory on each node.
+
+      .. code-block:: bash
+         
+	 sudo pdsh -P workers mkdir /etc/munge
+   
+   #. Copy the MUNGE key over.
+
+      .. code-block:: bash
+         
+	 sudo pdcp -P workers /etc/munge/munge.key /etc/munge
+
+   #. Set the ownership of the :file:`munge.key` file to `munge`.
+
+      .. code-block:: bash
+
+         sudo pdsh -P workers chown munge: /etc/munge/munge.key
+
+    #. Start the MUNGE service and set it to start automatically on boot.
+
+      .. code-block:: bash
+
+         sudo pdsh -P workers systemctl enable munge --now
+
+    #. Verify the MUNGE service is running.
+
+       .. code-block:: bash
+
+          sudo pdsh -P workers "systemctl status munge | grep Active"
+      
+       Example output:
+
+       .. code-block:: console
+
+          hpc-worker3:      Active: active (running) since Wed 2020-04-15 19:47:58 UTC; 55s ago
+          hpc-worker4:      Active: active (running) since Wed 2020-04-15 19:47:58 UTC; 55s ago
+          hpc-worker2:      Active: active (running) since Wed 2020-04-15 19:47:59 UTC; 54s ago
+          hpc-worker1:      Active: active (running) since Wed 2020-04-15 19:47:59 UTC; 54s ago
+
+#. On the controller node, copy the :file:`slurm.conf` file to all 
+   worker nodes and start the slurmd service on them.
+
+   a. Create the :file:`/etc/slurm/` directory on each worker node.
+
+      .. code-block:: bash
+         
+	 sudo pdsh -P workers mkdir /etc/slurm
+
+   #. Copy the :file:`slurm.conf` file over.
+
+      .. code-block:: bash
+         
+	 sudo pdcp -P workers /etc/slurm/slurm.conf /etc/slurm
+
+   #. Set the ownership of the :file:`slurm.conf` file to `slurm`.
+
+      .. code-block:: bash
+
+         sudo pdsh -P workers chown slurm: /etc/slurm/slurm.conf
+
+   #. Start the Slurm service and set it automatically start on boot.
+
+      .. code-block:: bash
+         
+	 sudo pdsh -P workers systemctl enable slurmd --now
+
+   #. Verify the slurmd service is running.
+
+      .. code-block:: bash
+
+         sudo pdsh -P workers systemctl status slurmd | grep Active
+      
+      Example output:
+
+      .. code-block:: console
+
+         hpc-worker3:      Active: active (running) since Wed 2020-04-15 19:39:22 UTC; 1min 17s ago
+         hpc-worker4:      Active: active (running) since Wed 2020-04-15 19:39:22 UTC; 1min 17s ago
+         hpc-worker2:      Active: active (running) since Wed 2020-04-15 19:39:23 UTC; 1min 17s ago
+         hpc-worker1:      Active: active (running) since Wed 2020-04-15 19:39:23 UTC; 1min 17s ago
+
+Verify controller can run jobs on all nodes
+*******************************************
+
+#. Check the state of the worker nodes.  
+
+   .. code-block:: bash
+
+      sinfo
+
+   Example output:
+
+   .. code-block:: console
+
+      PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
+      workers*     up   infinite      4   idle hpc-worker[1-4]
+      debug        up   infinite      2   idle hpc-worker[1,3]
+
+   .. tip::
+
+      If the nodes are in a "down" state, put them in the "idle" state.
+
+      For example:
+
+      .. code-block:: bash
+
+         sudo scontrol update nodename=hpc-worker[1-4] state=idle reason=""
+
+      Additional `Slurm troubleshooting tips`_.  
+
+#. And finally, verify Slurm can run jobs on all 4 worker nodes by issuing
+   a simple :command:`hostname` command.
+
+   .. code-block:: bash
+
+      srun -N4 -p workers hostname
+   
+   Example output:
+     
+   .. code-block:: console
+
+      hpc-worker4
+      hpc-worker3
+      hpc-worker1
+      hpc-worker2
+
+Create and run example scripts
+******************************
+
+Example 1: Return the hostname of each worker and output to :file:`show-hostnames.out`
+======================================================================================
+
+#. On the controller node, create the Slurm :file:`show-hostnames.sh` script.
+
+   .. code-block:: bash
+
+      cat > show-hostnames.sh << EOF
+      #!/bin/bash
+      #
+      #SBATCH --job-name=show-hostnames
+      #SBATCH --output=show-hostnames.out
+      #
+      #SBATCH --ntasks=4
+      #SBATCH --time=10:00
+      #SBATCH --mem-per-cpu=100
+      #SBATCH --ntasks-per-node=1
+
+      srun hostname
+      EOF
+
+#. Execute the script.
+
+   .. code-block:: bash
+
+      sbatch show-hostnames.sh
+   
+   The result will appear on the first node of the partition used. As no 
+   partition was explicitly specified, it would be the default partition.
+
+#. View the result.
+
+   .. code-block:: bash
+
+      pdsh -w hpc-worker1 "cat show-hostnames.out"
+
+   Example output:
+
+   .. code-block:: console
+
+      hpc-worker1: hpc-worker3
+      hpc-worker1: hpc-worker4
+      hpc-worker1: hpc-worker1
+      hpc-worker1: hpc-worker2
+
+Example 2: An MPI "Hello, World!" program
+=========================================
+
+#. On the controller node, create the :file:`mpi-helloworld.c` program.
+
+   .. code-block:: bash
+
+      cat > mpi-helloworld.c << EOF 
+      #include <stdio.h>
+      #include <unistd.h>
+      #include <mpi.h>
+
+      int main(int argc, char** argv)
+      {
+        // Init the MPI environment
+        MPI_Init(NULL, NULL);
+
+        // Get the number of processes
+        int world_size;
+        MPI_Comm_size(MPI_COMM_WORLD, &world_size);
+
+        // Get the rank of the process
+        int world_rank;
+        MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
+
+        // Get the name of the processor
+        char processor_name[MPI_MAX_PROCESSOR_NAME];
+        int name_len;
+        MPI_Get_processor_name(processor_name, &name_len);
+
+        // Print a hello world message
+        printf("Hello, World! from from processor %s, rank %d out of %d processors\n", processor_name, world_rank, world_size);
+
+        // Finalize the MPI environment
+        MPI_Finalize();
+     }
+     EOF
+
+#. Add the `c-basic` and `devpkg-openmpi` bundles, which are needed to compile
+   it.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add c-basic devpkg-openmpi
+
+#. Compile it.
+   
+   .. code-block:: bash
+
+      mpicc -o mpi-helloworld mpi-helloworld.c
+
+#. Copy the binary to all worker nodes.
+
+   .. code-block:: bash
+
+      pdcp -P workers ./mpi-helloworld $HOME
+
+#. Create a Slurm batch script to run it.
+
+   .. code-block:: bash
+
+      cat > mpi-helloworld.sh << EOF
+      #!/bin/sh
+      #SBATCH -o mpi-helloworld.out
+      #SBATCH --nodes=4
+      #SBATCH --ntasks-per-node=1
+
+      srun ./mpi-helloworld
+      EOF
+
+#. Run the batch script.
+
+   .. code-block:: bash
+
+      sbatch mpi-helloworld.sh
+
+#. View the results on first worker node in the partition.
+
+   .. code-block:: bash
+
+      pdsh -w hpc-worker1 "cat mpi-helloworld.out"
+   
+   Example output:
+
+   .. code-block:: console
+
+      Hello, World! from from processor hpc-worker3, rank 2 out of 4 processors
+      Hello, World! from from processor hpc-worker4, rank 3 out of 4 processors
+      Hello, World! from from processor hpc-worker1, rank 0 out of 4 processors
+      Hello, World! from from processor hpc-worker2, rank 1 out of 4 processors
+
+.. _Slurm: 
+   https://en.wikipedia.org/wiki/Slurm_Workload_Manager
+
+.. _MUNGE:
+   https://dun.github.io/munge/
+
+.. _pdsh:
+   https://linux.die.net/man/1/pdsh
+
+.. _Slurm troubleshooting tips:
+   https://slurm.schedmd.com/troubleshoot.html

source/tutorials/index.rst

@@ -3,9 +3,60 @@
 Tutorials
 #########
 
-Explore our tutorials to discover what you can do with |CL|!
+Explore our tutorials to discover what you can do with |CL|, conveniently
+sorted by difficulty level! Learn about :ref:`how we evaluate tutorials <tutorial-ratings>`.
+
+.. container:: multicolumns
+
+   .. container:: column narrow
+
+      .. rst-class:: colh3
+
+      Easy
+
+      -  :ref:`docker`
+      -  :ref:`flatpak-tutorial`
+      -  :ref:`kata`
+      -  :ref:`lamp-server-install`
+      -  :ref:`proxy`
+      -  :ref:`redis`
+      -  :ref:`smb-desktop`
+      -  :ref:`smb-server`
+      -  :ref:`yubikey-u2f`
+
+   .. container:: column narrow
+
+      .. rst-class:: colh3
+
+      Moderate
+
+      -  :ref:`hadoop`
+      -  :ref:`broadcom`
+      -  :ref:`fmv`
+      -  :ref:`hpc`
+      -  :ref:`kubernetes-bp`
+      -  :ref:`mirror-upstream-server`
+      -  :ref:`nvidia-cuda`
+      -  :ref:`php`
+      -  :ref:`vmware-workstation`
+      -  :ref:`wp-install`
+
+   .. container:: column narrow
+
+      .. rst-class:: colh3
+
+      Difficult
+
+      -  :ref:`kubernetes`
+      -  :ref:`nvidia`
+      -  :ref:`openfaas`
+      -  :ref:`multi-boot`
+      -  :ref:`machine-learning`
+      -  :ref:`zfs`
+
 
 .. toctree::
+   :hidden:
    :maxdepth: 1
    :glob:
 
@@ -21,4 +72,4 @@ Older tutorials that may still be relevant to some users.
    :maxdepth: 1
    :glob:
 
-   archive/*
+   archive/*

source/tutorials/lamp-server.rst

@@ -1,10 +1,10 @@
-.. _web-server-install:
+.. _lamp-server-install:
 
-Set up a LAMP web server on |CL-ATTR|
-#####################################
+LAMP Web Server
+###############
 
 This tutorial provides instructions on how to set up a
-:abbr:`LAMP (Linux, Apache\*, MySQL, PHP)` web server on |CL-ATTR| and how
+:abbr:`LAMP (Linux\*, Apache\*, MySQL\*, PHP)` web server on |CL-ATTR| and how
 to use phpMyAdmin\* to manage an associated database. Note that this
 tutorial installs MariaDB\*, which is a drop-in replacement for MySQL\*.
 
@@ -46,13 +46,12 @@ Apache software bundle on |CL|.
 
       sudo swupd bundle-add httpd
 
-
-#. To start the Apache service, enter the following commands:
+#. Start the Apache service and set it start automatically on boot, 
+   enter the following commands:
 
    .. code-block:: bash
 
-      sudo systemctl enable httpd.service
-      sudo systemctl start httpd.service
+      sudo systemctl enable --now httpd.service
 
 #. To verify that the Apache server application is running, open a web
    browser and navigate to: \http://localhost.
@@ -60,7 +59,9 @@ Apache software bundle on |CL|.
    If the service is running, a confirmation message will appear, similar to the
    message shown in figure 1.
 
-   .. figure:: /_figures/wordpress/web-server-install-1.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-1.png
       :alt: This web server is operational from host.
       :scale: 50%
 
@@ -150,12 +151,11 @@ default values, and define a custom DocumentRoot for your web server.
 
    "It works from its new location!"
 
-#. Stop and then restart ``httpd.service``.
+#. Restart ``httpd.service``.
 
    .. code-block:: bash
 
-      sudo systemctl stop httpd.service
-      sudo systemctl start httpd.service
+      sudo systemctl restart httpd.service
 
 #. Go to \http://localhost to view the new screen. You should see your updated
    default message from step 5.
@@ -168,12 +168,11 @@ default values, and define a custom DocumentRoot for your web server.
 
       sudo nano /etc/httpd/conf.d/httpd.conf
 
-#. Stop and then restart ``httpd.service``.
+#. Restart ``httpd.service``.
 
    .. code-block:: bash
 
-      sudo systemctl stop httpd.service
-      sudo systemctl start httpd.service
+      sudo systemctl restart httpd.service
 
 #. Go to \http://localhost and verify that you can see the default screen
    again.
@@ -198,12 +197,12 @@ functionality to your web server, install PHP on your system.
 
       sudo swupd bundle-add php-basic
 
-#. To enable PHP, enter the following commands:
+#. To enable PHP and set it to start automatically on boot, enter the 
+   following commands:
 
    .. code-block:: bash
 
-      sudo systemctl enable php-fpm.service
-      sudo systemctl start php-fpm.service
+      sudo systemctl enable --now php-fpm.service
       sudo systemctl restart httpd.service
 
    After restarting the Apache service, test your PHP installation.
@@ -225,7 +224,9 @@ functionality to your web server, install PHP on your system.
 
 #. Verify that the PHP information screen appears, similar to figure 2:
 
-   .. figure:: /_figures/wordpress/web-server-install-2.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-2.png
       :alt: PHP information screen
       :width: 600
 
@@ -245,14 +246,14 @@ and is available in the database-basic |CL| bundle.
 
    .. code-block:: bash
 
-      sudo swupd bundle-add database-basic
+      sudo swupd bundle-add mariadb
 
-#. To start MariaDB after it is installed, enter the following commands:
+#. To start MariaDB after it is installed and set it to start automatically on
+   boot, enter the following commands:
 
    .. code-block:: bash
 
-      sudo systemctl enable mariadb
-      sudo systemctl start mariadb
+      sudo systemctl enable --now mariadb
 
 #. To check the status of MariaDB, enter the following command:
 
@@ -280,7 +281,7 @@ hardening.
 
       Our suggested responses follow each question.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       Enter current password for root (enter for none):
 
@@ -288,7 +289,7 @@ hardening.
    user. For a newly installed MariaDB without a set root password, the
    password is blank. Thus, press enter to continue.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       OK, successfully used password, moving on...
 
@@ -299,19 +300,19 @@ hardening.
    Set the root password to prevent unauthorized MariaDB root user logins.
    To set a root password, type 'y'.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       New password:
 
    Type the desired password for the root user.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       Re-enter new password:
 
    Re-type the desired password for the root user.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       Password updated successfully!
       Reloading privilege tables..
@@ -324,7 +325,7 @@ hardening.
    is intended only for testing and for a smoother installation. To remove
    the anonymous user and make your database more secure, type 'y'.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       ... Success!
       Disallow root login remotely? [Y/n]
@@ -333,7 +334,7 @@ hardening.
    ensures that someone cannot guess the root password from the network. To
    block any remote root login, type 'y'.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       ... Success!
       Remove test database and access to it? [Y/n]
@@ -342,7 +343,7 @@ hardening.
    This database is also intended only for testing and should be removed. To
    remove the test database, type 'y'.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       - Dropping test database...
       ... Success!
@@ -353,7 +354,7 @@ hardening.
    Reloading the privilege tables ensures all changes made so far take
    effect immediately. To reload the privilege tables, type 'y'.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       ... Success!
 
@@ -415,7 +416,9 @@ steps below for setting up a database called "WordPress".
    :ref:`mysql_secure_installation command <set-password>`. Enter your
    credentials and select :guilabel:`Go` to log in:
 
-   .. figure:: /_figures/wordpress/web-server-install-3.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-3.png
       :alt: phpMyAdmin login page
       :width:     600
 
@@ -424,7 +427,9 @@ steps below for setting up a database called "WordPress".
 #. Verify a successful login by confirming that the main phpMyAdmin page
    displays, as shown in figure 4:
 
-   .. figure:: /_figures/wordpress/web-server-install-4.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-4.png
       :alt: phpMyAdmin dashboard
       :width:     600
 
@@ -441,7 +446,9 @@ steps below for setting up a database called "WordPress".
 
 #. Click :guilabel:`Create`.
 
-   .. figure:: /_figures/wordpress/web-server-install-5.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-5.png
       :alt: Databases tab
       :width:     600
 
@@ -452,7 +459,9 @@ steps below for setting up a database called "WordPress".
 
 #. Select the :guilabel:`Privileges` tab. Figure 6 shows its contents.
 
-   .. figure:: /_figures/wordpress/web-server-install-6.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-6.png
       :alt: Privileges tab
       :width:     600
 
@@ -462,7 +471,9 @@ steps below for setting up a database called "WordPress".
    :guilabel:`Privileges` tab. The `Add user account` page appears, as shown
    in figure 7.
 
-   .. figure:: /_figures/wordpress/web-server-install-7.png
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/web-server-install-7.png
       :alt: User accounts tab
       :width:     600
 
@@ -484,7 +495,9 @@ steps below for setting up a database called "WordPress".
 
 If successful, you should see the screen shown in figure 8:
 
-.. figure:: /_figures/wordpress/web-server-install-8.png
+.. rst-class:: dropshadow
+
+.. figure:: ../_figures/wordpress/web-server-install-8.png
    :alt: User added successfully
    :width:     600
 

source/tutorials/mirror-upstream-server.rst

@@ -0,0 +1,244 @@
+.. _mirror-upstream-server:
+  
+Mirror Upstream |CL| Update Server
+##################################
+
+For organizations that want to use the |CL| upstream updates, but want the 
+benefits of a local mirror, this tutorial shows how to set up one and
+configure your |CL| clients to use it.
+
+.. contents::
+    :local:
+    :depth: 1
+
+Prerequisites
+*************
+
+* The recommended disk space for the mirror server should have at least 100GB 
+  of disk space as each complete update content is approximately 45GB.  
+
+Install up |CL| server to host updates
+**************************************
+
+#. Follow the :ref:`bare-metal-install-server` guide to install |CL| server.
+   Add a user with `Administrator` privilege.
+
+#. After installation is complete, boot it up.
+
+#. Add the `wget` bundle.  This will be used to clone the upstream |CL| server.  
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add wget
+
+Clone the |CL| update content
+*****************************
+
+|CL| periodically releases a "minversion", which is a complete update.  
+Then, subsequent releases are small updates until the next minversion.
+
+Download a minversion to start your mirror.  
+
+#. Determine a proper minversion by looking at a `Manifest.MoM`_ file 
+   for a particular release of |CL|.  
+
+   For example, if you look at 
+   https://cdn.download.clearlinux.org/update/33010/Manifest.MoM, 
+   you will see that the minversion is 32900.  So clone this version as the 
+   starting point.
+
+   .. code-block:: console
+      :emphasize-lines: 4
+
+      MANIFEST        30
+      version:        33010
+      previous:       33000
+      minversion:     32900
+      filecount:      1131
+      timestamp:      1588358889
+      contentsize:    0
+
+#. Make a directory to store the cache.
+
+   .. code-block:: bash
+
+      mkdir ~/mirror-download-clearlinux-org && cd $_
+
+#. Recursively download the :file:`update/0` folder.
+
+   .. code-block:: bash
+
+      wget --no-verbose \
+      --no-parent --recursive \
+      --no-host-directories -erobots=off \
+      --reject "index.html" https://cdn.download.clearlinux.org/update/0/
+
+#. Recursively download the :file:`update/version` folder.
+
+   .. code-block:: bash
+
+      wget --no-verbose \
+      --no-parent --recursive \
+      --no-host-directories -erobots=off \
+      --reject "index.html" https://cdn.download.clearlinux.org/update/version/
+
+#. Now, recursively download the determined minversion, which for this example 
+   is 32900.
+
+   .. code-block:: bash
+
+      wget --no-verbose \
+      --no-parent --recursive \
+      --no-host-directories -erobots=off \
+      --reject "index.html" https://cdn.download.clearlinux.org/update/32900/
+
+   .. note::
+
+      A minversion is pretty big, which is approximately 45GB.  Depending on your
+      proximity to the upstream server and your connection speed to the Internet,
+      it may take up to a couple of days or more to complete the download.  So
+      be patient.  
+
+#. Download later versions, up to the latest, if you like.
+
+Setup a web server to host the mirrored content
+***********************************************
+
+By design, the |CL| swupd client communicates with the update server using
+HTTPS for security reasons.  However, it can use HTTP by adding the 
+:command:`--allow-insecure-http` flag, if needed.  Setting an HTTPS is a lot 
+more involved.  For this tutorial, we'll just use an HTTP server for 
+demonstration purpose. 
+
+#. Install the `nginx` bundle.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add nginx
+
+#. Configure the web server.
+
+   a. Create a symbolic link to the mirrored update content directory.
+
+      .. code-block:: bash
+      
+         sudo mkdir -p /var/www && cd $_
+         sudo ln -sf $HOME/mirror-download-clearlinux-org mirror-download-clearlinux-org
+
+   #. Set up nginx configuration files.
+
+      .. code-block:: bash
+
+         sudo mkdir -p /etc/nginx/conf.d
+         sudo cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf
+
+   #. Grant $USER permission to run the web server.
+      
+      .. code-block:: bash
+
+         sudo tee -a /etc/nginx/nginx.conf << EOF
+         user $USER;
+         EOF
+
+   #. Configure the web server.
+      
+      .. code-block:: bash
+
+         sudo tee -a /etc/nginx/conf.d/mirror-download-clearlinux-org.conf << EOF
+         server {
+           listen 80;
+	   listen [::]:80;
+	   server_name localhost;
+	   location / {
+	     root /var/www/mirror-download-clearlinux-org;
+             autoindex on;
+           }
+         }
+         EOF
+
+   #. Set nginx to start automatically on boot and then start it.
+
+      .. code-block:: bash
+
+         sudo systemctl enable nginx --now
+
+Test your mirror
+****************
+
+Now, try out your mirror by installing |CL| and adding bundles from it.  
+
+#. Download either the live desktop or live server installer ISO of the 
+   `same version` as the mirrored version, which is 32900 for this tutorial.  
+   Go to `https://cdn.download.clearlinux.org/releases/<release-version>/clear`.
+
+#. Burn the ISO to a thumb drive.  See :ref:`bootable-usb`.
+
+#. Boot it up and start the installer.  Depending on which version of 
+   |CL| you want to install, follow one of these guides:
+
+   * *Desktop* version: :ref:`bare-metal-install-desktop`
+   * *Server* version: :ref:`bare-metal-install-server`
+
+   In the :guilabel:`Advanced options` tab of the installer, select 
+   :guilabel:`Swupd Mirror`.  See Figure 1.
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/mirror-upstream-server/mirror-upstream-server-01.png
+      :scale: 100%
+      :alt: Advanced options > Swupd Mirror
+
+      `Figure 1: Advanced options > Swupd Mirror`
+
+   In the :guilabel:`Mirror URL` field, set it to the IP address of your mirror.    It should be something like this: http://<IP address of mirror server>/update.
+   And check the option :guilabel:`Allow installation over insecure connections (http://)`.  See Figure 2.
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/mirror-upstream-server/mirror-upstream-server-02.png
+      :scale: 100%
+      :alt: Advanced options > Mirror URL setting
+
+      `Figure 2: Advanced options > Mirror URL setting`
+
+#. After installation completes, boot up, and log in.
+
+#. Verify that the swupd client is pointing to your mirror.
+
+   .. code-block:: bash
+
+      sudo swupd info
+
+   Example output:
+
+   .. code-block:: console
+
+      Warning: This is an insecure connection
+      The --allow-insecure-http flag was used, be aware that this poses a threat the system
+
+      Distribution:      Clear Linux OS
+      Installed version: 32900
+      Version URL:       https://192.168.1.100/update
+      Content URL:       https://192.168.1.100/update
+
+#. Try listing available bundles on your mirror.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-list -a
+
+#. Add a bundle.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add <bundle-name>
+
+Keep your mirror in sync with upstream
+**************************************
+
+Be sure to keep your mirror in sync with upstream so that your clients have the 
+latest and greatest software and security updates.  You can do that continuing
+to clone the newer upstream releases.  
+
+.. _Manifest.MoM:
+   https://docs.01.org/clearlinux/latest/reference/manpages/swupd.1.html 

source/tutorials/nvidia.rst

@@ -41,21 +41,21 @@ Known issues
 Systems with multiple graphics devices, including integrated graphics (iGPU),
 are known to be problematic.
 
-.. note:: NVIDIA Optimus
+.. note:: NVIDIA Optimus\*
    
    Some systems come with a hybrid graphics configuration for a balanced power
    and performance profile. This configuration is commonly found on
-   laptops. `NVIDIA Optimus* technology
+   laptops. `NVIDIA Optimus technology
    <https://www.geforce.com/hardware/technology/optimus>`_, is designed to
    allow switching seamlessly between a NVIDIA device and another graphics
    devices sharing the same display.
    
-   Getting NVIDIA Optimus* on Linux working well with both graphics devices
+   Getting NVIDIA Optimus on Linux working well with both graphics devices
    adds an additional level of complexity with platform specific steps and may
-   require additional software. Installation for systems with NVIDIA Optimus*
+   require additional software. Installation for systems with NVIDIA Optimus
    with both graphics devices operating is not covered by the scope of this
    documentation. As a simple workaround, some systems can disable one of the
-   graphics devices or NVIDIA Optimus* in the system firmware.
+   graphics devices or NVIDIA Optimus in the system firmware.
 
 .. note::
    The :ref:`Long Term Support (LTS) kernel <compatible-kernels>` variant is
@@ -76,7 +76,7 @@ and sustainable on |CL|.
 
 #. Remove the kernel command-line parameter *intel_iommu=igfx_off* or disable
    input–output memory management unit (IOMMU), also known as Intel®
-   Virtualization Technology for Directed I/O (VT-d), in your system EFI/BIOS.
+   Virtualization Technology (Intel® VT) for Directed I/O (Intel® VT-d), in your system EFI/BIOS.
    See `this GitHub report
    <https://github.com/clearlinux/distribution/issues/1274>`_ and the NVIDIA
    documentation on `DMA issues
@@ -121,7 +121,7 @@ and sustainable on |CL|.
     
          sudo systemctl daemon-reload
        
-   c. Add the service as a depndency to the |CL| updates trigger causing the
+   c. Add the service as a dependency to the |CL| updates trigger causing the
       service to run after every update.
   
       .. code-block:: bash  
@@ -428,8 +428,8 @@ installing the NVIDIA drivers until an Xorg configuration has been defined for
 your monitors.
 
 
-"Oh no! Something has gone wrong" GNOME crash
-=============================================
+"Oh no! Something has gone wrong" GNOME\* crash
+===============================================
 
 
 .. figure:: /_figures/nvidia/nvidia-gnome-crash.png
@@ -446,6 +446,26 @@ Try disabling other graphics devices, including integrated graphics, in your
 system's EFI/BIOS. 
 
 
+Slow boot times
+===============
+
+
+There have been reports of slow boot times with NVIDIA drivers installed.
+Normally, when GDM detects NVIDIA proprietary drivers, it will disable Wayland
+and enable X11. Should GDM fail to disbale Wayland, it may results in slow boot
+times, according to `this GitHub reprot
+<https://github.com/clearlinux/distribution/issues/1780>`_.
+
+To manually disable Wayland:
+
+.. code-block:: bash
+
+      sudo tee /etc/gdm/custom.conf > /dev/null <<'EOF'
+      [daemon]
+      WaylandEnable=false
+      EOF
+
+
 Brightness control
 ==================
 
@@ -502,6 +522,8 @@ Additional resources
 
 * `NVIDIA Accelerated Linux Graphics Driver Installation Guides <https://download.nvidia.com/XFree86/Linux-x86_64/>`_
 
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
 .. _`nouveau project`:  https://nouveau.freedesktop.org/wiki/
 
 .. _`NVIDIA Driver Downloads website`: https://www.nvidia.com/download/index.aspx

source/tutorials/openfaas.rst

@@ -196,7 +196,7 @@ In this example, we'll imagine a FaaS solution where: a user provides a URL to
 a pictures, which invokes a function to do image classification and outputs
 the result.
 
-We will use the `OpenVINO Deep Learning Deployment Toolkit (DLDT)
+We will use the `OpenVINO™ toolkit - Deep Learning Deployment Toolkit (DLDT)
 <https://software.intel.com/en-us/openvino-toolkit/>`_ to do the image
 inference. As inference development is not the focus of this example, we will
 just use the built-in sample "`classification_sample_async
@@ -208,7 +208,7 @@ We'll use the *python3-clearlinux* template as a base and customize it by:
 
 * Adding additional |CL| bundles (*bundles.txt*)
 * Adding additional required python packages (*requirements.txt*)
-* Adding a script to download and convert dldt models (*helper_script.sh*)
+* Adding a script to download and convert DLDT models (*helper_script.sh*)
 * Finally, we'll develop the python function to be run (*handler.py*)
 
 More ways to customize the |CL| based OpenFaaS templates can be found in the
@@ -385,3 +385,4 @@ More ways to customize the |CL| based OpenFaaS templates can be found in the
 
       Figure 3: OpenFaaS web interface invoke function
       
+*Intel, OpenVINO, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*

source/tutorials/openvino.rst

@@ -1,290 +0,0 @@
- .. _openvino:
-
-OpenVINO™ for Deep Learning
-###########################
-
-This tutorial shows how to install OpenVINO™ on |CL-ATTR|, run an
-OpenVINO sample application for image classification, and run a benchmark_app
-for estimating inference performance---using Squeezenet 1.1.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Prerequisites
-*************
-
-* |CL| installed on the host OS
-
-Install OpenVINO
-****************
-
-OpenVINO in |CL| offers pre-built OpenVINO sample applications with which
-developers can try inferencing immediately.
-
-#. In |CL| OpenVINO is included in the :command:`computer-vision-basic`
-   bundle. To install OpenVINO, enter:
-
-   .. code-block:: bash
-
-      sudo swupd bundle-add computer-vision-basic
-
-#. OpenVINO Inference Engine libraries are located in :file:`/usr/lib64/`
-   To view one added package, enter:
-
-   .. code-block:: bash
-
-      ls /usr/lib64/libinference_engine.so
-
-   If bundle installation is successful, the output shows:
-
-   .. code-block:: console
-
-      /usr/lib64/libinference_engine.so
-
-#. To view the OpenVINO Model Optimizer, enter:
-
-   .. code-block:: console
-
-      ls /usr/share/openvino/model-optimizer
-
-#. To view the OpenVINO sample application Executables, enter:
-
-   .. code-block:: bash
-
-      ls /usr/bin/benchmark_app \
-      /usr/bin/classification_sample_async \
-      /usr/bin/hello_classification \
-      /usr/bin/hello_nv12_input_classification \
-      /usr/bin/hello_query_device \
-      /usr/bin/hello_reshape_ssd \
-      /usr/bin/object_detection_sample_ssd \
-      /usr/bin/speech_sample \
-      /usr/bin/style_transfer_sample \
-
-   .. note::
-
-      If bundle installation is successful, the above files should appear.
-
-#. To view the pre-built OpenVINO sample application source code, enter:
-
-   .. code-block:: bash
-
-      ls /usr/share/doc/inference_engine/samples
-
-In the next section, you learn how to use an OpenVINO sample application.
-
-Run OpenVINO sample application
-*******************************
-
-After installing OpenVINO on |CL|, you need a model against which to test.
-In this example, we use the public squeezenet 1.1 model for image
-classification. Test results vary based on the system used.
-
-Use model to test
-=================
-
-#. If you don’t have any model, you can download an
-   **intel_model** or a public model using OpenVINO Model Downloader.
-
-   - Check the list of public models you can download from
-     :file:`/usr/share/open_model_zoo/models/public`
-
-   - Check the list of Intel® models you can download from
-     :file:`/usr/share/open_model_zoo/intel_models`
-
-#. View the location of OpenVINO Model Downloader:
-
-   .. code-block:: console
-
-      cd /usr/share/open_model_zoo/tools/downloader
-
-#. In general, download models with the following command:
-
-   .. code-block:: bash
-
-      python3 downloader.py --name <model_name> -o <downloading_path>
-
-   .. note::
-
-      * Where :file:`<model_name>` is the one you chose from previous step
-
-      * Where :file:`<downloading_path>` is your project directory
-
-#. For this example, enter:
-
-   .. code-block:: bash
-
-      python3 downloader.py --name squeezenet1.1 -o $HOME/.
-
-#. After running this command, the model appears as downloading at your
-   :file:`$HOME/classification/squeezenet/1.1/caffe` as follows:
-
-   .. code-block:: console
-
-      ###############|| Downloading topologies ||###############
-
-      ========= Downloading /$HOME/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel
-      ... 100%, 4834 KB, 2839 KB/s, 1 seconds passed
-
-      ...
-
-Convert model to IR format
-==========================
-
-#. As necessary, follow the instruction on :ref:`convert-dl-models`
-   to convert deep learning models.
-
-#. Navigate to the model:
-
-   .. code-block:: bash
-
-	  cd $HOME/classification/squeezenet/1.1/caffe
-
-#. Enter the command:
-
-   .. code-block:: bash
-
-      python3 /usr/share/openvino/model-optimizer/mo.py --input_model squeezenet1.1.caffemodel
-
-
-   The output will show these files being generated:
-
-   .. code-block:: console
-
-      squeezenet1.1.xml
-
-      squeezenet1.1.bin
-
-#. Finally, enter :command:`ls` to view the newly added model and files.
-
-Run image classification
-========================
-
-This sample application demonstrates how to run the Image Classification in asynchronous mode on supported devices. In this example, we use the image of a specific type of automobile to test the inference engine. Squeezenet 1.1 is designed to perform image classification and has been trained on the `ImageNet`_ database.
-
-#. We provide an image of an automobile, shown in Figure 1. For ease of use,
-   save this image into the :file:`classification` model directory.
-
-   .. figure:: ../_figures/openvino/automobile.png
-      :height: 375 px
-      :width: 500 px
-      :scale: 100 %
-      :alt: Photo by Goh Rhy Yan on Unsplash
-
-      Figure 1: Photo by Goh Rhy Yan on Unsplash
-
-#. To execute the sample application enter the command:
-
-   .. code-block:: bash
-
-      classification_sample_async -i <path_to_image> -m <path_to_model_ir> -d <device>
-
-   .. note::
-
-      * Where :file:`<path_to_image>` is the image that you selected
-
-      * Where :file:`<path_to_model_ir>` is the path to the IR model file
-
-      * Where :file:`<device>` is your choice of CPU, GPU, etc.
-
-#. In this case, we replace the :file:`<path_to_image>` with the previously
-   saved image for CPU inferencing.
-
-   .. code-block:: bash
-
-      classification_sample_async -i ./automobile.png -m squeezenet1.1.xml
-
-   .. note::
-
-      If you do not specify the :file:`device`, the CPU is used by default.
-
-#. The results show the highest probability is 67% for a sports car.
-
-   .. code-block:: bash
-
-      classid probability
-      ------- -----------
-      817     0.6717085
-      511     0.1611409
-
-   +-----------------------+-----------------------------------+
-   |:command:`classid` 817 | :command:`sports car, sport car`  |
-   +-----------------------+-----------------------------------+
-   |:command:`classid` 511 |:command:`convertible`             |
-   +-----------------------+-----------------------------------+
-
-   .. note:
-
-      Label definitions are provided by `ImageNet`_.
-
-#. Next, add :command:`-d GPU` to the end of the above command for GPU
-   inferencing.
-
-   .. code-block:: bash
-
-      classification_sample_async -i ./automobile.png -m squeezenet1.1.xml -d GPU
-
-Run benchmark_app
-*****************
-
-This sample application demonstrates how to use benchmark application to
-estimate deep learning inference **performance** on supported devices.
-We use the same image of an automobile, Figure 1, from the previous section.
-
-#. To execute this sample application, enter:
-
-   .. code-block:: bash
-
-      benchmark_app -i <path_to_image> -m <path_to_model> -d <device>
-
-   .. note::
-
-      * Where :file:`<path_to_image>` is the image that you selected
-
-      * Where :file:`<path_to_model_ir>` is the path to the IR model file
-
-      * Where :file:`<device>` is local your choice of CPU, GPU, etc.
-
-#. Change directory:
-
-   .. code-block:: bash
-
-      cd $HOME/classification/squeezenet/1.1/caffe
-
-#. Enter the following command for CPU inferencing.
-
-   .. code-block:: bash
-
-      benchmark_app -i ./automobile.png -m squeezenet1.1.xml
-
-#. For the CPU, the results show a :guilabel:`Throughput` of 243.202 FPS.
-
-   .. code-block:: console
-      :linenos:
-      :emphasize-lines: 4
-
-      Count:      1464 iterations
-      Duration:   60196.8 ms
-      Latency:    164.104 ms
-      Throughput: 243.202 FPS
-
-#. Next, add :command:`-d GPU` to the end of the same command for GPU
-   inferencing.
-
-   .. code-block:: bash
-
-      benchmark_app -i ./automobile.png -m squeezenet1.1.xml -d GPU
-
-#. For the GPU, the results show a :guilabel:`Throughput` of 372.677 FPS.
-
-   .. code-block:: console
-      :linenos:
-      :emphasize-lines: 4
-
-      Count:      2240 iterations
-      Duration:   60105.7 ms
-      Latency:    107.554 ms
-      Throughput: 372.677 FPS
-
-.. _ImageNet: http://image-net.org/

source/tutorials/php.rst

@@ -61,8 +61,8 @@ Prerequisites
 .. note::
 
    PHP does not require a web server for operation. If you need a web
-   server, refer to :ref:`web-server-install` for instructions on setting
-   up a :abbr:`LAMP (Linux, Apache\*, MySQL, PHP)` server, or use
+   server, refer to :ref:`lamp-server-install` for instructions on setting
+   up a :abbr:`LAMP (Linux, Apache\*, MySQL\*, PHP)` server, or use
    :command:`swupd` to install :file:`nginx` or similar.
 
 

source/tutorials/proxy.rst

@@ -1,118 +0,0 @@
-.. _proxy:
-
-Proxy configuration
-###################
-
-This tutorial shows you how to configure your system for use behind an
-outbound proxy to access the Internet.
-
-|CL-ATTR| applications already benefit from the :ref:`autoproxy`
-feature.
-
-.. contents::
-    :local:
-    :depth: 1
-
-Prerequisites
-*************
-
-* You have installed |CL| on your host system.
-
-  For detailed instructions on installing |CL| on a bare metal system, visit
-  the :ref:`bare metal installation guide <bare-metal-install-desktop>`.
-
-Shells and programs in a desktop session
-****************************************
-
-Terminal
-========
-
-Add the following to your ~/.bashrc:
-
-.. code-block:: bash
-
-   export http_proxy=http://your.http-proxy.url:port
-   export https_proxy=http://your.https-proxy.url:port
-   export ftp_proxy=http://your.ftp-proxy.url:port
-   export socks_proxy=http://your.socks-proxy.url:port
-   export no_proxy=".your-company-domain.com,localhost"
-   export HTTP_PROXY=$http_proxy
-   export HTTPS_PROXY=$https_proxy
-   export FTP_PROXY=$ftp_proxy
-   export SOCKS_PROXY=$socks_proxy
-   export NO_PROXY=$no_proxy
-
-wget
-====
-
-Run the following command to enable downloading from websites from the terminal:
-
-.. code-block:: bash
-
-   echo >> ~/.wgetrc <<EOF
-   http_proxy = your.http-proxy.url:port
-   https_proxy = your.https-proxy.url:port
-   ftp_proxy = your.http-proxy.url:port
-   no_proxy = your-company-domain.com, localhost
-   EOF
-
-System service (Docker\*)
-=========================
-
-For Docker (and other services that use systemd), follow these steps to
-configure and check proxy settings.
-
-.. note::
-
-   Use :command:`sudo`, as you will need elevated permissions.
-
-#. Create the :file:`/etc/systemd/system/docker.service.d` directory to host
-   configuration information for the Docker service.
-
-#. Create the :file:`/etc/systemd/system/docker.service.d/http-proxy.conf`
-   and add the following lines:
-
-   .. code-block:: bash
-
-      [Service]
-      Environment="HTTP_PROXY=http://your.http-proxy.url:port/"
-      Environment="HTTPS_PROXY=http://your.https-proxy.url:port/"
-
-#. Load the changes and restart the service.
-
-   .. code-block:: bash
-
-      sudo systemctl daemon-reload
-      sudo systemctl restart docker
-
-#. Verify that changes have loaded.
-
-   .. code-block:: bash
-
-      systemctl show --property=Environment docker
-
-   .. code-block:: console
-
-      Environment=HTTP_PROXY=http://your.http-proxy.url:port/ HTTPS_PROXY=http://your.https-proxy.url:port/
-
-.. note::
-
-   This process enables the ability to successfully run ``docker pull``.
-   Containers themselves must be configured independently.
-
-git over ssh
-************
-
-Add the following to your :file:`~/.ssh/config`:
-
-.. code-block:: bash
-
-   host github.com
-        port 22
-        user git
-        ProxyCommand connect-proxy -S your.ssh-proxy.url:port %h %p
-
-.. note::
-
-   Though netcat is included with |CL|, it is not the BSD version,
-   which is the one usually used to enable git over ssh.

source/tutorials/redis.rst

@@ -5,7 +5,7 @@ Redis\*
 
 In this tutorial, you'll install :abbr:`Redis (REmote DIctionary Server)`
 and launch a `Redis-server` on |CL-ATTR|. We invite you to pull our
-`Clear Linux Redis instance`_ on dockerhub\* for application or
+`Clear Linux Redis instance`_ on Docker Hub\* for application or
 infrastructure development.
 
 .. contents::
@@ -15,57 +15,49 @@ infrastructure development.
 Description
 ***********
 
-Redis is an in-memory key:value store designed for quick lookups, accessible over a network. While the `redis data structure store`_ can serve
+Redis is an in-memory key:value store designed for quick lookups, accessible 
+over a network. While the `redis data structure store`_ can serve
 as a NoSQL database for a web application, it's also easy to integrate into an
 existing stack. For example, you could use the Redis caching layer for
 real-time responses on a leaderboard in a gaming app. Redis offers many client
 libraries with language-specific bindings for Python\*, Perl\*, Ruby, and more.
 
-Prerequisites
-*************
-
-* Install the :command:`redis-native` bundle in |CL|
-* Install the :command:`containers-basic` bundle in |CL| (only required in
-  Example 2)
-
 Install the Redis bundle
 ************************
 
-In |CL|, find Redis in the :command:`redis-native` bundle.
+#. Log in as a user with administrative privilege.
 
-#. Open a terminal and login as an administrative user.
+#. Open a terminal.
 
-#. Add :command:`redis-native`.
+#. Update your |CL| to the latest version.
+
+   .. code-block:: bash
+
+      sudo swupd update
+
+#. Install the `redis-native` bundle.
 
    .. code-block:: bash
 
       sudo swupd bundle-add redis-native
 
-   .. note::
-
-      If the bundle already exists, no action is required.
-
 Start the Redis-server
 **********************
 
 A :command:`systemd` service unit is available to control the Redis-server.
 By default, Redis runs on port 6379.
 
-#. Start the service.
+#. Start the service and set it to start automatically on boot.
 
    .. code-block:: bash
 
-      systemctl start redis
-
-   .. note::
-
-      To stop Redis, run :command:`systemctl stop redis`.
+      sudo systemctl enable --now redis
 
 #. Confirm the service is running.
 
    .. code-block:: bash
 
-      systemctl status redis
+      sudo systemctl status redis
 
 #. Verify that the Redis-server sends a reply.
 
@@ -73,16 +65,21 @@ By default, Redis runs on port 6379.
 
       redis-cli ping
 
-   .. note::
+   Expected output: 
 
-      Expected output: `PONG`.
+   .. code-block:: console
 
-#. Optional: If you wish to apply the advanced configuration, copy the
-   `redis.conf` into /etc/ directory.
+      PONG
+
+.. note::
+
+   If you wish to customize settings for Redis, copy the
+   default :file:`/usr/share/defaults/etc/redis.conf` file into the 
+   /etc/ directory, make changes as needed, and restart the service.
 
    .. code-block:: bash
 
-      sudo cp /usr/share/defaults/etc/redis.conf /etc/
+      sudo cp -v /usr/share/defaults/etc/redis.conf /etc/
 
 The Redis-server is now ready to use on |CL|. Try some of the examples shown
 below.
@@ -97,7 +94,7 @@ programming languages. These exercises are inspired by `try redis io`_.
 
 After your Redis-server is running, try some basic commands.
 
-#. Enter the `redis-cli`. It provides syntax suggestions as you type.
+#. Start `redis-cli`. It provides syntax suggestions as you type.
 
    .. code-block:: bash
 
@@ -106,66 +103,67 @@ After your Redis-server is running, try some basic commands.
 #. :command:`SET` a key to hold a string value. In the set, create connections
    and increment.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       SET server:name "clearlinux"
 
-   .. code-block:: bash
+   .. code-block:: none
 
       MGET server:name
 
    .. note::
+
       If the key does not exist or hold a key value, `nil` is returned.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       SET connections 100
 
-   .. code-block:: bash
+   .. code-block:: none
 
       INCR connections
 
-   .. code-block:: bash
+   .. code-block:: none
 
       INCR connections
 
-   .. code-block:: bash
+   .. code-block:: none
 
       DEL connections
 
 #. Create a `friends` list and insert new values at the end of the list.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       RPUSH friends "Deb"
 
-   .. code-block:: bash
+   .. code-block:: none
 
       RPUSH friends "David"
 
-   .. code-block:: bash
+   .. code-block:: none
 
       RPUSH friends "Mary"
 
 #. Modify the `friends` list, using a common slice method with a 0-index.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       LRANGE friends 0 1
 
-   .. code-block:: bash
+   .. code-block:: none
 
       LLEN friends
 
-   .. code-block:: bash
+   .. code-block:: none
 
       LPOP friends
 
-   .. code-block:: bash
+   .. code-block:: none
 
       RPOP friends
 
-   .. code-block:: bash
+   .. code-block:: none
 
       LLEN friends
 
@@ -174,19 +172,19 @@ After your Redis-server is running, try some basic commands.
 
    Enter many user key:values with `HMSET`. Then try `HGET` and `HGETALL`.
 
-   .. code-block:: bash
+   .. code-block:: none
 
       HMSET user:1000 name "Robert Noyce" password "SuperEngi9eer" email "robert.noyce@intel.com"
 
-   .. code-block:: bash
+   .. code-block:: none
 
       HGET user:1000 name
 
-   .. code-block:: bash
+   .. code-block:: none
 
       HGET user:1000 email
 
-   .. code-block:: bash
+   .. code-block:: none
 
       HGETALL user:1000
 

source/tutorials/smb-server.rst

@@ -0,0 +1,150 @@
+.. _smb-server:
+
+Samba Server
+############
+
+This tutorial describes how to enable simple file sharing on a system
+running |CL-ATTR| and how to access the share from clients on other operating 
+systems. 
+
+.. contents::
+   :local:
+   :depth: 1
+
+Prerequisites
+*************
+
+* |CL| is installed on your host system.
+
+  For detailed instructions, follow one of these guides:
+
+  * *Desktop* version: :ref:`bare-metal-install-desktop`
+  * *Server* version: :ref:`bare-metal-install-server`
+
+* Update your |CL| system to the latest version. 
+
+  .. code-block:: bash
+
+     sudo swupd update
+
+Set up file sharing
+*******************
+
+#. Add the :command:`samba` bundle, which includes the Samba binaries.
+
+   .. code-block:: bash
+
+      sudo swupd bundle-add samba
+
+#. Create the :file:`/etc/samba` directory.
+
+   .. code-block:: bash
+
+      sudo mkdir /etc/samba
+
+#. Create a configuration file named :file:`/etc/samba/smb.conf`. In this
+   simple example, :envvar:`[SharedDrive]` enables a folder share located in 
+   :file:`/home/clear/mysharedrive` granting users `samba-user-1` and 
+   `samba-user-2` access. 
+
+   If :envvar:`valid users` is not specified, then anyone with a user account
+   on the machine and with their Samba password already set can access the
+   folder. However, the account is only able to access files and folders for
+   which they have appropriate permissions.
+
+   For more advanced sharing, refer to the `Samba guide`_.
+
+   .. code-block:: console
+
+      [Global]
+      map to guest = Bad User
+
+      [SharedDrive]
+      path=/home/clear/myshareddrive
+      read only = no
+      guest ok = no
+      browsable = yes
+      valid users = samba-user-1 samba-user-2
+
+#. Start the Samba service and set it to start automatically on boot.
+
+   .. code-block:: bash
+
+      sudo systemctl enable --now smb
+
+#. Verify the service started properly.
+
+   .. code-block:: bash
+
+      sudo systemctl status smb
+
+#. Use :command:`smbpasswd` to add the initial password for the user
+   account to access the share. Be aware that Samba maintains its own list of
+   passwords for user accounts. The Samba password list can be different from
+   the password used to log in.
+
+   For example:
+
+   .. code-block:: bash
+
+      sudo smbpasswd -a samba-user-1
+
+Access the shared drive
+***********************
+
+Depending on your operating system, connect to the shared drive using one 
+of the methods belows:
+
+* On |CL|:
+
+  a. Add the `samba` bundle.
+
+     .. code-block:: bash
+
+        sudo swupd bundle-add samba
+  
+  #. List available shares.
+
+     .. code-block:: bash
+
+        smbclient -L //<ip-address-of-smb-server>
+
+  #. Connect to a shared drive.
+     
+     .. code-block:: bash
+
+        smbclient //<ip-address-of-smb-server>/<shared-drive> -U <user>
+
+* On Windows:
+
+  a. Open `File Explorer`.
+
+  #. Enter :command:`\\<ip-address-of-samba-server>\<shared-drive>` in the
+     URL field. See Figure 1.
+
+     .. rst-class:: dropshadow
+
+     .. figure:: ../_figures/samba/smb-server-01.png
+        :scale: 100%
+        :alt: Windows > File Explorer > Connect to Samba share
+
+        `Figure 1: Windows > File Explorer > Connect to Samba share`
+
+* On macOS:
+
+  a. Open the `Finder`.
+
+  #. Press :kbd:`Command` + :kbd:`K` to open the dialog box for 
+     connecting to a server.
+
+  #. Enter :command:`smb://<ip-address-of-samba-server>/<shared-drive>`.
+     in the URL field. See Figure 2.
+
+     .. figure:: ../_figures/samba/smb-server-02.png
+        :scale: 100%
+        :alt: macOS > Finder > Connect to Samba share
+
+        `Figure 2: macOS > Finder > Connect to Samba share`
+     
+.. _Samba guide: 
+   https://www.samba.org/samba/docs/using_samba/ch00.html

source/tutorials/smb.rst

@@ -1,139 +0,0 @@
-.. _clear-samba-share-to-Windows:
-
-Samba\* as a host
-#################
-
-This tutorial describes how to enable simple file sharing from a system
-running |CL-ATTR| to a Windows\* computer using Samba. For more advanced
-sharing, refer to the `Samba guide`_.
-
-.. contents::
-   :local:
-   :depth: 1
-
-Prerequisites
-*************
-
-*	|CL| is installed on your host system.
-
-	For detailed instructions, follow the steps in
-	:ref:`bare-metal-install-desktop`.
-
-*	Before you install any new packages, update |CL| with the following
-	command:
-
-	.. code-block:: bash
-
-	   sudo swupd update
-
-
-Set up file sharing
-*******************
-
-#.	Log in and get root privileges.
-
-	.. code-block:: bash
-
-		sudo -s
-
-#.	Add the :command:`storage-utils` bundle, which includes the Samba binaries.
-
-	.. code-block:: bash
-
-		swupd bundle-add storage-utils
-
-	.. note::
-
-		The :command:`os-clr-on-clr` bundle also includes the Samba binaries.
-
-#.	Create a configuration file named :file:`/etc/samba/smb.conf`. In this
-	example, :envvar:`[Downloads]` enables a folder share with a specific user.
-	:envvar:`[Documents]` enables a folder share with any user. The example
-	assumes that the user account :envvar:`clearlinuxuser` already exists.
-
-	If :envvar:`valid users` is not specified, then anyone with a user account
-	on the machine and with their Samba password already set can access the
-	folder. However, the account is only able to access files and folders for
-	which they have appropriate permissions.
-
-	Use either :command:`chown` to change the owner of the file or use
-	:command:`chmod` to change the permissions to allow other users to access
-	the file.
-
-	.. code-block:: console
-
-		[Global]
-		map to guest = bad user
-
-		[Downloads]
-		path=/home/clearlinuxuser/Downloads
-		read only = no
-		guest ok = no
-		valid users = clearlinuxuser
-
-		[Documents]
-		path=/home/clearlinuxuser/Documents
-		read only = no
-		browsable = yes
-		guest ok = yes
-
-#.	Enable the Samba daemon to start every time.
-
-	.. code-block:: bash
-
-		systemctl enable smb
-		systemctl start smb
-
-#.	Use :command:`smbpasswd` to add the initial password for the user
-	account to access the share. Be aware that Samba maintains its own list of
-	passwords for user accounts. The Samba password list can be different from
-	the password used to log in.
-
-	.. code-block:: bash
-
-		smbpasswd -a clearlinuxuser
-
-The setup is now complete and a Windows machine on the same network can access
-the shares. Windows uses the format
-:file:`\\\\[server IP or hostname]\\folder` to access shares. Access the shares
-directly with Windows Explorer or by mapping a network drive.
-
-Use the IP address of the |CL| machine for an easy access method. If the
-|CL| machine is behind an Active Directory domain controller or a DNS server,
-use the hostname of the |CL| machine. For other ways to access shares using a
-hostname instead of an IP address, see `Chapter 7 of the Samba guide`_.
-
-
-Map |CL| drive in Windows
-*************************
-
-#.	Open Windows Explorer and click on the left sidebar on :guilabel:`This PC`
-	to change the options available at the top.
-
-#.	Click the :guilabel:`Map Network Drive` icon and enter the path in the
-	format: :file:`\\\\[server IP or hostname]\\[shared folder]`
-
-#.	Check the box :guilabel:`Connect using different credentials`. Enter
-	the Samba user `clearlinuxuser` and the password created with
-	:command:`smbpasswd`. See Figure 1 for details.
-
-	.. figure:: /_figures/samba/smb-1.png
-		:scale: 70%
-		:alt: Map a network drive in Windows Explorer
-
-		Figure 1: Map a network drive in Windows Explorer.
-
-When complete, Windows Explorer displays the share drive as shown in Figure 2.
-
-.. figure:: /_figures/samba/smb-2.png
-	:scale: 70%
-	:alt: View a share drive in Windows Explorer
-
-	Figure 2: View a share drive in Windows Explorer.
-
-
-
-
-
-.. _Samba guide: https://www.samba.org/samba/docs/using_samba/ch00.html
-.. _Chapter 7 of the Samba guide: https://www.samba.org/samba/docs/using_samba/ch07.html

source/tutorials/wordpress.rst

@@ -1,27 +1,214 @@
-.. _wordpress:
+.. _wp-install:
 
-WordPress\*
-###########
+WordPress\* Server
+##################
 
-This tutorial describes how to setup a web server to host a WordPress\* site on a
-|CL-ATTR| system. The tutorial is split into two sections:
+This tutorial shows you how to install the WordPress\* components on your |CL|
+:abbr:`LAMP (Linux, Apache\*, MySQL\*, PHP)` server. At the end of
+:ref:`lamp-server-install`, you created a WordPress-ready database using
+phpMyAdmin\* and MariaDB\*. Now that the LAMP server is up and running, you
+can add the WordPress components needed to host a WordPress website on your system.
 
-#. :ref:`web-server-install`
+Before you begin
+****************
 
-#. :ref:`wp-install`
+This tutorial assumes that you have successfully completed
+:ref:`bare-metal-install-desktop` and you have :ref:`lamp-server-install`.
 
-If you already have a |CL| based :abbr:`LAMP (Linux, Apache, MySQL, PHP)` web server,
-you can skip ahead to the second section of this tutorial.
+Create a WordPress server
+*************************
+
+WordPress can be installed in a variety of ways. These instructions are
+written for users who have followed our instructions for installing phpMyAdmin
+when they :ref:`set up a LAMP web server <lamp-server-install>`. Note that
+all steps in this tutorial have been tested using a NUC6i5SYH Intel® NUC.
+Visit the `NUC6i5SYH product page`_ for detailed information.
+
+Numerous online articles are available to help you name your website and
+acquire the necessary certificates. Those tasks are beyond the scope of this tutorial.
+
+You can take several actions to harden your website from attacks. The security
+of your website and the data it contains are complex and ever-evolving tasks.
+Prioritize security if you plan to expose your website to the outside world.
+This tutorial does not address security measures that you can take to harden
+your site but we strongly encourage you to take action.
 
 .. note::
 
-   This tutorial is for development and testing purposes only. Additional steps
-   are required to secure production systems. Those steps are beyond the scope
-   of this tutorial.
+   Throughout this tutorial, we reference your website name as <your_website>.
 
-.. toctree::
-   :maxdepth: 1
-   :hidden:
 
-   wordpress/web-server-install.rst
-   wordpress/wp-install.rst
+Download WordPress and manage directories
+=========================================
+
+For this tutorial, you will create a WordPress blog that can be accessed at:
+\http://<your_website>/blog.
+
+To accomplish this setup, you must add WordPress components to the :file:`/var/www/html/blog`
+directory.
+
+Follow these steps:
+
+
+#. Navigate to the top level of the website’s root directory:
+
+   .. code-block:: bash
+
+      cd /var/www/html
+
+#. Download the latest version of WordPress:
+
+   .. code-block:: bash
+
+      sudo curl -O https://wordpress.org/latest.tar.gz
+
+#. Extract all files and directories from the downloaded file:
+
+   .. code-block:: bash
+
+      sudo tar –xzvf latest.tar.gz
+
+#. Rename the top-level WordPress directory to “blog”:
+
+   .. code-block:: bash
+
+      sudo mv wordpress blog
+
+#. Remove the downloaded tar file:
+
+   .. code-block:: bash
+
+      sudo rm latest.tar.gz
+
+Set up WordPress with web-based GUI
+===================================
+
+Recall that you created a database and user when you installed phpMyAdmin when you
+set up a |CL| based :ref:`web server <lamp-server-install>`. Next, you must
+connect WordPress to the database and install WordPress.
+
+To continue with the setup, go to: \http://<your_website>/blog/wp-admin/install.php.
+The WordPress language option screen appears, as shown in figure 1.
+
+#. Select :guilabel:`English` and click :guilabel:`Continue`.
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/wp-install-1.png
+      :alt: WordPress language selection
+      :width:     600
+
+      `Figure 1: WordPress language selection screen.`
+
+
+   The WordPress installation continues until the Welcome screen appears, as shown in
+   figure 2:
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/wp-install-2.png
+      :alt: WordPress welcome screen
+      :width:     600
+
+      `Figure 2: WordPress Welcome screen.`
+
+#. Click :guilabel:`Let’s go!`.
+
+#. Enter database connection specifics in the screen that appears, as shown in figure 3
+   below.
+
+   * Database name:       WordPress
+   * Database username:   wordpressuser
+   * Database password:   wp-example  (asterisks will not appear in the text box)
+   * Database host:  localhost
+   * Table prefix:   wp\_
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/wp-install-3.png
+      :alt: Database connection details
+      :width:     600
+
+      `Figure 3: Information necessary for WordPress to connect to the database.`
+
+#. Click :guilabel:`Submit` to complete the setup.
+
+   Figure 4 shows the confirmation screen that verifies a successful setup. WordPress
+   is connected to the MariaDB database.
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/wp-install-4.png
+      :alt: Successful database connection.
+      :width:     600
+
+      `Figure 4: Successful WordPress connection.`
+
+#. Click :guilabel:`Run the install`.
+   The installer runs until WordPress is fully installed on your system.
+
+Complete successful login
+=========================
+
+Once the installation is complete, you can name your blog and create a WordPress username
+and password. See figure 5.
+
+.. rst-class:: dropshadow
+
+.. figure:: ../_figures/wordpress/wp-install-5.png
+   :alt: WordPress user creation
+   :width:     600
+
+   `Figure 5: WordPress site information screen.`
+
+
+#. Enter all required information.
+#. Click :guilabel:`Install WordPress`.
+#. Verify that the initial login screen appears once the installation is complete. See figure 6:
+
+   .. rst-class:: dropshadow
+
+   .. figure:: ../_figures/wordpress/wp-install-6.png
+      :alt: WordPress login
+      :width:     600
+
+      `Figure 6: The WordPress login screen.`
+
+#. Enter your WordPress username and password.
+#. Check :guilabel:`Remember me` to save your credentials.
+#. Click :guilabel:`Log in`.
+
+Figure 7 shows the WordPress dashboard after a successful login:
+
+.. rst-class:: dropshadow
+
+.. figure:: ../_figures/wordpress/wp-install-7.png
+   :alt: WordPress Dashboard
+   :width:     600
+
+   `Figure 7: The WordPress dashboard.`
+
+You are ready to go!
+
+To check out your blog as it is seen by the outside world, enter:
+\http://<your_website>/blog on your browser. Figure 8 shows the result:
+
+.. figure:: ../_figures/wordpress/wp-install-8.png
+   :alt: WordPress blog
+   :width:     600
+
+   `Figure 8: Your WordPress blog.`
+
+**Congratulations, your WordPress blog is up and running!**
+
+You have successfully installed WordPress on a host system.
+
+Add new entries to your blog and share them with the world using |CL|!
+
+*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.*
+
+.. _`step-by-step guide`:
+   https://codex.wordpress.org/Installing_WordPress#Famous_5-Minute_Install
+
+.. _`NUC6i5SYH product page`:
+   http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html

source/tutorials/wordpress/wp-install.rst

@@ -1,198 +0,0 @@
-.. _wp-install:
-
-Set up WordPress\* on a LAMP web server
-#######################################
-
-This tutorial shows you how to install the WordPress\* components on your |CL|
-:abbr:`LAMP (Linux, Apache, MySQL, PHP)` server. At the end of
-:ref:`web-server-install`, you created a WordPress-ready database using
-phpMyAdmin\* and MariaDB\*. Now that the LAMP server is up and running, you
-can add the WordPress components needed to host a WordPress website on your system.
-
-Before you begin
-****************
-
-This tutorial assumes that you have successfully completed
-:ref:`bare-metal-install-desktop` and you have :ref:`web-server-install`.
-
-Create a WordPress server
-*************************
-
-WordPress can be installed in a variety of ways. These instructions are
-written for users who have followed our instructions for installing phpMyAdmin
-when they :ref:`set up a LAMP web server <web-server-install>`. Note that
-all steps in this tutorial have been tested using a NUC6i5SYH Intel® NUC.
-Visit the `NUC6i5SYH product page`_ for detailed information.
-
-Numerous online articles are available to help you name your website and
-acquire the necessary certificates. Those tasks are beyond the scope of this tutorial.
-
-You can take several actions to harden your website from attacks. The security
-of your website and the data it contains are complex and ever-evolving tasks.
-Prioritize security if you plan to expose your website to the outside world.
-This tutorial does not address security measures that you can take to harden
-your site but we strongly encourage you to take action.
-
-.. note::
-
-   Throughout this tutorial, we reference your website name as <your_website>.
-
-
-Download WordPress and manage directories
-=========================================
-
-For this tutorial, you will create a WordPress blog that can be accessed at:
-\http://<your_website>/blog.
-
-To accomplish this setup, you must add WordPress components to the :file:`/var/www/html/blog`
-directory.
-
-Follow these steps:
-
-
-#. Navigate to the top level of the website’s root directory:
-
-   .. code-block:: bash
-
-      cd /var/www/html
-
-#. Download the latest version of WordPress:
-
-   .. code-block:: bash
-
-      sudo curl -O https://wordpress.org/latest.tar.gz
-
-#. Extract all files and directories from the downloaded file:
-
-   .. code-block:: bash
-
-      sudo tar –xzvf latest.tar.gz
-
-#. Rename the top-level WordPress directory to “blog”:
-
-   .. code-block:: bash
-
-      sudo mv wordpress blog
-
-#. Remove the downloaded tar file:
-
-   .. code-block:: bash
-
-      sudo rm latest.tar.gz
-
-Set up WordPress with web-based GUI
-===================================
-
-Recall that you created a database and user when you installed phpMyAdmin when you
-set up a |CL| based :ref:`web server <web-server-install>`. Next, you must
-connect WordPress to the database and install WordPress.
-
-To continue with the setup, go to: \http://<your_website>/blog/wp-admin/install.php.
-The WordPress language option screen appears, as shown in figure 1.
-
-#. Select :guilabel:`English` and click :guilabel:`Continue`.
-
-   .. figure:: /_figures/wordpress/wp-install-1.png
-      :alt: WordPress language selection
-      :width:     600
-
-      `Figure 1: WordPress language selection screen.`
-
-
-   The WordPress installation continues until the Welcome screen appears, as shown in
-   figure 2:
-
-   .. figure:: /_figures/wordpress/wp-install-2.png
-      :alt: WordPress welcome screen
-      :width:     600
-
-      `Figure 2: WordPress Welcome screen.`
-
-#. Click :guilabel:`Let’s go!`.
-
-#. Enter database connection specifics in the screen that appears, as shown in figure 3
-   below.
-
-   * Database name:       WordPress
-   * Database username:   wordpressuser
-   * Database password:   wp-example  (asterisks will not appear in the text box)
-   * Database host:  localhost
-   * Table prefix:   wp\_
-
-   .. figure:: /_figures/wordpress/wp-install-3.png
-      :alt: Database connection details
-      :width:     600
-
-      `Figure 3: Information necessary for WordPress to connect to the database.`
-
-#. Click :guilabel:`Submit` to complete the setup.
-
-   Figure 4 shows the confirmation screen that verifies a successful setup. WordPress
-   is connected to the MariaDB database.
-
-   .. figure:: /_figures/wordpress/wp-install-4.png
-      :alt: Successful database connection.
-      :width:     600
-
-      `Figure 4: Successful WordPress connection.`
-
-#. Click :guilabel:`Run the install`.
-   The installer runs until WordPress is fully installed on your system.
-
-Complete successful login
-=========================
-
-Once the installation is complete, you can name your blog and create a WordPress username
-and password. See figure 5.
-
-.. figure:: /_figures/wordpress/wp-install-5.png
-   :alt: WordPress user creation
-   :width:     600
-
-   `Figure 5: WordPress site information screen.`
-
-
-#. Enter all required information.
-#. Click :guilabel:`Install WordPress`.
-#. Verify that the initial login screen appears once the installation is complete. See figure 6:
-
-   .. figure:: /_figures/wordpress/wp-install-6.png
-      :alt: WordPress login
-      :width:     600
-
-      `Figure 6: The WordPress login screen.`
-
-#. Enter your WordPress username and password.
-#. Check :guilabel:`Remember me` to save your credentials.
-#. Click :guilabel:`Log in`.
-
-Figure 7 shows the WordPress dashboard after a successful login:
-
-.. figure:: /_figures/wordpress/wp-install-7.png
-   :alt: WordPress Dashboard
-   :width:     600
-
-   `Figure 7: The WordPress dashboard.`
-
-You are ready to go!
-
-To check out your blog as it is seen by the outside world, enter:
-\http://<your_website>/blog on your browser. Figure 8 shows the result:
-
-.. figure:: /_figures/wordpress/wp-install-8.png
-   :alt: WordPress blog
-   :width:     600
-
-   `Figure 8: Your WordPress blog.`
-
-**Congratulations, your WordPress blog is up and running!**
-
-You have successfully installed WordPress on a host system.
-
-Add new entries to your blog and share them with the world using |CL|!
-
-.. _`step-by-step guide`:
-   https://codex.wordpress.org/Installing_WordPress#Famous_5-Minute_Install
-
-.. _`NUC6i5SYH product page`:
-   http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html

source/tutorials/zfs.rst

@@ -0,0 +1,225 @@
+.. _zfs:
+
+OpenZFS*
+########
+
+This tutorial shows how to set up `OpenZFS* file system and volume manager`_ on |CL-ATTR|, using a non-root device for zpools. 
+
+.. contents::
+   :local:
+   :depth: 1
+
+Background
+**********
+
+The OpenZFS storage platform provides volume management, snapshot capabilities, and redundancy detection. |CL| **does not** ship with a binary ZFS kernel module (zfs.ko). |CL| users who wish to incorporate the zfs.ko kernel module must build and maintain this work themselves.
+
+.. CAUTION::
+
+   Use of the OpenZFS kernel module in connection with |CL| is neither recommended nor officially endorsed by the Clear Linux* Project.  Users who follow this tutorial and build zfs.ko kernel module are encouraged to seek independent legal counsel regarding any plan to redistribute a software package containing zfs.ko and |CL|. 
+
+Known Issues
+************
+
+Using a long-term-support (LTS) kernel when running OpenZFS reduces the risk of incompatibilities with kernel updates. When new kernels or new versions of OpenZFS are released, users bear the responsibility to test those releases and ensure compatibility before deploying any updates. 
+
+Prerequisites
+*************
+
+* Learn to :ref:`kernel-modules-dkms`
+* Learn to use :ref:`swupd-guide`
+
+Install the DKMS kernel
+=======================
+
+.. include:: ../guides/kernel/kernel-modules-dkms.rst
+   :start-after: kernel-modules-dkms-install-begin-alt:
+   :end-before: kernel-modules-dkms-install-end:
+
+Bundles
+=======
+
+Before installing OpenZFS, install the bundles that contain the build dependencies.
+
+.. code-block:: console
+
+    sudo swupd bundle-add wget devpkg-openssl devpkg-util-linux
+
+Install
+*******
+
+Download OpenZFS release
+========================
+
+In this section, we download release 2.0.0 directly from the `OpenZFS repository` (the latest available as of the latest revision of this page).
+
+Download release 2.0.0
+
+   .. code-block:: console
+   
+      cd /usr/src
+      sudo wget https://github.com/openzfs/zfs/releases/download/zfs-2.0.0/zfs-2.0.0.tar.gz
+      sudo tar -xvf zfs-2.0.0.tar.gz
+
+Compile the module
+==================
+
+We will build the module using DKMS. This will enable us to keep the module up to date as new kernels are released in the future.
+
+The ZFS distribution provides a script to build a suitable dkms.conf file.
+
+   Build dkms.conf and install it into the DKMS tree.
+
+   .. code-block:: console
+
+      cd /usr/src/zfs-2.0.0
+      sudo scripts/dkms.mkconf -n zfs -v 2.0.0 -f dkms.conf
+      sudo dkms add -m zfs -v 2.0.0
+      sudo dkms build -m zfs -v 2.0.0
+      sudo dkms install -m zfs -v 2.0.0
+      
+Observe that this install the zfs kernel modules to: 
+
+   :file:`/usr/lib/modules/<kernel-name>/extra/zfs`
+   
+Compile userspace tools
+=======================
+
+Here we compile and install the zfs userspace tools (e.g., zpool, zfs, etc.).
+
+   .. code-block:: console
+   
+      cd /usr/src/zfs-2.0.0
+      sudo ./configure
+      sudo make
+      sudo make install
+
+The binaries are installed at the following directory. While not required, it's recommended to add :file:`/usr/local/sbin` to your path variable.
+
+   .. code-block:: console
+
+      + /usr/local/
+      |--+ bin/
+         |--zvol_wait
+         |--zgenhostid
+         |--raidz_test
+      |--+ etc/
+         |--+ zfs/
+            |--* zed.d/
+            |--+ zpool.d/
+            |--zfs-functions
+      |--+ include/libzfs/ [contents omitted]
+      |--+ lib/
+      |--+ libexec/
+         |--+ zfs/
+            |-- zpool.d/
+            |-- zed.d
+      |--+ share/zfs/ [contents omitted]
+      |--+ sbin/
+         |--fsck.zfs
+         |--zpool
+         |--zdb
+         |--zed
+         |--zfs
+         |--zhack
+         |--zinject
+         |--zpool
+         |--ztest
+         |--zstreamdump
+      |--+ src/
+         |--+ zfs-2.0.0/
+         |--+ spl-2.0.0/
+
+
+Set up systemd
+==============
+
+We now have these unit files available. 
+
+.. code-block:: console
+
+   zfs-env-bootfs.service
+   zfs-zed.service
+   zfs-import-cache.service
+   zfs-import-scan.service
+   zfs-mount.service
+   zfs-share.service
+   zfs-volume-wait.service
+  
+  
+OpenZFS requires that we  explicitly install and enable the services desired. 
+
+To use ZFS automatic zpool import and filesystem mount services, enable them.
+
+.. code-block:: console
+
+   sudo systemctl enable zfs-import-cache
+   sudo systemctl enable zfs-import.target
+   sudo systemctl enable zfs-import-scan
+   sudo systemctl enable zfs-mount
+   sudo systemctl enable zfs.target
+
+
+Load the kernel module at boot
+==============================
+
+OpenZFS kernel modules must be loaded before any OpenZFS filesystems are mounted. For convenience, load the kernel modules at boot.
+
+Systemd uses the `/etc/modules-load.d/` directory to load out-of-tree kernel modules. Make sure that the directory exists:
+
+   .. code-block:: bash
+
+      sudo mkdir -p /etc/modules-load.d
+
+Create the configuration file:
+
+   .. code-block:: bash
+
+      echo "zfs" | sudo tee /etc/modules-load.d/01-zfs.conf
+
+Reboot your system. zfs.ko should be loaded automatically (the module should appear in the outout of command :file:`lsmod`).
+
+.. CAUTION::
+
+   When the |CL| kernel is upgraded, DKMS will attempt to rebuild the OpenZFS module for the new kernel. 
+
+   - DKMS may not have rebuilt the module
+   - DKMS may not have auto-installed the module
+   - The new kernel might introduce breaking changes that prevent zfs
+     from compiling
+
+To fix this situation, recompile zfs.ko with the new kernel code. OpenZFS *might* not compile at all with the new kernel.
+
+.. CAUTION::
+   
+   **Be sure not to put anything on an OpenZFS pool that will be needed to rebuild kernel modules.** Ensure compatibility of OpenZFS with new Linux kernels when released.
+
+Troubleshooting
+===============
+
+If you suspect an issue with DKMS rebuilding your module, you can check two places for information. The dkms-new-kernel service will show status that may help in troubleshooting:
+
+.. code-block:: console
+
+   systemctl status dkms-new-kernel.service
+
+The systemd journal may also have important information:
+
+.. code-block:: console
+
+   journalctl -xe
+
+.. CAUTION::
+   As of OpenZFS 2.0.0, the included file :file:`script/dkms.mkconf` contains a minor incompability -- it calls the command :file:`lsb_release`, which is not available on Clear    Linux by default. It is trivial to edit :file:`dkms.mkconf` and remove the singular reference to :file:`lsb_release` without any ill effects, and then execute the :file:`dkms` commands above. However, keeping the file as provided is perfectly fine, but **will result in :file:`dkms` warnings**.
+
+Next steps
+**********
+You're now ready to create zpools and datasets! For more information on using ZFS, see:
+
+* `FreeBSD Handbook chapter on ZFS`_
+* `ZFS-on-Linux issue tracker`_
+
+.. _FreeBSD Handbook chapter on ZFS: https://www.freebsd.org/doc/handbook/zfs.html
+.. _ZFS-on-Linux issue tracker: https://github.com/openzfs/zfs/issues/10068
+.. _ZFS on Linux repository: https://github.com/openzfs/zfs
+.. _OpenZFS* file system and volume manager: https://github.com/openzfs/zfs

tox.ini

@@ -0,0 +1,13 @@
+[tox]
+envlist = py3-{mylinux,mywindows}
+
+[testenv]
+platform = mylinux: linux
+	mywindows: win32
+whitelist_externals = make.bat
+	/usr/bin/make
+deps = -rrequirements.txt
+commands = 
+	mylinux: make {posargs}
+	mywindows: make.bat {posargs}
+