Merge pull request #7 from clearlinux/master

Update from upstream origin
2026-08-25 01:06:37 +00:00 · 2019-01-16 12:13:41 -08:00
parent 57eb03e316 4205346fd4
commit 549f99f0f3
53 changed files with 3318 additions and 318 deletions
@@ -166,10 +166,8 @@ changes:
 	@echo "The overview file is in $(BUILDDIR)/changes."

 linkcheck:
-	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
-	@echo
-	@echo "Link check complete; look for any errors in the above output " \
-	      "or in $(BUILDDIR)/linkcheck/output.txt."
+	-$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	python3 scripts/_python/linkcheck/parse-link-check.py $(BUILDDIR)/linkcheck

 doctest:
 	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@@ -88,12 +88,12 @@ Minimum installation requirements
 *********************************

 To fulfill minimum installation requirements, complete the
-:guilabel:`Required options`. :guilabel:`Advanced options` are optional. Additional user help is described below.
+:guilabel:`Required options`. :guilabel:`Advanced options` are optional.

 .. note::

   * The :kbd:`Install` button is only highlighted **after** you complete the
-     :ref:`required-options-installation`, and after you enter all required
+     :ref:`required-options-installation`, and after you enter required
     values in submenus of :ref:`required-options-installation` and
     :ref:`advanced-options-installation`.

@@ -105,17 +105,24 @@ To fulfill minimum installation requirements, complete the

 Main Menu
 *********
+The |CL| Installer Main Menu appears as shown in Figure 3.

-In the |CL| Installer Main Menu, two tabs appear:
-:guilabel:`[R] Required options` and :guilabel:`[A] Advanced options`.
-Navigate between tabs using the arrow keys or these shortcut keys:
+.. figure:: figures/bare-metal-install-beta-3.png
+   :scale: 100 %
+   :alt: Clear Linux OS Installer Main Menu
+
+   Figure 3: :guilabel:`Clear Linux OS Installer Main Menu`
+
+The |CL| Installer Main Menu has two tabs: :guilabel:`[R] Required options`
+and :guilabel:`[A] Advanced options`. Navigate between tabs using the arrow
+keys or these shortcut keys:

 * :kbd:`Shift+A` for :guilabel:`[A] Advanced options`
 * :kbd:`Shift+R` for :guilabel:`[R] Required options`

 To meet the minimum requirements, enter your choices in the
 :guilabel:`Required options`. After confirmation, your selections appear
-below menu options in the Main Menu, beside the :guilabel:`>>` chevron.
+beside the :guilabel:`>>` chevron, below the menu options.

 Navigation
 **********
@@ -126,14 +133,6 @@ Navigation

 * Select :kbd:`Cancel` or :kbd:`Esc` to cancel your choice.

-  The |CL| Installer Main Menu appears as shown in Figure 3.
-
-  .. figure:: figures/bare-metal-install-beta-3.png
-     :scale: 100 %
-     :alt: Clear Linux OS Installer Main Menu
-
-     Figure 3: :guilabel:`Clear Linux OS Installer Main Menu`
-
 .. _required-options-installation:

 Required options
@@ -279,13 +278,35 @@ Auto partition will configure the media to meet the minimum requirements.

 #. You may skip to the :ref:`telemetry-tui` section below.

+Disk encryption
+===============
+
+For greater security, disk encryption is supported using LUKS for all
+partitions **except boot** on |CL|. To encrypt a partition, see the example
+below to encrypt :ref:`encrypt-root`. Encryption is optional.
+
+Encryption Passphrase
+---------------------
+
+|CL| uses a single passphrase for all encrypted partitions. Therefore, if
+you turn on encryption for one partition, the same passphrase is used for
+all partitions. Additional or different keys may be configured
+post-installation using the ``cryptsetup`` tool.
+
+.. note::
+
+   During installation, if you encrypt the `root` partition first and the
+   `swap` partition second, after you select encrypt for `swap`, the
+   :guilabel:`Encryption Passphrase` dialogue will show the same Passphrase that was assigned to `root`. Select :guilabel:`Confirm` and
+   press :kbd:`Enter` to use the same Passphrase.
+
 .. _add-partition:

 Add Partition
 =============

 Minimum requirements
-====================
+--------------------

 To add partitions manually, assign partitions per the minimum
 requirements shown in Table 1:
@@ -307,6 +328,7 @@ requirements shown in Table 1:
     - /
     - *Size depends upon use case/desired bundles.*

+
 #. In the Configure Media menu, select :guilabel:`Add Partition`.

   .. note:: To change an existing partition, navigate to the partition,
@@ -356,13 +378,47 @@ swap partition

      Figure 10: :guilabel:`swap partition`

+.. _encrypt-root:
+
 root partition
 --------------

 #. In the :guilabel:`File System` menu, select :kbd:`Up/Down` arrows to
   select the file system type. See Figure 11.

-#. In :guilabel:`Mount Point`, enter ``/root``.
+   #. Optional: Select :guilabel:`[X] Encrypt` to encrypt the partition.
+
+      .. figure:: figures/bare-metal-install-beta-11.png
+         :scale: 100 %
+         :alt: Encrypt partition
+
+         Figure 11: :guilabel:`Encrypt partition`
+
+
+   #. The :guilabel:`Encryption Passphrase` dialogue appears.
+
+      .. note::
+
+         Minimum length is 8 characters. Maximum length is 94 characters.
+
+      .. figure:: figures/bare-metal-install-beta-12.png
+         :scale: 100 %
+         :alt: Encryption Passphrase
+
+         Figure 12: :guilabel:`Encryption Passphrase`
+
+   #. Enter the same passphrase in the first and second field.
+
+   #. Navigate to :guilabel:`Confirm` and press :kbd:`Enter`.
+
+      .. note::
+
+         :guilabel:`Confirm` is only highlighted if passphrases match.
+
+#. Optional: In :guilabel:`[Optional] Label:`, enter your desired
+   label for the partition.
+
+#. In :guilabel:`Mount Point`, enter ``/``. See Figure 11.

 #. In :guilabel:`Size`, enter an appropriate size per Table 1.

@@ -376,21 +432,16 @@ root partition

   You are returned to the :guilabel:`Configure media` menu.

-#. Select :guilabel:`Confirm` to complete the media selection.
+#. Select :guilabel:`Confirm` to complete the media selection. See Figure 13.

-   .. figure:: figures/bare-metal-install-beta-11.png
-      :scale: 100 %
-      :alt: root partition
+#. You are returned to the :guilabel:`Configure media` menu. Your partitions
+   should be similar to those shown in Figure 13.

-      Figure 11: :guilabel:`root partition`
-
-   You are returned to the :guilabel:`Configure media` menu. Your partitions should appear similar to those shown in Figure 12.
-
-   .. figure:: figures/bare-metal-install-beta-12.png
+   .. figure:: figures/bare-metal-install-beta-13.png
      :scale: 100 %
      :alt: Partitions

-   Figure 12: :guilabel:`Partitions`
+      Figure 13: :guilabel:`Partitions`

 #. Navigate to :guilabel:`Confirm` until highlighted.

@@ -413,21 +464,21 @@ For more detailed information, visit our :ref:`telemetry-about` page.

 #. Select :kbd:`Enter` to confirm.

-   .. figure:: figures/bare-metal-install-beta-13.png
+   .. figure:: figures/bare-metal-install-beta-14.png
      :scale: 100 %
      :alt: Enable Telemetry

-      Figure 13: :guilabel:`Enable Telemetry`
+      Figure 14: :guilabel:`Enable Telemetry`

 Skip to finish installation
 ===========================

 After selecting values for all :guilabel:`Required options`, you may skip
-directly to :ref:`finish-cl-installation`
+directly to :ref:`finish-cl-installation`.

-In the Main Menu, select the :guilabel:`Advanced options` if you wish to
-configure network interfaces or proxy settings, add bundles, add/manage
-users, and more. Continue to the next section.
+Otherwise, continue below. In the Main Menu, select
+:guilabel:`Advanced options` to configure network interfaces or proxy
+settings, add bundles, add/manage users, add kernel arguments, and more.

 .. _advanced-options-installation:

@@ -459,22 +510,22 @@ interface settings are automatically applied.

   .. note:: Multiple network interfaces may appear.

-   .. figure:: figures/bare-metal-install-beta-14.png
+   .. figure:: figures/bare-metal-install-beta-15.png
      :scale: 100 %
      :alt: Configure Network Interfaces

-      Figure 14: :guilabel:`Configure Network Interfaces`
+      Figure 15: :guilabel:`Configure Network Interfaces`

 #. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom).

   Optional: Navigate to the checkbox :guilabel:`Automatic / dhcp` and select
   :kbd:`Spacebar` to deselect.

-   .. figure:: figures/bare-metal-install-beta-15.png
+   .. figure:: figures/bare-metal-install-beta-16.png
      :scale: 100 %
      :alt: Network interface configuration

-      Figure 15: :guilabel:`Network interface configuration`
+      Figure 16: :guilabel:`Network interface configuration`

 #. Navigate to the appropriate fields and assign the desired
   network configuration.
@@ -510,11 +561,11 @@ instruction.

 #. Navigate to the field :guilabel:`HTTPS Proxy`.

-   .. figure:: figures/bare-metal-install-beta-16.png
+   .. figure:: figures/bare-metal-install-beta-17.png
      :scale: 100 %
      :alt: Configure the network proxy

-      Figure 16: :guilabel:`Configure the network proxy`
+      Figure 17: :guilabel:`Configure the network proxy`

 #. Enter the desired proxy address and port using conventional syntax,
   such as: \http://address:port.
@@ -534,13 +585,13 @@ Test Network Settings
 To manually assure network connectivity before installing |CL|,
 select :guilabel:`Test Network Settings` and select :guilabel:`Enter`.

-A progress bar appears as shown in Figure 17.
+A progress bar appears as shown in Figure 18.

-.. figure:: figures/bare-metal-install-beta-17.png
+.. figure:: figures/bare-metal-install-beta-18.png
   :scale: 100 %
   :alt: Testing Networking dialogue

-   Figure 17: :guilabel:`Testing Networking dialogue`
+   Figure 18: :guilabel:`Testing Networking dialogue`

 .. note::

@@ -558,13 +609,11 @@ Bundle Selection

 #. Select :kbd:`Spacebar` to select the checkbox for each desired bundle.

-   .. figure:: figures/bare-metal-install-beta-18.png
+   .. figure:: figures/bare-metal-install-beta-19.png
      :scale: 100 %
      :alt: Bundle Selection

-      Figure 18: :guilabel:`Bundle Selection`
-
-..TODO:Add new screenshot that shows more bundles.
+      Figure 19: :guilabel:`Bundle Selection`

 #. Select :kbd:`Confirm` or :kbd:`Cancel`.

@@ -580,13 +629,13 @@ Add New User

 #. In Advanced Options, select :guilabel:`User Manager`.

-#. Select :guilabel:`Add New User` as shown in Figure 19.
+#. Select :guilabel:`Add New User` as shown in Figure 20.

-   .. figure:: figures/bare-metal-install-beta-19.png
+   .. figure:: figures/bare-metal-install-beta-20.png
      :scale: 100 %
      :alt: Add New User, User Name

-      Figure 19: :guilabel:`Add New User`
+      Figure 20: :guilabel:`Add New User`

 #. Optional: Enter a :guilabel:`User Name`.

@@ -594,11 +643,11 @@ Add New User

      The User Name must be alphanumeric and can include spaces, commas, or hyphens. Maximum length is 64 characters.

-   .. figure:: figures/bare-metal-install-beta-20.png
+   .. figure:: figures/bare-metal-install-beta-21.png
      :scale: 100 %
      :alt: User Name

-      Figure 20: :guilabel:`User Name`
+      Figure 21: :guilabel:`User Name`

 #. Enter a :guilabel:`Login`.

@@ -621,11 +670,11 @@ Add New User

      Selecting this option enables sudo privileges for the user.

-   .. figure:: figures/bare-metal-install-beta-21.png
+   .. figure:: figures/bare-metal-install-beta-22.png
      :scale: 100 %
      :alt: Administrative rights

-      Figure 21: :guilabel:`Administrative rights`
+      Figure 22: :guilabel:`Administrative rights`

 #. Select :kbd:`Confirm`.

@@ -641,15 +690,15 @@ Modify / Delete User
 --------------------

 #. In User Manager, select :guilabel:`Tab` to highlight the user, as shown
-   in Figure 22.
+   in Figure 23.

 #. Select :kbd:`Enter` to modify the user.

-   .. figure:: figures/bare-metal-install-beta-22.png
+   .. figure:: figures/bare-metal-install-beta-23.png
      :scale: 100 %
      :alt: Modify User

-      Figure 22: :guilabel:`Modify User`
+      Figure 23: :guilabel:`Modify User`

 #. Modify user details as desired.

@@ -683,11 +732,11 @@ new kernel.

 #. To add or remove arguments, choose one of the following options.

-   .. figure:: figures/bare-metal-install-beta-23.png
+   .. figure:: figures/bare-metal-install-beta-24.png
      :scale: 100 %
      :alt: kernel command line

-      Figure 23: :guilabel:`kernel command line`
+      Figure 24: :guilabel:`kernel command line`

 #. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`.

@@ -702,17 +751,15 @@ Kernel Selection
 ================

 #. Select a kernel option. By default, the latest kernel release is
-   selected.
+   selected. Native kernel is shown in Figure 25.

-   .. note::
+#. To select a differnt kernel, navigate to it using :guilabel:`Tab`.

-      To choose a different kernel, such as Long Term Support (LTS), shown in Figure 24, navigate to it using :guilabel:`Tab`.
-
-   .. figure:: figures/bare-metal-install-beta-24.png
+   .. figure:: figures/bare-metal-install-beta-25.png
      :scale: 100 %
      :alt: Kernel selection

-      Figure 24: :guilabel:`Kernel selection`
+      Figure 25: :guilabel:`Kernel selection`

 #. Select :kbd:`Spacebar` to select the desired option.

@@ -731,11 +778,11 @@ If you have your own custom mirror of |CL|, you can add its URL.

 #. Select :kbd:`Confirm`.

-   .. figure:: figures/bare-metal-install-beta-25.png
+   .. figure:: figures/bare-metal-install-beta-26.png
      :scale: 100 %
      :alt: Swupd Mirror

-      Figure 25: :guilabel:`Swupd Mirror`
+      Figure 26: :guilabel:`Swupd Mirror`

 Optional: :ref:`Skip to finish installation <finish-cl-installation>`.

@@ -752,11 +799,11 @@ Assign Hostname
      alphanumeric character but may also contain hyphens. Maximum length of
      63 characters.

-   .. figure:: figures/bare-metal-install-beta-26.png
+   .. figure:: figures/bare-metal-install-beta-27.png
      :scale: 100 %
      :alt: Assign Hostname

-      Figure 26: :guilabel:`Assign Hostname`
+      Figure 27: :guilabel:`Assign Hostname`


 #. Navigate to :kbd:`Confirm` until highlighted.
@@ -775,11 +822,11 @@ follow the onscreen instructions.

 #. Select the desired option.

-   .. figure:: figures/bare-metal-install-beta-27.png
+   .. figure:: figures/bare-metal-install-beta-28.png
      :scale: 100 %
      :alt: Automatic OS Updates

-      Figure 27: :guilabel:`Automatic OS Updates`
+      Figure 28: :guilabel:`Automatic OS Updates`

 You are returned to the :guilabel:`Main Menu`.

@@ -35,4 +35,5 @@ appropriate set of step-by-step instructions to proceed.
   vmw-player
   vmw-player-preconf
   hyper-v
+   ../../guides/maintenance/increase-virtual-disk-size.rst

@@ -71,7 +71,8 @@ Build kernel module

      $ uname -r
      4.XX.YY-ZZZZ.native
-      # Ensure *.native* is in the kernel name
+
+   Ensure *.native* is in the kernel name

 #. Install the `linux-dev` bundle to obtain the kernel headers, which are
   required for compiling kernel modules.
@@ -173,4 +174,4 @@ You can find more info on module loading in the modules-load.d manual page:
   man modules-load.d

 .. _`on GitHub`: https://github.com/clearlinux/distribution 
-.. _`mixer tool`: https://clearlinux.org/features/mixer-tool
+.. _`mixer tool`: https://clearlinux.org/features/mixer-tool
@@ -122,7 +122,7 @@ To set up |CL| manually, perform the steps below.
   .. code-block:: console

      sudo cat > $ipxe_root/ipxe_boot_script.ipxe << EOF
-      sudo!ipxe
+      #!ipxe
      kernel linux quiet init=/usr/lib/systemd/systemd-bootchart \
      initcall_debug tsc=reliable no_timer_check noreplace-smp rw \
      initrd=initrd
@@ -16,4 +16,4 @@ telemetry backend solution. For more details, learn how to
   telemetry-config
   telemctl
   telemetry-journal
-   telemetry-z-api
+   telemetry-api
@@ -3,7 +3,9 @@
 Available bundles
 #################

-This document provides a current list of `available bundles`_. To learn more about how |CL-ATTR| uses bundles for software deployment, visit :ref:`bundles-about`.  
+This document provides a current list of available bundles. To learn more about how |CL-ATTR| uses bundles for software deployment, visit :ref:`bundles-about`.
+
+To view bundles or packages on GitHub, select the :file:`Name` in column one.

 Bundle list
 ===========
@@ -11,4 +13,3 @@ Bundle list
 .. raw:: html
   :file: bundles.html.txt

-.. _available bundles: https://github.com/clearlinux/clr-bundles
@@ -0,0 +1,235 @@
+.. _dlrs:
+
+Deep Learning Reference Stack
+#############################
+
+This tutorial shows you how to run benchmarking workloads in |CL-ATTR| using
+TensorFlow\* and Kubeflow with the Deep Learning Reference Stack.
+
+The Deep Learning Reference Stack is available in two versions.
+The first is `Eigen`_, which includes `TensorFlow`_ optimized for Intel®
+architecture. The second is `Intel MKL-DNN`_, which includes the TensorFlow
+framework optimized using Intel® Math Kernel Library for Deep Neural
+Networks (Intel® MKL-DNN) primitives.
+
+.. contents:: :local:
+   :depth: 1
+
+Release notes
+=============
+
+View current `release notes`_ for the Deep Learning Reference Stack.
+
+View current `benchmark results`_ for the Deep Learning Reference Stack.
+
+.. note::
+
+   Performance test numbers in the Deep Learning Reference Stack were obtained using `runc` as the runtime.
+
+Prerequisites
+=============
+
+* |CL| installed on host system. If not installed, :ref:`bare-metal-install`
+* `containers-basic` bundle
+* `cloud-native-basic` bundle
+
+In |CL|, `containers-basic` provides Docker\*, which is required for
+TensorFlow benchmarking. Use the :command:`swupd` utility to check if
+`containers-basic` and `cloud-native-basic` are present:
+
+.. code-block:: bash
+
+   sudo swupd bundle-list
+
+If you need to install the `containers-basic` or `cloud-native-basic`, enter:
+
+.. code-block:: bash
+
+   sudo swupd bundle-add containers-basic cloud-native-basic
+
+To ensure that kubernetes is correctly installed and configured,
+:ref:`kubernetes`.
+
+We have validated these steps against the following software package
+versions:
+
+* |CL| 26240--lowest version permissible.
+* Docker 18.06.1
+* Kubernetes 1.11.3
+* Go 1.11.12
+
+TensorFlow single and multi-node benchmarks
+============================================
+
+This section describes running the `TensorFlow benchmarks`_ in single node.
+For multi-node testing, replicate these steps for each node. These steps
+provide a template to run other benchmarks, provided that they can invoke
+TensorFlow.
+
+#. Download and run either the `Eigen`_ or the `Intel MKL-DNN`_ docker image
+   from `Docker Hub`_.
+
+   .. note::
+
+      You will enter the following commands in the running container.
+
+      Replace <docker_name> with the name of the image.
+
+#. Clone the benchmark repository:
+
+   .. code-block:: bash
+
+      docker exec -t <docker_name> bash -c ‘git clone http://github.com/tensorflow/benchmarks -b cnn_tf_v1.12_compatible’
+
+#. Next, execute the benchmark script to run the benchmark.
+
+   .. code-block:: bash
+
+      docker exec -i <docker_name> bash -c ‘python benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC ’.
+
+.. note::
+
+   You can replace the model with one of your choice supported by the
+   TensorFlow benchmarks.
+
+Kubeflow multi-node benchmarks
+==============================
+
+The benchmark workload will run in a Kubernetes cluster. We will use
+`Kubeflow`_ for the Machine Learning workload deployment on three nodes.
+
+Kubernetes setup
+****************
+
+Follow the instructions in the :ref:`kubernetes` tutorial to get set up on
+|CL|. The kubernetes community also has
+`instructions for creating a cluster`_.
+
+Kubernetes networking
+*********************
+
+We used `flannel`_ as the network provider for these tests. If you are
+comfortable with another network layer, refer to the Kubernetes
+`networking documentation`_ for setup.
+
+Images
+******
+
+We need to add `launcher.py` to our docker image to include the Deep
+Learning Reference Stack and put the benchmarks repo in the correct
+location. From the docker image, run the following:
+
+.. code-block:: bash
+
+   mkdir -p /opt
+   git clone https://github.com/tensorflow/benchmarks.git /opt/tf-benchmarks
+   cp launcher.py /opt
+   chmod u+x /opt/*
+
+Your entry point now becomes "/opt/launcher.py".
+
+This will build an image which can be consumed directly by TFJob from
+kubeflow. We are working to create these images as part of our release
+cycle.
+
+ksonnet\*
+*********
+
+Kubeflow uses ksonnet* to manage deployments, so we need to install that before setting up Kubeflow. On |CL|, follow these steps:
+
+.. code-block:: bash
+
+   swupd bundle-add go-basic-dev
+   export GOPATH=$HOME/go
+   export PATH=$PATH:$GOPATH/bin
+   go get github.com/ksonnet/ksonnet
+   cd $GOPATH/src/github.com/ksonnet/ksonnet
+   make install
+
+After the ksonnet installation is complete, ensure that binary `ks` is
+accessible across the environment.
+
+Kubeflow
+********
+
+Once you have Kubernetes running on your nodes, you can setup `Kubeflow`_ by
+following these instructions from their `quick start guide`_.
+
+.. code-block:: bash
+
+   export KUBEFLOW_SRC=$HOME/kflow
+   export KUBEFLOW_TAG=”v0.3.2”
+   export KFAPP=”kflow_app”
+   export K8S_NAMESPACE=”kubeflow”
+
+   mkdir ${KUBEFLOW_SRC}
+   cd ${KUBEFLOW_SRC}
+   ks init ${KFAPP}
+   cd ${KFAPP}
+   ks registry add kubeflow github.com/kubeflow/kubeflow/tree/${KUBEFLOW_TAG}/kubeflow
+   ks pkg install kubeflow/core
+
+Now you have all the required kubeflow packages, and you can deploy the primary one for our purposes: tf-job-operator.
+
+.. code-block:: bash
+
+   ks env rm default
+   kubectl create namespace ${K8S_NAMESPACE}
+   ks env add default --namespace "${K8S_NAMESPACE}"
+   ks generate tf-job-operator tf-job-operator
+   ks apply default -c tf-job-operator
+
+This creates the CustomResourceDefinition(CRD) endpoint to launch a TFJob.
+
+Run a TFJob
+===========
+
+#. Select this link for the `ksonnet registries for deploying TFJobs`_.
+
+   #. Install the TFJob componets as follows:
+
+      .. code-block:: bash
+
+         ks registry add dlrs-tfjob github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
+
+         ks pkg install dlrs-tfjob/dlrs-bench
+
+#. Next, generate Kubernetes manifests for the workloads and apply them to
+   create and run them using these commands
+
+   .. code-block:: bash
+
+      ks generate dlrs-resnet50 dlrsresnet50 --name=dlrsresnet50
+      ks generate dlrs-alexnet dlrsalexnet --name=dlrsalexnet
+      ks apply default -c dlrsresnet50
+      ks apply default -c dlrsalexnet
+
+This will replicate and deploy three test setups in your Kubernetes cluster.
+
+Results of Running this Tutorial
+================================
+You need to parse the logs of the Kubernetes pod to get the performance
+numbers. The pods will still be around post completion and will be in
+‘Completed’ state. You can get the logs from any of the pods to inspect the
+benchmark results. More information about `Kubernetes logging`_ is available from the Kubernetes community.
+
+.. _TensorFlow: https://www.tensorflow.org/
+.. _Kubeflow: https://www.kubeflow.org/
+.. _Docker Hub: https://hub.docker.com/
+.. _TensorFlow benchmarks: https://www.tensorflow.org/guide/performance/benchmarks
+.. _instructions for creating a cluster: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
+.. _flannel: https://github.com/coreos/flannel
+.. _networking documentation: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network
+.. _quick start guide: https://www.kubeflow.org/docs/started/getting-started/
+
+.. _Eigen: https://hub.docker.com/r/clearlinux/stacks-dlrs-oss/
+.. _Intel MKL-DNN: https://hub.docker.com/r/clearlinux/stacks-dlrs-mkl/
+
+.. _release notes: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs
+
+.. _ksonnet registries for deploying TFJobs: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
+
+
+.. _Kubernetes logging: https://kubernetes.io/docs/concepts/cluster-administration/logging/
+
+.. _benchmark results: https://clearlinux.org/stacks/deep-learning-reference-stack
@@ -27,7 +27,7 @@ Supported Platforms
 Description of Samples
 **********************

-The AWS Greengrass samples are located at the `Edge-Analytics-FaaS`_.
+The AWS Greengrass samples are located at the `Edge-Analytics-FaaS`_. For this tutorial we will use the 1.0 version of the source code. 

 We provide the following AWS Greengrass samples:

@@ -109,7 +109,7 @@ Converting Deep Learning Models
 Locate Sample Models
 --------------------

-There are two types of provided models that can be used in conjunction with AWS Greengrass 
+There are two types of provided models that can be used in conjunction with AWS Greengrass
 for this tutorial: classification or object detection.

 To complete this tutorial using an image classification model,
@@ -119,8 +119,8 @@ Any custom pre-trained classification models can be used with the
 classification sample.

 For object detection, the sample models optimized for Intel® edge platforms
-are included with the computer-vision-basic bundle installation at :file:`/usr/share/openvino/models`. 
-These models are provided as an example; however, you may also use a custom SSD model 
+are included with the computer-vision-basic bundle installation at :file:`/usr/share/openvino/models`.
+These models are provided as an example; however, you may also use a custom SSD model
 with the Greengrass object detection sample.

 Running Model Optimizer
@@ -132,9 +132,9 @@ For classification using BVLC Alexnet model:

 .. code-block:: bash

-   python3 mo.py --framework caffe --input_model 
-   <model_location>/bvlc_alexnet.caffemodel --input_proto 
-   <model_location>/deploy.prototxt --data_type <data_type> --output_dir 
+   python3 mo.py --framework caffe --input_model
+   <model_location>/bvlc_alexnet.caffemodel --input_proto
+   <model_location>/deploy.prototxt --data_type <data_type> --output_dir
   <output_dir> --input_shape [1,3,227,227]

 For object detection using SqueezeNetSSD-5Class model:
@@ -175,7 +175,7 @@ cloud and edge.
   `AWS Greengrass developer guide`_

 #. To install and configure AWS Greengrass core on edge platform, follow
-   the instructions at `Start AWS Greengrass on the Core Device`_. In 
+   the instructions at `Start AWS Greengrass on the Core Device`_. In
   step 8(b), download the x86_64 Ubuntu configuration of the AWS Greengrass
   core software.

@@ -196,7 +196,7 @@ cloud and edge.
 Creating and Packaging Lambda Functions
 =======================================

-#. Complete steps 1-4 of the tutorial at `Create and Package Lambda Function`_ . 
+#. Complete steps 1-4 of the tutorial at `Create and Package Lambda Function`_ .

   .. note::

@@ -204,7 +204,7 @@ Creating and Packaging Lambda Functions
      environment on the edge device.


-#. In step 5, replace greengrassHelloWorld.py with the classification or object detection 
+#. In step 5, replace greengrassHelloWorld.py with the classification or object detection
   Greengrass sample from `Edge-Analytics-Faas`_:

   * Classification: `greengrass_classification_sample.py`_
@@ -264,8 +264,8 @@ configuring the lambda function for AWS Greengrass.
        - Value
      * - PARAM_MODEL_XML
        - <MODEL_DIR>/<IR.xml>, where <MODEL_DIR> is user specified and
-          contains IR.xml, the Intermediate Representation file from Intel® Model Optimizer. 
-          For this tutorial, <MODEL_DIR> should be set to '/usr/share/openvino/models' 
+          contains IR.xml, the Intermediate Representation file from Intel® Model Optimizer.
+          For this tutorial, <MODEL_DIR> should be set to '/usr/share/openvino/models'
          or one of its subdirectories.
      * - PARAM_INPUT_SOURCE
        - <DATA_DIR>/input.webm to be specified by user. Holds both input and
@@ -375,15 +375,15 @@ References
 2. AWS Lambda: https://aws.amazon.com/lambda/
 3. AWS Kinesis: https://aws.amazon.com/kinesis/

-.. _Edge-Analytics-FaaS: https://github.com/intel/Edge-Analytics-FaaS/tree/master/AWS%20Greengrass
+.. _Edge-Analytics-FaaS: https://github.com/intel/Edge-Analytics-FaaS/tree/v1.0/AWS%20Greengrass

 .. _bvlc_alexnet.caffemodel: http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel

 .. _deploy.prototxt: https://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/deploy.prototxt

-.. _greengrass_classification_sample.py: https://github.com/intel/Edge-Analytics-FaaS/blob/master/AWS%20Greengrass/greengrass_classification_sample.py
+.. _greengrass_classification_sample.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_classification_sample.py

-.. _greengrass_object_detection_sample_ssd.py: https://github.com/intel/Edge-Analytics-FaaS/blob/master/AWS%20Greengrass/greengrass_object_detection_sample_ssd.py
+.. _greengrass_object_detection_sample_ssd.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_object_detection_sample_ssd.py

 .. _converting deep learning models to Intermediate Representation using Model Optimizer: https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer

@@ -4,32 +4,31 @@ Run Kubernetes\* on |CL-ATTR|
 #############################

 This tutorial describes how to install, configure, and run the
-`Kubernetes container orchestration system`_ on |CL-ATTR| using different
-container engines and runtimes.
+`Kubernetes container orchestration system`_ on |CL-ATTR| using CRI+O and
+kata-runtime.

 Kubernetes\* is an open source system for automating deployment, scaling, and
 management of containerized applications. It groups containers that make up
 an application into logical units for easy management and discovery.

-Runc and Kata Containers\* kata-runtime adhere to :abbr:`OCI (Open Container Initiative*)`
-guidelines and work seamlessly with Kubernetes. `Kata Containers`_ provide
-strong isolation for untrusted workloads or  multi-tenant scenarios. Runc and
-Kata Containers can be allocated on a  per-pod basis so you can mix and match
-both on the same host to suit your needs.
+Kata Containers\* kata-runtime adheres to
+:abbr:`OCI (Open Container Initiative*)` guidelines and work seamlessly with
+Kubernetes. `Kata Containers`_ provide strong isolation for untrusted
+workloads or  multi-tenant scenarios. Kata Containers can be
+allocated on a per-pod basis so you can mix and match both on the same host
+to suit your needs.

-This tutorial describes the following combinations:
-
-* Kubernetes with Docker and runc
-* Kubernetes with CRI-O and kata-runtime
+.. contents:: :local:
+   :depth: 1

 Prerequisites
 *************

-This tutorial assumes you have installed |CL| and updated to the latest
-release on your host system. You can learn about the benefits of having an
-up-to-date system for cloud orchestration on the :ref:`swupd-about`
-page. For detailed instructions on installing |CL| on a bare metal system,
-follow the :ref:`bare metal installation tutorial<bare-metal-install>`.
+This tutorial assumes you have already installed |CL|. For detailed
+instructions on installing |CL| on a bare metal system, follow the
+:ref:`bare metal installation tutorial<bare-metal-install>`. Learn about the
+benefits of having an up-to-date system for cloud orchestration on the
+:ref:`swupd-about` page.

 Before you install any new packages, update |CL| with the following command:

@@ -40,9 +39,16 @@ Before you install any new packages, update |CL| with the following command:
 Install Kubernetes and CRI runtimes
 ***********************************

-Kubernetes and a set of supported :abbr:`CRI (Container Runtime Interface)`
-runtimes are included in the `cloud-native-basic`_ bundle. To install the
-framework, enter the following command:
+Kubernetes, a set of supported :abbr:`CRI (Container Runtime Interface)`
+runtimes, and networking plugins, are included in the `cloud-native-basic`_
+bundle.
+
+.. note::
+
+   CRI-O’s default plugin_dir is :file:`/opt/bin/cni`.
+   CNI plugins are installed as part of ``cloud-native-basic``.
+
+To install this framework, enter the following command:

 .. code-block:: bash

@@ -105,7 +111,7 @@ deployment and your security needs.
         On systems with limited resources, some performance degradation may
         be observed while swap is disabled.

-#. Switch to root to modify `hostname`:
+#. Switch to root to modify the `hosts` file:

   .. code-block:: bash

@@ -127,49 +133,7 @@ deployment and your security needs.
 Configure and run Kubernetes
 ****************************

-This section describes how to configure and run Kubernetes with:
-
-* Docker and runc
-* CRI-O and kata-runtime
-
-Configure and run Docker + runc
-===============================
-
-#.  Enable the Docker service:
-
-    .. code-block:: bash
-
-       sudo systemctl enable docker.service
-
-#.  Create (or edit if it exists) the file
-    :file:`/etc/systemd/system/docker.service.d/51-runtime.conf` and include the following lines:
-
-    .. code-block:: bash
-
-       [Service]
-       Environment="DOCKER_DEFAULT_RUNTIME=--default-runtime runc"
-
-#.  Create (or edit if it exists) the file :file:`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and include the following lines:
-
-    .. code-block:: bash
-
-       [Service]
-       Environment="KUBELET_EXTRA_ARGS="
-
-#.  Enter the commands:
-
-    .. code-block:: bash
-
-       sudo systemctl daemon-reload
-       sudo systemctl restart docker
-       sudo systemctl restart kubelet
-
-#.  Initialize the master control plane with the command:
-
-    .. code-block:: bash
-
-       sudo kubeadm init --ignore-preflight-errors=SystemVerification
-
+This section describes how to configure and run Kubernetes with CRI-O and kata-runtime.

 Configure and run CRI-O + kata-runtime
 ======================================
@@ -211,67 +175,57 @@ If you choose the `flannel` add-on, then you must add the following to the

    --pod-network-cidr 10.244.0.0/16

-If you are using CRI-O and `flannel` and you want to use Kata Containers, edit
-the :file:`/etc/crio/crio.conf` file to add:
+If you are using CRI-O and `flannel` and you want to use Kata Containers, edit the :file:`/etc/crio/crio.conf` file to add:

 ..  code-block:: bash

    [crio.runtime]
    manage_network_ns_lifecycle = true

+Create a symlink for the network overlays:
+
+.. code-block:: bash
+
+   sudo ln -s /usr/libexec/cni /opt/cni/bin
+
+.. note::
+
+   |CL| installs CNI plugins that are part of the `cloud-native-basic`
+   bundle to :file:`/usr/libexec/cni`. The directory is required because `
+   swupd verify` may use it to repair a system to a known good state.
+
 **Notes about Weave Net add-on**

-If you choose the `Weave Net` add-on, then you must make the following changes
-because it installs itself in the :file:`/opt/cni/bin` directory.
+If you choose the `Weave Net` add-on, you must make the following
+changes because it installs itself in the :file:`/opt/cni/bin` directory.

-If you are using Docker and `Weave Net`, edit the :file:`kubeadm.conf` file to
-add:
-
-..  code-block:: bash
-
-    Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
-
-If you are using CRI-O and `Weave Net`, you must complete the following steps.
-
-#.  Edit the :file:`/etc/crio/crio.conf` file to change `plugin_dir` from:
-
-    ..  code-block:: bash
-
-        plugin_dir = "/usr/libexec/cni/"
-
-    to:
-
-    ..  code-block:: bash
-
-        plugin_dir = "/opt/cni/bin"
+For using CRI-O and ``Weave Net``, complete the following step.

 #.  Add the `loopback` CNI plugin to the plugin path with the command:

-    ..  code-block:: bash
-
-        sudo ln -s /usr/libexec/cni/loopback /opt/cni/bin/loopback
+    code-block:: bash

+    sudo ln -s /usr/libexec/cni/loopback /opt/bin/cni/loopback

 Use your cluster
 ****************

 Once your master control plane is successfully initialized, instructions on
 how to use your cluster and its *IP*, *token*, and *hash* values are
-displayed. It is important that you record the cluster values because they are
-needed when joining worker nodes to the cluster. Some values have a valid
+displayed. It is important that you record the cluster values because they
+are needed when joining worker nodes to the cluster. Some values have a valid
 period. The values are presented in a format similar to:

 .. code-block:: bash

   kubeadm join <master-ip>:<master-port> --token <token> --discovery-token-ca-cert-hash <hash>

-
 **Congratulations!**

-You've successfully installed and set up Kubernetes in |CL| using Docker and
-runc or CRI-O and kata-runtime. You are now ready to follow on-screen
-instructions to deploy a pod network to the cluster and join worker nodes
-with the displayed token and IP information.
+You've successfully installed and set up Kubernetes in |CL| using CRI-O and
+kata-runtime. You are now ready to follow on-screen instructions to deploy a
+pod network to the cluster and join worker nodes with the displayed token
+and IP information.

 Related topics
 **************
@@ -314,8 +268,7 @@ Proxy configuration (optional)
 ******************************

 If you use a proxy server, you must set your proxy environment variables and
-create an appropriate proxy configuration file for both CRI-O and Docker
-services. Consult your IT department if you are behind a corporate proxy for
+create an appropriate proxy configuration file for both CRI-O services. Consult your IT department if you are behind a corporate proxy for
 the appropriate values. Ensure that your local IP is **explicitly included**
 in the environment variable *NO_PROXY*. (Setting *localhost* is not enough.)

@@ -324,17 +277,17 @@ commands as a shell script to configure all of these services in one step:

 .. code-block:: bash

-      services=('crio' 'docker')
-      for s in "${services[@]}"; do
-      sudo mkdir -p "/etc/systemd/system/${s}.service.d/"
-      cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf"
-      [Service]
-      Environment="HTTP_PROXY=${http_proxy}"
-      Environment="HTTPS_PROXY=${https_proxy}"
-      Environment="SOCKS_PROXY=${socks_proxy}"
-      Environment="NO_PROXY=${no_proxy}"
-      EOF
-      done
+   services=('crio')
+   for s in "${services[@]}"; do
+   sudo mkdir -p "/etc/systemd/system/${s}.service.d/"
+   cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf"
+   [Service]
+   Environment="HTTP_PROXY=${http_proxy}"
+   Environment="HTTPS_PROXY=${https_proxy}"
+   Environment="SOCKS_PROXY=${socks_proxy}"
+   Environment="NO_PROXY=${no_proxy}"
+   EOF
+   done

 Troubleshooting
 ***************
@@ -376,6 +329,44 @@ Troubleshooting
  to set the proxy variables permanently, and how to make them available for
  all the types of access that you will use, such as remote SSH access.

+  If the result of the above commands is blank, you may need to add a
+  ``profile`` to the :file:`/etc` directory. To do so, follow these steps.
+
+  #. Create a `profile` in :file:`/etc`
+
+     .. code-block:: bash
+
+        sudo touch profile
+
+  #. With a preferred editor, open `profile`, and enter your proxy settings.
+     Example shown below.
+
+     .. code-block:: bash
+
+        export "HTTP_PROXY=http://proxy.example.com:443"
+        export "HTTPS_PROXY=http://proxy.example.com:445"
+        export "SOCKS_PROXY=http://proxy.example.com:1080"
+        export "NO_PROXY= site.com,.site.com,localhost,127.0.0.1,<master IP>
+
+  <master IP> can be obtained by running :command:`ifconfig`.
+
+  #. Save and exit the `profile`.
+
+  #. Run:
+
+     .. code-block:: bash
+
+        sudo source profile
+
+  #. To assure your system isn't running previous session variables, run:
+
+     .. code-block:: bash
+
+        sudo kubeadm reset --cri-socket=/run/crio/crio.sock
+
+  #. Continue below while pass `-E` in the command as shown.
+
+
 * Missing environment variables.

  If you are behind a proxy server, pass environment variables by adding *-E*
@@ -383,9 +374,6 @@ Troubleshooting

  .. code-block:: bash

-    /* Kubernetes with Docker + runc */
-    sudo -E kubeadm init --ignore-preflight-errors=SystemVerification
-
    /* Kubernetes with CRI-O + kata-runtime */
    sudo -E kubeadm init --cri-socket=/run/crio/crio.sock

@@ -386,7 +386,6 @@ server onto and type in ``localhost`` in the address bar.  You should see a
 web page similar to the one shown in figure 1:

 .. figure:: figures/telemetry-backend-1.png
-   :scale: 50 %
   :alt: Telemetry UI

   Figure 1: :guilabel:`Telemetry UI`
@@ -429,11 +428,11 @@ setting to your new server location.
   You can also use the fully qualified domain name for your server instead of
   :guilabel:`localhost`.

-#. Restart the :command:`telemd` daemon to reload the configuration file.
+#. Restart the telemetry daemons to reload the configuration file.

   .. code-block:: console

-      systemctl restart telemd
+      telemctl restart

 Test the new telemetry backend server
 *************************************
@@ -450,7 +449,6 @@ command to send a ``hello`` record to the server as follows:
 The record should show up on your new server console as shown in figure 2:

 .. figure:: figures/telemetry-backend-2.png
-   :scale: 50 %
   :alt: Telemetry UI

   Figure 2: :guilabel:`Telemetry UI`
@@ -0,0 +1,617 @@
+.. _telemetry-e2e:
+
+Develop with telemetry
+######################
+
+This tutorial shows you how to set up a telemetry backend server to
+manage your records and how to instrument your application with the telemetry
+API.
+
+|CL-ATTR| includes a telemetry and analytics solution (also known as
+telemetrics) as part of the OS, which records events of interest and reports
+them back to the development team using the telemetrics client daemons.
+
+The |CL| telemetry client can be enabled or disabled and records can be
+redirected to a desired location. More detailed information about using and
+configuring the telemetrics client is found in
+the :ref:`telemetrics` guide.
+
+
+.. contents:: :local:
+   :depth: 1
+
+
+Prerequisites
+=============
+
+
+For this tutorial, you can use an existing |CL| system, or you can start with a clean installation of |CL| on a new system.
+
+New Installation
+****************
+
+To setup a new system for your telemetry backend server, follow the :ref:`bare-metal-install` getting started guide and:
+
+#. Choose to install |CL|.
+#. Join the :guilabel:`Stability Enhancement Program` during the installation process to enable the telemetrics client components.
+#. Select the manual installation method with the following settings:
+   * Set the hostname to :guilabel:`clr-telem-server`
+   * Create an administrative user named :guilabel:`clear` and add this user to sudoers :ref:`enable-user-space`
+   * Choose the :file:`dev-utils`, :file:`network-basic`, and :file:`openssh-server` bundles from the bundle list
+
+
+.. note::
+
+   Bundles can also be added to your system after this install process completed.  The bundles listed here are a minimal set needed to complete the setup of the telemetry backend server and applications.
+
+Existing System
+***************
+
+If you are using an existing |CL| system, make sure you have installed the telemetry and dev-utils bundles.  Use the :command:`swupd` utility with the `bundle-list` option and check for "telemetrics" in the list:
+
+.. code-block:: bash
+
+   sudo swupd bundle-list
+
+If you need to install the bundles, use :command:`swupd` to do so.
+
+.. code-block:: bash
+
+   sudo swupd bundle-add telemetrics dev-utils
+
+More information about enabling and configuring the telemetry client can be found at :ref:`telemetry-enable`.
+
+You will need to run some of the commands in this tutorial with root privileges.  You can create a new user or add your user to the sudoers list :ref:`enable-user-space`.
+
+Setting up the telemetry backend server
+=======================================
+We'll be using the :file:`deploy.sh` file from the `clearlinux/telemetrics-backend`_ Git repository to install required dependencies for the web server applications.  The script also configures nginx and uwsgi, deploys snapshots of the applications, and starts all required services.
+
+Clone the clearlinux/telemetrics-backend Git repository
+*******************************************************
+
+With all prerequisite software bundles installed, log in with your administrative user, and from your :file:`$HOME` directory, run :command:`git` to clone the :guilabel:`telemetrics-backend` repository into the :file:`$HOME/telemetrics-backend` directory:
+
+.. code-block:: bash
+
+   git clone https://github.com/clearlinux/telemetrics-backend
+
+.. note::
+
+   You may need to set up the :envvar:`https_proxy` environment variable if you have issues reaching github.com.
+
+Run the deploy.sh script to install the backend server
+******************************************************
+
+Change your current working directory to :file:`telemetrics-backend/scripts`.
+
+.. code-block:: bash
+
+   cd telemetrics-backend/scripts
+
+Run the :command:`./deploy.sh -h` to see the list of options for the :command:`deploy.sh` script:
+
+.. code-block:: console
+
+   ./deploy.sh -h
+   Deploy snapshot of the telemetrics-backend
+
+        -a    Perform specified action (deploy, install, migrate, resetdb,
+              restart, uninstall; default: deploy)
+        -d    Distro to deploy to (ubuntu, centos or clr; default: ubuntu)
+        -h    Print these options
+        -H    Set domain for deployment (only accepted value is "localhost" for
+              now)
+        -r    Set repo location to deploy from
+              (default: https://github.com/clearlinux/telemetrics-backend)
+        -s    Set source location (default: "master" branch from git repo)
+        -t    Set source type (tarball, or git; default: git)
+        -u    Perform complete uninstallation
+
+The :command:`deploy.sh` is a bash shell script that allows you to perform the following actions:
+
+* *deploy* - install a complete instance of the telemetrics backend server and all required components. This is the default action if no *-a* argument is given on the command line.
+* *install* - installs and enables all required components for the telemetrics backend server.
+* *migrate* - migrate database to new schema.
+* *resetdb* - reset the database.
+* *restart* - restart the nginx and uWSGI services.
+* *uninstall* - uninstall all packages.
+
+.. note::
+
+   The *uninstall* option does not perform any actions if the distro is set to |CL| and will only uninstall packages if the distro is Ubuntu
+
+Next, we will install the telemetrics backend server with the following options:
+
+* *-a install* to perform an install
+* *-d clr* to install to a |CL| distro
+* *-H localhost* to set the domain to localhost
+
+We do not need to set the following options since the values are set to the correct values we want by default:
+
+* *-r https://github.com/clearlinux/telemetrics-backend* sets the repo location for :command:`git` to clone from.
+* *-s master* to set the location, or branch.
+* *-t git* to set the source type to git.
+
+.. caution::
+
+   The :file:`deploy.sh` shell script has minimal error checking and makes several changes to your system.  Be sure that the options you define on the cmdline are correct before proceeding.
+
+To begin the installation with the options defined:
+
+Run the shell script from the :file:`$HOME/telemetrics-backend/scripts` directory:
+
+.. code-block:: bash
+
+   ./deploy.sh -H localhost -a install -d clr
+
+The script will start and list all the defined options and prompt you for the :guilabel:`PostgreSQL` database password as shown below:
+
+.. code-block:: console
+
+    Options:
+    host: localhost
+    distro: clr
+    action: install
+    repo: https://github.com/clearlinux/telemetrics-backend
+    source: master
+    type: git
+    DB password: (default: postgres):
+
+For the :guilabel:`DB password:`, press the :kbd:`Enter` key to accept the default password `postgres`.
+
+The :command:`swupd` begins installing the required software bundles to set up the telemetrics backend server. The output will look similar to the following:
+
+.. code-block:: console
+
+   swupd-client bundle adder 3.12.7
+   Copyright (C) 2012-2017 Intel Corporation
+
+   Downloading packs...
+
+   Extracting application-server pack for version 18740
+         ...5%
+   Extracting database-basic-dev pack for version 18670
+         ...10%
+   Extracting database-basic pack for version 18670
+         ...15%
+   ...
+
+   Extracting c-basic pack for version 18800
+         ...89%
+   Extracting os-core-dev pack for version 18800
+         ...94%
+   Extracting web-server-basic pack for version 18680
+         ...100%
+   Installing bundle(s) files...
+         ...100%
+   Calling post-update helper scripts.
+   Possible filedescriptor leak : 8 (socket:[30833])
+   Bundle(s) installation done.
+
+.. note::
+
+   This script uses :command:`sudo` to run commands and you may be prompted to enter your user password at any time while the script is executing. If this occurs, enter your user password to execute the :command:`sudo` command.
+
+   .. code-block:: console
+
+      Password:
+
+   You may also see an informational message about setting the :envvar:`
+   https_proxy` environment variable if this variable isn't set.
+
+Once the :command:`swupd` command is complete, the script begins processing
+the requirements to install and implement the telemetrics server. Finally,
+the script enables the server and provides output that finishes with
+something similar to:
+
+.. code-block:: console
+
+   .
+   .
+   Successfully built alembic Flask-Migrate itsdangerous Mako MarkupSafe python-editor SQLAlchemy uWSGI WTForms
+   Installing collected packages: SQLAlchemy, MarkupSafe, Mako, python-editor, six, python-dateutil, alembic, click, Werkzeug, Jinja2, itsdangerous, Flask, Flask-SQLAlchemy, Flask-Migrate, WTForms, Flask-WTF, psycopg2, uWSGI
+   Running setup.py install for psycopg2 ... done
+   Successfully installed Flask-0.12.2 Flask-Migrate-2.1.0 Flask-SQLAlchemy-2.2 Flask-WTF-0.14.2 Jinja2-2.9.6 Mako-1.0.7 MarkupSafe-1.0 SQLAlchemy-1.1.13 WTForms-2.1 Werkzeug-0.12.2 alembic-0.9.5 click-6.7 itsdangerous-0.24 psycopg2-2.7.3 python-dateutil-2.6.1 python-editor-1.0.3 six-1.10.0 uWSGI-2.0.15
+
+Once all the server components have been installed you are prompted to enter the :guilabel:`PostgreSQL` database password to change it as illustrated below:
+
+.. code-block:: console
+
+   Enter password for 'postgres' user:
+   New password:
+   Retype new password:
+   passwd: password updated successfully
+
+Enter `postgres` for the current value of the password and then enter a new
+password, retype it to verify the new password and the :guilabel:`PostgreSQL`
+database password will be updated.
+
+The script finalizes installation and finishes.
+
+.. code-block:: console
+
+   Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service → /usr/lib/systemd/system/postgresql.service.
+   Cloning into 'telemetrics-backend'...
+   remote: Counting objects: 344, done.
+   remote: Compressing objects: 100% (53/53), done.
+   remote: Total 344 (delta 30), reused 50 (delta 20), pack-reused 268
+   Receiving objects: 100% (344/344), 130.20 KiB | 1.40 MiB/s, done.
+   Resolving deltas: 100% (177/177), done.
+   ...
+   Already using interpreter /usr/bin/python3
+   Using base prefix '/usr'
+   New python executable in /var/www/telemetry/venv/bin/python3
+   Not overwriting existing python script /var/www/telemetry/venv/bin/python (you must use /var/www/telemetry/venv/bin/python3)
+   Installing setuptools, pip, wheel...done.
+   INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
+   INFO  [alembic.runtime.migration] Will assume transactional DDL.
+   INFO  [alembic.runtime.migration] Running upgrade  -> 3230c615d6e0, empty message
+   INFO  [alembic.runtime.migration] Running upgrade 3230c615d6e0 -> 466cf2f35d67, empty message
+
+   Install complete (installation folder: /var/www/telemetry)
+
+Once the installation is complete you can use your web browser and view the new server by opening the web browser on your system and type in
+``localhost`` in the address bar.
+
+You should see a web page similar to the one shown in figure 1:
+
+  .. figure:: telemetry-backend/figures/telemetry-backend-1.png
+     :alt: Telemetry UI
+     :scale: 50%
+
+     Figure 1: :guilabel:`Telemetry UI`
+
+Redirect telemetry records
+**************************
+
+Telemetry records generated by the telemetrics clients are sent to the
+server location defined in the :file:`/usr/share/defaults/telemetrics/
+telemetrics.conf` configuration file. You can customize this setting by
+copying this file to :file:`/etc/telemetrics/telemetrics.conf` and changing
+the ``server=`` setting to your new server location.
+
+#. Create the :file:`/etc/telemetrics` directory and make it your current
+   working directory.
+
+   .. code-block:: bash
+
+      sudo mkdir -p /etc/telemetrics
+      cd /etc/telemetrics
+
+#. Copy the default :file:`telemetrics.conf` file to the new
+   :file:`/etc/telemetrics` directory.
+
+    .. code-block:: bash
+
+       sudo cp /usr/share/defaults/telemetrics/telemetrics.conf
+
+#. Edit the new :file:`/etc/telemetrics/telemetrics.conf` file with your
+   editor using the :command:`sudo` directive and change the
+   :guilabel:`server=` setting to ``http://localhost/v2/collector`` and save
+   this change in the new file.
+
+   .. code-block:: console
+
+      server=http://localhost/v2/collector
+
+   You can also use the fully qualified domain name for your server instead of :guilabel:`localhost`.
+
+#. Restart the telemetry daemons to reload the configuration file.
+
+   .. code-block:: bash
+
+      telemctl restart
+
+Test the new telemetry backend server
+*************************************
+
+|CL| includes a telemetry test probe called :command:`hprobe` that will send
+a ``hello`` record to the telemetry backend server.  To test that the
+telemetry records are now going to your new destination, run the :command:`
+hprobe` command to send a ``hello`` record to the server as follows:
+
+.. code-block:: bash
+
+   hprobe
+
+The record should show up on your new server console as shown in figure 2:
+
+  .. figure:: telemetry-backend/figures/telemetry-backend-2.png
+     :alt: Telemetry UI
+     :scale: 50%
+
+     Figure 2: :guilabel:`Telemetry UI`
+
+You have now set up the |CL| telemetry backend server, and redirected records from your client to your server.
+
+Creating custom telemetry events
+================================
+For the following steps, we'll be sending records to the backend server we've just set up. If you prefer to keep records locally and not send them to a server, follow the :ref:`telemetrics` guide and enable :record_retention_enabled: in your :file:`etc/telemetrics/telemetrics.conf` to keep the records locally.
+
+There are two ways to create custom telemetry events: using :command:`telem-record-gen` and using the telemetry API in your applications.
+
+Using telem-record-gen
+**********************
+
+Enabling telemetry during installation gives us everything we need to create custom telemetry events, even from C programs, because the telemetry bundle provides a simple pipe-based :abbr:`CLI (Commandline Interface)` program named :file:`telem-record-gen` that can be called trivially:
+
+
+.. code-block:: bash
+
+   ~ $ telem-record-gen --help
+
+.. code-block:: console
+
+   Usage:
+     telem-record-gen [OPTIONS] - create and send a custom telemetry record
+
+   Help Options:
+     -h, --help            Show help options
+
+   Application Options:
+     -f, --config-file     Path to configuration file (not implemented yet)
+     -V, --version         Print the program version
+     -s, --severity        Severity level (1-4) - (default 1)
+     -c, --class           Classification level_1/level_2/level_3
+     -p, --payload         Record body (max size = 8k)
+     -P, --payload-file    File to read payload from
+     -R, --record-version  Version number for format of payload (default 1)
+     -e, --event-id        Event id to use in the record
+
+
+.. note::
+
+   The C library (:file:`libtelemetry.so - man 3 telemetry`) uses the same API parameters and will yield the same effect as :command:`telem-record-gen`.
+
+Let's try generating a simple heartbeat event with
+:command:`telem-record-gen`, similar to the hprobe heartbeat probe that |CL|
+includes by default.
+
+.. code-block:: bash
+
+   ~ $ telem-record-gen -c org.clearlinux/hello/world -p "hello there"
+
+We won't see anything happen on the console, but we can track existing and
+previous telemetry events with :command:`telemctl`:
+
+.. code-block:: bash
+
+   ~$ sudo telemctl journal -V -c org.clearlinux/hello/world -i
+
+.. code-block:: console
+
+   org.clearlinux/hello/world     Tue 2018-11-06 23:00:48 UTC 72e55923fd21c75142c24dcfe0ae0a79 143f2580dcf80267f8f1dfe448f3c975 75f547ff-e55b-44b1-9333-1106098bd448
+   hello there
+
+Using the telemetry API in your C application
+*********************************************
+
+.. note::
+
+   More details about the :ref:`the telemetry API <telemetry-api>` are
+   available in the telemetry guide.
+
+Confirm that the telemetrics header file is located on the system at
+:file:`usr/include/telemetry.h`  The `latest version`_ of the file can also
+be found on github for reference, but installing the `telemetry` bundle will
+install the header file that matches your |CL| version.
+
+You will need to include the following headers in your code to use the API:
+
+::
+
+ #define _GNU_SOURCE
+ #include <stdlib.h>
+ #include <stdio.h>
+ #include <string.h>
+ #include <telemetry.h>
+
+
+Use the following code to create the variables we need to hold the data for the record we will be creating:
+
+::
+
+ uint32_t severity = 1;
+ uint32_t payload_version = 1;
+ char classification[30] = "org.clearlinux/hello/world";
+ struct telem_ref *tm_handle = NULL;
+ char *payload;
+ int ret = 0;
+
+
+
+Severity:
+ | Type: uint32_t
+ | Value:  Severity field value. Accepted values are in the range 1-4, with 1 being the lowest severity, and 4 being the highest severity. Values provided outside of this range are clamped to 1 or 4. [low, med, high, crit]
+
+Payload_version:
+ | Type: uint32_t
+ | Value: Payload format version. The only supported value right now is 1, which indicates that the payload is a freely-formatted (unstructured) string. Values greater than 1 are reserved for future use.
+
+Classification:
+  | Type: char array
+  | Value: It should have the form, DOMAIN/PROBENAME/REST: DOMAIN is the reverse domain to use as a namespace for the probe (e.g. org.clearlinux); PROBENAME is the name of the probe; and REST is an arbitrary value that the probe should use to classify the record. The maximum length for the classification string is 122 bytes. Each sub-category may be no longer than 40 bytes long. Two / delimiters are required.
+
+Tm_handle:
+  | Type: Telem_ref struct pointer
+  | Value:  Struct pointer declared by the caller, The struct is initialized if the function returns success.
+
+Payload:
+  | Type: char pointer
+  | Value: The payload to set
+
+
+
+For this example, we'll set the payload to “hello” by using ``asprintf()``
+
+::
+
+    if (asprintf(&payload, "hello\n") < 0) {
+       exit(EXIT_FAILURE);
+       }
+
+
+
+The functions ``asprintf()`` and ``vasprintf()`` are analogs of ``sprintf(3)`` and    ``vsprintf(3)``, except that they allocate a string large enough to hold the output including the terminating null byte ('\0'), and return a pointer to it via the first argument.  This pointer should be passed to ``free(3)`` to release the allocated storage when it is no longer needed.
+
+
+Create the new telemetry record
+*******************************
+
+The  function  ``tm_create_record()`` initializes a telemetry record and sets the severity and classification of that record, as well as the payload version number. The memory needed to store the telemetry record is allocated and should be freed with ``tm_free_record()`` when no longer needed.
+
+::
+
+ if ((ret = tm_create_record(&tm_handle, severity,  classification, payload_version)) < 0) {
+  printf("Failed to create record: %s\n", strerror(-ret));
+  ret = 1;
+  goto fail;
+  }
+
+
+Set the payload field of a telemetrics record
+*********************************************
+
+The function ``tm_set_payload()`` attaches the provided telemetry record data to the telemetry record. The current maximum payload size is 8192b.
+
+::
+
+  if ((ret = tm_set_payload(tm_handle, payload)) < 0) {
+    printf("Failed to set record payload: %s\n", strerror(-ret));
+    ret = 1;
+    goto fail;
+  }
+  free(payload);
+
+The ``free()`` function frees the memory space pointed to by ptr, which must have been returned by a previous call to ``malloc()``, ``calloc()``, or ``realloc()``.  Otherwise, or if ``free(ptr)`` has already been called before, undefined behavior occurs.  If ptr is NULL, no operation is performed.
+
+Send a record to the telemetrics daemon
+***************************************
+
+The function ``tm_send_record()`` delivers the record to the local
+``telemprobd(1)`` service. Since the telemetry record was allocated by the
+program it should be freed with ``tm_free_record()`` when it is no longer
+needed.
+
+::
+
+  if ((ret = tm_send_record(tm_handle)) < 0) {
+    printf("Failed to send record to daemon: %s\n", strerror(-ret));
+    ret = 1;
+    goto fail;
+  } else {
+    printf("Successfully sent record to daemon.\n");
+    ret = 0;
+  }
+  fail:
+  tm_free_record(tm_handle);
+  tm_handle = NULL;
+
+  return ret;
+
+
+Full sample application with compiling flags
+============================================
+
+Create a new file test.c  add the following code.
+
+::
+
+  #define _GNU_SOURCE
+  #include <stdlib.h>
+  #include <stdio.h>
+  #include <string.h>
+  #include <telemetry.h>
+
+  int main(int argc, char **argv)
+  {
+        uint32_t severity = 1;
+        uint32_t payload_version = 1;
+        char classification[30] = "org.clearlinux/hello/world";
+        struct telem_ref *tm_handle = NULL;
+        char *payload;
+
+        int ret = 0;
+
+        if (asprintf(&payload, "hello\n") < 0) {
+                exit(EXIT_FAILURE);
+        }
+
+        if ((ret = tm_create_record(&tm_handle, severity, classification,
+                                    payload_version)) < 0) {
+                printf("Failed to create record: %s\n", strerror(-ret));
+                ret = 1;
+                goto fail;
+        }
+
+        if ((ret = tm_set_payload(tm_handle, payload)) < 0) {
+                printf("Failed to set record payload: %s\n", strerror(-ret));
+                ret = 1;
+                goto fail;
+        }
+
+        free(payload);
+
+        if ((ret = tm_send_record(tm_handle)) < 0) {
+                printf("Failed to send record to daemon: %s\n", strerror(-ret));
+                ret = 1;
+                goto fail;
+        } else {
+                printf("Successfully sent record to daemon.\n");
+                ret = 0;
+        }
+  fail:
+        tm_free_record(tm_handle);
+        tm_handle = NULL;
+
+        return ret;
+   }
+
+
+
+Compile with the gcc compiler, using this command:
+
+.. code-block:: bash
+
+   gcc test.c -ltelemetry -o test_telem
+
+
+Test to ensure the program is working:
+
+.. code-block:: bash
+
+   ./test_telem
+   Successfully sent record to daemon.
+
+Verify record was received
+*****************************
+
+To verify that the heartbeat message was received by the telemetry backend
+server you can check the telemetry client journal, and specify the
+classification as org.clearlinux/hello/world
+
+.. code-block:: bash
+
+   sudo telemctl journal -V -c org.clearlinux/hello/world -i
+
+.. code-block:: console
+
+   Classification                 Time stamp                  Record ID                        Event ID                         Boot ID
+   org.clearlinux/hello/world     Tue 2018-11-06 22:58:25 UTC b11db07c58c90d8f496ff963df6c43de 24699c2d60c12d154692875b599ca957 75f547ff-e55b-44b1-9333-1106098bd448
+   hello
+   Total records: 1
+
+
+
+A full example of the `heartbeat probe`_ in C is documented in the source code.  For more information about telemetrics in |CL| refer to the
+:ref:`telemetrics` guide.
+
+You can also look for the record on the telemetry backend server.
+
+.. _latest version: https://github.com/clearlinux/telemetrics-client/tree/master/src
+
+.. _heartbeat probe: https://github.com/clearlinux/telemetrics-client/tree/master/src/probes/hello.c
+
+
+.. _clearlinux/telemetrics-backend: https://github.com/clearlinux/telemetrics-backend
@@ -24,6 +24,7 @@ Explore our tutorials to discover what you can do with |CL|!
   hadoop
   fmv
   aws-web/aws-web
+   telemetry-e2e
   telemetry-backend/telemetry-backend
   smb/smb
   spark
@@ -31,3 +32,4 @@ Explore our tutorials to discover what you can do with |CL|!
   kata_migration
   kubernetes
   greengrass
+   dlrs
@@ -7,6 +7,7 @@ Clear Linux - Documentation
   :maxdepth: 1

   clear-linux/clear-linux
+   sitemap
   
 License and disclaimers
 =======================
@@ -0,0 +1,9 @@
+
+py:
+	python bundle_lister.py
+	cp bundles.html.txt ../../clear-linux/reference/bundles
+	rm -rf cloned_repo/*
+	rm bundles.html.txt
+	@echo "Python script finished successfully!"
+
+
@@ -0,0 +1,26 @@
+bundle_lister.py
+----------------
+
+`bundle_lister.py` is a Python (3.6.0) web scraper and html file generator. First, it clones the
+[clr-bundles directory](https://github.com/clearlinux/clr-bundles). Second, it parses the content of all bundles in the clr-bundles/ directory and the `packages` file. Third, it uses Jinja2 template engine to output the result as: bundles.html.txt. This file is copied to reference/bundles location, and it is invoked in `bundles.rst`, which currently appears as [Available bundles](https://clearlinux.org/documentation/clear-linux/reference/bundles).
+
+`bundle_lister.py` automates clear linux documentation so it reflects
+current bundles and packages per developer updates to the
+[clr-bundles GitHub repository](https://github.com/clearlinux/clr-bundles). Therefore, it increases efficiency, automatically aligns documentation with Clear Linux Engineering development, and it eliminates potential for human error, and saves labor hours in contrast to the previous manual method.
+
+`bundle_lister.py` will be invoked in a bash script in the `source/Makefile` of clear-linux documentation. Therefore, `bundle_lister.py` will automatically create newly scraped and parsed data upon each build of the
+[website](https://clearlinux.org) and output an accurate, up-to-date table showing all bundles and packages for interested developers and admins.
+
+See `requirements.txt` for dependencies necessary to run this application.
+
+Built in:`Python==3.6.0`
+
+To run `bundle_lister.py` in the terminal, enter: `python bundle_lister.py`.
+
+Note: The `cloned_repo` directory must remain in the parent directory in order for this code to work; the template.html must remain as
+well.
+
+Note: A successful build will produce a file `bundles.html.txt` showing a table of current bundles and pundles (packages) alphabetized, with a (UTC) time and date stamp in the upper right corner.
+
+
+`~$~`
@@ -0,0 +1,98 @@
+import io
+import os
+import re
+import urllib
+import jinja2
+from jinja2 import Environment, FileSystemLoader, Template
+import git
+from operator import itemgetter
+from datetime import datetime 
+
+GITHUB_BASE = "https://github.com/clearlinux/clr-bundles/tree/master/bundles/"
+PUNDLES = "https://github.com/clearlinux/clr-bundles/blob/master/packages"
+
+PATTERN1 = re.compile(r"#\s?\[TITLE]:\w?(.*)")
+PATTERN2 = re.compile(r"#\s?\[DESCRIPTION]:\w?(.*)")
+PATTERN3 = re.compile(r"\(([^()]*|include)\)", re.MULTILINE)
+PATTERN4 = re.compile(r"^((?:(?!#)\w+[^-\s][-])\w+|\w+[^\s-])", re.MULTILINE)
+# ALT PATTERN4 = re.compile(r"^((?:(?!#)(\w+[^-\s])[-]\w+.)[^\s]{1,}[^\s]|\w+[^\s-])", re.MULTILINE)
+
+PATTERN5 = re.compile(r"^(?!=a)\w.+\s[#]\s(\w+.*)?", re.MULTILINE)
+# Previous version: PATTERN5 = re.compile(r"^[^#].*(?<=\s\-\s)(\w+.*)?", re.MULTILINE)
+
+def extractor(lines):
+    bundle_title = "title"
+    data_desc = "description"
+    url = "url"
+    include_list = []
+
+    for i in lines:
+        title = PATTERN1.match(i)
+        desc = PATTERN2.match(i)
+        includes = PATTERN3.findall(i)
+
+        if title:
+            bundle_title = title.groups(0)[0].strip()
+        if desc:
+            data_desc = desc.groups(0)[0].strip()
+        if url:
+            url = os.path.join(GITHUB_BASE, bundle_title)
+
+        if includes:
+            include_text = includes[0].strip("()")
+            include_list.append(include_text)
+    return {"title": bundle_title, "data_desc": data_desc, "include_list": include_list, "url": url}
+
+def pundler():
+    with io.open("./cloned_repo/clr-bundles/packages") as file_obj:
+        lines = file_obj.readlines()
+        pundle_title = "pundle_title"
+        pundle_desc = "pundle_desc"
+        purl = "purl" 
+        pundle_list = []
+        pun_desc = []
+        pundle_master = []
+        
+        for i in lines:
+            pundle = PATTERN4.findall(i)
+            pundle_plus = PATTERN5.findall(i)
+            
+            if pundle:
+                pundle_title = pundle[0]
+                pundle_list.append(pundle_title)
+
+            if pundle_plus:
+                pundle_desc = pundle_plus[0].strip("[]")
+                pun_desc.append(pundle_desc)
+
+        for pun, desc in zip(pundle_list, pun_desc): 
+                pundle_master.append({"title": pun, "pun_desc": desc, "purl": PUNDLES})
+    return pundle_master
+
+def bundler():
+    data = []
+    try:
+        git.Git("./cloned_repo/").clone("https://github.com/clearlinux/clr-bundles.git")
+    except:
+        pass
+    for root, dirs, files in os.walk("./cloned_repo/clr-bundles/bundles", topdown=False):
+        for name in files:
+            with open(os.path.join(root, name)) as file_obj:
+                lines = file_obj.readlines()
+                data.append(extractor(lines))
+
+    pundle_master = pundler()
+    data = data + pundle_master 
+    filtered = list(filter(lambda x: x.get('title'), data))
+    sortedData = sorted(filtered, key=lambda x:x['title'].lower())
+    #ALT sortedData2 = sorted(sortedData, key=itemgetter('title'))
+    loader = jinja2.FileSystemLoader(searchpath='./')
+    env = jinja2.Environment(loader=loader)
+    template = env.get_template('template.html')
+    template.globals['now'] = datetime.utcnow
+
+    output = template.render(data=sortedData, now=datetime.utcnow())
+    with io.open('bundles.html.txt', 'w') as file:
+        file.write(output)   
+          
+bundler()
@@ -0,0 +1,19 @@
+http://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html
+https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices
+http://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html
+https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices
+https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices
+https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html
+https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html
+https://github.com/clearlinux/common#build-rpms-for-a-package
+https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html
+http://ark.intel.com
+https://clearlinux.org/documentation/clear-linux/concepts/bundles-about#related-concepts
+https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer
+https://github.com/kata-containers/documentation/blob/master/Upgrading.md#maintenance-warning
+https://github.com/clearcontainers/runtime#configuration
+https://github.com/kata-containers/runtime#configuration
+https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3
+https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2
+https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html
+http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html
@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+
+#***********************************************************
+#
+# parse-link-check.py
+# 
+# Arguments:
+#     1. path to input file 
+#
+# External file dependencies:
+#     1. output.txt - the output of sphinx-build
+#     2. link-whitelist.txt - broken links that should be ignored
+#
+# Output:
+#     1. broken_links.html - provides count of broken and whitelist 
+#        matches. Also provides links to all flagged links. Will 
+#        appear in the same directory as output.txt
+#     2. Error code 255 if unexpected broken links are found
+#
+#***********************************************************
+
+
+import sys
+import re
+import os
+
+fileName = "output.txt"
+outFile = "broken_links.html"
+whitelistFile = "link-whitelist.txt"
+
+if len(sys.argv) < 2:
+    print ("Enter path of input directory")
+    sys.exit()
+
+scriptPath = sys.argv[0]
+outputPath = sys.argv[1]
+fileNamePath = outputPath + "/" + fileName
+outFilePath = outputPath + "/" +  outFile
+
+whitelistFilePath = os.path.dirname(scriptPath) + "/" + whitelistFile
+
+with open (whitelistFilePath) as w:
+	whLines = w.readlines()
+
+whitelist = []
+for line in whLines:
+	link = line.rstrip()
+	whitelist.append(link)
+
+with open (fileNamePath) as f:
+    lines = f.readlines()
+
+numBrokenLinks = 0
+numWhiteListMatches = 0
+newLines = ["<!DOCTYPE html><html><head><style>body {font-family: sans-serif;}</style></head><body>"]
+whiteListLines = []
+
+for line in lines:
+    if "[broken]" in line:
+        strings = line.split(" ")
+        link = strings[2][:-1]
+        link = link.strip()
+        if link in whitelist:
+            whiteListLines.append("<b>" + strings[0] + "</b>\n<blockquote><a href=\"" + link + "\">[whitelist] " + link + "</a></blockquote>\n")
+            numWhiteListMatches += 1
+        else:
+            newLines.append("<b>" + strings[0] + "</b>\n<blockquote><a href=\"" + link + "\">[broken] " + link + "</a></blockquote>\n")
+            numBrokenLinks += 1
+
+newLines.insert(0,"<h1>" + str(numBrokenLinks + numWhiteListMatches) + " broken links found in Sphinx link check</h1>\n")
+newLines.insert(1,"<h2>" + str(numBrokenLinks) + " unmatched broken links</h2>\n")
+newLines.append("<h2>" + str(numWhiteListMatches) + " links matched whitelist</h2>\n")
+for line in whiteListLines:
+	newLines.append(line)
+newLines.append("</body></html>")
+
+with open (outFilePath, "w") as outF:
+    for line in newLines:
+        outF.write(line)
+
+print("See ./" + outFilePath + " for a detailed breakdown of broken links.")
+
+if numBrokenLinks != 0:
+    print (numBrokenLinks + " detected. Exiting with error code 255.")
+    sys.exit(-1) 
+else:
+    print ("No unexpected broken links detected.")
@@ -0,0 +1,2 @@
+Jinja2==2.10
+GitPython==2.1.11
@@ -0,0 +1,48 @@
+table {	
+  margin: 32px;	
+  border: 1px solid #e0e0e0;	
+  border-collapse: collapse;	
+  width: auto;    	
+}	
+th {	
+  font-family: IntelClear-Regular,Helvetica,Arial,sans-serif;	
+  align-content: center;	
+  padding: 5px;	
+  border: #ccc solid 1px;	
+  background-color: #555;	
+  color: #fff;	
+  text-transform: uppercase;	
+  font-size: 18px;	
+}	
+tr {	
+  padding-top: 20px ;	
+  padding-bottom: 10px;	
+}	
+ tbody tr:nth-child(odd) {	
+  background-color: #e0e0e0;	
+}	
+ .bundlename {	
+  font-family: IntelClear-Regular,Helvetica,Arial,sans-serif;	
+  font-size: 16px;	
+  font-weight: bolder;	
+  padding-left: 6px;	
+  line-height: 18px;	
+  padding-top:7px ;	
+  padding-bottom: 5px; 	
+}	
+.bundledesc {	
+  font-family: IntelClear-Regular,Helvetica,Arial,sans-serif;	
+  font: italic;	
+  font-size: 16px;	
+  padding-left: 6px;	
+  line-height: 18px;	
+  padding-top: 7px ;	
+  padding-bottom: 5px;	
+}	
+ ul, li {	
+  margin-left: 8px;	
+  /* padding: 0; */	
+  padding-left: 5px;	
+  padding-top: 2px; 	
+  line-height: 16px;	
+}	
@@ -0,0 +1,55 @@
+<!DOCTYPE html>
+
+<html>
+
+<head>
+  <meta charset="UTF-8">
+  <title>Bundles in Clear Linux* OS</title>
+</head>
+
+<body>
+  <table id="bundletable">
+
+    <thead>
+      <tr>
+        <th></th>
+        <th style="text-align:right; font-family:IntelClear-Regular,Helvetica,Arial; font-style:italic">
+          Updated: {{ now.strftime('%x %H:%M') }} UTC
+        </th>
+      </tr>
+      <tr>
+        <th> Name</th>
+        <th> Description</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td></td>
+      </tr>
+      <tr></tr>
+      {% for d in data %}
+      {% if d.url %}
+      <tr id="bundle">
+        <td class="bundlename" id="bundle"><a href="{{d.url}}">{{d.title}}</a></td>
+        <td class="bundledesc">{{d.data_desc}} <br />
+          {% if d.include_list %}
+          <p>Includes bundle(s):
+            {% for include in d.include_list %}
+            <li>{{include}}</li>
+            {% endfor %}
+          </p>
+          {% endif %}
+        </td>
+      </tr>
+      {% else %}
+      <tr id="pundle">
+        <td class="bundlename"><a href="{{d.purl}}">{{d.title}}</a></td>
+        <td class="bundledesc"> {{d.pun_desc}} </td>
+      </tr>
+      {% endif %}
+      {% endfor %}
+    </tbody>
+  </table>
+</body>
+
+</html>
@@ -0,0 +1,13 @@
+.. _clear-linux-sitemap:
+
+Clear Linux - Documentation Sitemap
+###################################
+
+.. toctree::
+   :includehidden:
+
+   clear-linux/concepts/concepts
+   clear-linux/get-started/get-started
+   clear-linux/guides/guides
+   clear-linux/reference/reference
+   clear-linux/tutorials/tutorials