From 603207ea433a97976cd480520885eb6be6a927d9 Mon Sep 17 00:00:00 2001 From: Beth Dean Date: Mon, 13 May 2019 19:31:22 -0600 Subject: [PATCH] Add DARS v1 tutorial (#486) * Add DARS getting started tutorial * Update links to live DARS images * updating link text in bash cmd * Fix formatting & link ref * Corrects sphinx directives; revises intro and other para for readability. Signed-off-by: Michael Vincerra --- source/clear-linux/tutorials/dars.rst | 102 +++++++++++++++++++++ source/clear-linux/tutorials/tutorials.rst | 1 + 2 files changed, 103 insertions(+) create mode 100644 source/clear-linux/tutorials/dars.rst diff --git a/source/clear-linux/tutorials/dars.rst b/source/clear-linux/tutorials/dars.rst new file mode 100644 index 00000000..191dbff6 --- /dev/null +++ b/source/clear-linux/tutorials/dars.rst @@ -0,0 +1,102 @@ +.. _dars: + +Deep Analytics Reference Stack +############################## + +This tutorial shows you how to use the Deep Analytics Reference Stack +(DARS), and to optionally build your own images with the baseline Dockerfiles +provided in the `DARS repository`_. Our tutorial assumes |CL-ATTR| is the +host OS. However, any system that supports Docker containers can be used to +follow these steps. + +.. contents:: + :local: + :depth: 1 + +The Deep Analytics Reference Stack release +****************************************** + +The Deep Analytics Reference Stack provides two prebuilt Docker images, available on `Docker Hub`_: + +* A Clear Linux derived `DARS with OpenBlas`_ stack optimized for `OpenBLAS`_ +* A Clear Linux derived `DARS with MKL`_ stack optimized for `MKL`_ + +You can get the latest component versions for each image in the +:file:`README` found in the `DARS repository`_. Because |CL| is a rolling +distribution, the package version numbers in the |CL| based containers may +not be the latest released by |CL|. + +Using the Docker Images +*********************** + +To immediately start using the latest stable DARS images, pull directly +from `Docker Hub`_. For this tutorial we'll use the `Dars with MKL`_ version of the stack. + +Once you have downloaded the image, you can run it with + +.. code-block:: bash + + docker run -it --ulimit nofile=1000000:1000000 --name mkl + +This will launch the image and drop you into a bash shell inside the +container. You will see output similar to the following: + +.. code-block:: console + + root@fd5155b89857 /root # spark-shell + spark-shell + Config directory: /usr/share/defaults/spark/ + Welcome to + ____ __ + / __/__ ___ _____/ /__ + _\ \/ _ \/ _ `/ __/ '_/ + /___/ .__/\_,_/_/ /_/\_\ version 2.4.0 + /_/ + + Using Scala version 2.12.7 (OpenJDK 64-Bit Server VM, Java 1.8.0-internal) + Type in expressions to have them evaluated. + Type :help for more information. + + scala> + +The :command:`--ulimit nofile` parameter is currently required in order to +increase the number of open files opened at certain point by the spark +engine. + +Building DARS Images +******************** + +If you choose to build your own DARS container images, you can customize +them as needed. Use the provided Dockerfile as a baseline. To construct +images with Clear Linux, start with a Clear Linux development platform that +has the :command:`containers-basic-dev` bundle installed. Learn more about +bundles and installing them by using :ref:`swupd-guide`. + +First, clone the `DARS repository`_ from GitHub. + +.. code-block:: bash + + git clone https://github.com/clearlinux/dockerfiles/tree/master/stacks/dars -b master + +Then, inside the DARS directory, run :command:`make` to build OpenBLAS and +MKL images, and run :command:`make baseline` to build the baseline CentOS +image. Depending on the system, it may take a while to finish building. +Once completed, check the resulting images with :command:`Docker` + +.. code-block:: bash + + docker images | grep dars + +You can use any of the resulting images to launch fully functional +containers. If you need to customize the containers, you can edit the +provided :file:`Dockerfile`. + + +.. _DARS repository: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dars +.. _Docker Hub: https://hub.docker.com/ +.. _OpenBLAS: http://www.openblas.net/ +.. _MKL: https://software.intel.com/en-us/mkl +.. _CentOS: https://www.centos.org/ +.. _DARS with OpenBLAS: https://hub.docker.com/r/clearlinux/stacks-dars-openblas/ +.. _DARS with MKL: https://hub.docker.com/r/clearlinux/stacks-dars-mkl/ +.. _DARS on CentOS: https://hub.docker.com/r/clearlinux....... diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index 80c27277..2e102f10 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -27,3 +27,4 @@ Explore our tutorials to discover what you can do with |CL|! dlrs/dlrs yubikey-u2f nvidia + dars