From ec0b823a9172b099bf910b358937a435e2ec38cb Mon Sep 17 00:00:00 2001 From: michael vincerra Date: Fri, 4 Feb 2022 13:46:27 -0800 Subject: [PATCH] Remove guides/stacks; no longer supported. (#1238) Signed-off-by: michael vincerra --- source/guides/stacks/dars.rst | 703 -------------- source/guides/stacks/dbrs.rst | 714 -------------- source/guides/stacks/dlrs-inference.rst | 1047 -------------------- source/guides/stacks/dlrs.rst | 1173 ----------------------- source/guides/stacks/greengrass.rst | 430 --------- source/guides/stacks/hpcrs.rst | 242 ----- source/guides/stacks/mers.rst | 564 ----------- 7 files changed, 4873 deletions(-) delete mode 100644 source/guides/stacks/dars.rst delete mode 100644 source/guides/stacks/dbrs.rst delete mode 100644 source/guides/stacks/dlrs-inference.rst delete mode 100644 source/guides/stacks/dlrs.rst delete mode 100644 source/guides/stacks/greengrass.rst delete mode 100644 source/guides/stacks/hpcrs.rst delete mode 100644 source/guides/stacks/mers.rst diff --git a/source/guides/stacks/dars.rst b/source/guides/stacks/dars.rst deleted file mode 100644 index 1a1fcbf3..00000000 --- a/source/guides/stacks/dars.rst +++ /dev/null @@ -1,703 +0,0 @@ -.. _dars: - -Data Analytics Reference Stack -############################## - -This guide explains how to use the :abbr:`DARS (Data Analytics Reference Stack)`, and to optionally build your own DARS container image. - -Any system that supports Docker\* containers can be used with DARS. The steps -in this guide use |CL-ATTR| as the host system. - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -The Data Analytics Reference Stack (DARS) provides developers and enterprises a straightforward, highly optimized software stack for storing and processing large amounts of data. More detail is available on the `DARS architecture and performance benchmarks`_. - -Stack Features -============== - -The Data Analytics Reference Stack provides two pre-built Docker images, -available on `Docker Hub`_: - -* A |CL|-derived `DARS with OpenBlas`_ stack optimized for `OpenBLAS`_ -* A |CL|-derived `DARS with Intel® MKL`_ stack optimized for - `Intel® Math Kernel Library`_ (Intel® MKL) - -We recommend you view the latest component versions for each image in the -:file:`releasenote` found in the `Data Analytics Reference Stack`_ GitHub\* -repository. Because |CL| is a rolling distribution, the package version numbers -in the |CL|-based containers may not be the latest released by |CL|. - -.. note:: - - The Data Analytics Reference Stack is a collective work, and each piece - of software within the work has its own license. Please see the - `DARS Terms of Use`_ for more details about licensing and usage of the Data - Analytics Reference Stack. - -Using the Docker images -*********************** - -Launching the Image -=================== - -#. To use the latest stable DARS images, pull an image - directly from `Docker Hub`_. This example uses the - `DARS with Intel® MKL`_ Docker image. - - .. code-block:: bash - - docker pull clearlinux/stacks-dars-mkl - - -#. Once you have downloaded the image, you can run it with this command, which will launch the image and drop you into a bash shell inside the container. - - .. code-block:: bash - - docker run -it --ulimit nofile=1000000:1000000 --name mkl --network host --rm -i -t - - - Command Flags - - :command:`--ulimit nofile=` is required in order to increase the allowed number of open files for the Apache Spark\* engine. - - :command:`--name` can be any name of your choice. This guide is using `mkl` - - :command:`--network host` enables the host machine's IP address to be used to access the container. - - If you need to verify the name of the DARS image for the flag, you can use the :command:`docker image ls` command to see which images reside on your system. - - .. code-block:: bash - - docker image ls - - - .. code-block:: console - - REPOSITORY TAG IMAGE ID CREATED SIZE - clearlinux/stacks-dars-mkl test-img 49a70a22231f 23 hours ago 2.66GB - ubuntu latest 2ca708c1c9cc 7 days ago 64.2MB - katadocker/kata-deploy latest bd6dc92f8060 7 days ago 673MB - clearlinux/stacks-dars-mkl latest 2c9555536d5f 4 weeks ago 2.62GB - - - - -.. note:: - - All of the DARS components are compiled on Open JDK11\*. The container will have preinstalled JDK11 at :file:`/usr/lib/jvm/java-1.11.0-openjdk/` and it has been set as the default Java version. While the DARS containers also contain Open JDK8, it is not covered in this guide. - - -Building DARS images -==================== - -If you choose to build your own DARS container images, you can customize them as needed. Use the :file:`Dockerfile` included in the Github\* repository as your baseline. - -To construct images with |CL|, start with a |CL| development platform that has the :command:`containers-basic-dev` bundle installed. Learn more about bundles and installing them by using :ref:`swupd-guide`. - -#. The `Data Analytics Reference Stack`_ is part of the Intel stacks GitHub\* repository. Clone the :file:`stacks` repository. - - .. code-block:: bash - - git clone https://github.com/intel/stacks.git - -#. Inside the :file:`stacks/dars/clearlinux/mkl` directory, use docker with the :file:`Dockerfile` to build the MKL image. - - .. code-block:: bash - - cd ./stacks/dars/clearlinux/mkl - docker build --no-cache -t clearlinux/stacks-dars-mkl . - - -#. Once completed, check the resulting images with :command:`Docker` - - .. code-block:: bash - - docker images | grep dars - -#. You can use any of the resulting images to launch fully functional containers. If you need to customize the containers, you can edit the provided :file:`Dockerfile`. - -.. note:: - - The environment variables for Apache Hadoop* and Apache Spark have been configured in the Dockerfile for the DARS container. For Apache Hadoop\* use :file:`/etc/hadoop` as `HADOOP_CONF_DIR` folder. For Apache Spark use :file:`/etc/spark` as `SPARK_CONF_DIR` folder. - - -Using Apache Spark\* in DARS -**************************** - -After launching the container, you can start Apache Spark with either the Scala or PySpark environment. For these examples we will use PySpark, which is the Python\* API for Apache Spark. - -.. code-block:: bash - - pyspark - - -Launching is as simple as this. Depending on your system configuration and capabilities, you may need to define proxy or memory allocation settings on the command line or in a config file for optimal performance. Refer to the `Apache Spark documentation`_ for more detail. - -After executing :command:`pyspark`, you will see output similar to this. - -.. code-block:: console - - root@fd5155b89857 /root # pyspark - Welcome to - ____ __ - / __/__ ___ _____/ /__ - _\ \/ _ \/ _ `/ __/ '_/ - /__ / .__/\_,_/_/ /_/\_\ version 2.4.0 - /_/ - - Using Python version 3.7.4 (default, Jul 13 2019 06:59:17) - SparkSession available as 'spark'. - >>> - - -Execute code directly in PySpark -================================ - -A simple example for verifying that pyspark is working correctly is to run a small python function from a `PySpark getting started guide`_ to estimate the value of Pi. Run these lines in the PySpark shell. - -.. code-block:: console - - import random - NUM_SAMPLES = 100000000 - def inside(p): - x, y = random.random(), random.random() - return x*x + y*y < 1 - - count = sc.parallelize(range(0, NUM_SAMPLES)).filter(inside).count() - pi = 4 * count / NUM_SAMPLES - print(“Pi is roughly”, pi) - - -Run Python programs with spark-submit -===================================== - -You can also run python scripts in Apache Spark from the command line. We'll use the Apache Spark example found in the :file:`/usr/share/apache-spark/examples/src/main/python/pi.py` file. Note that we have turned off the INFO and WARN messages in Apache Spark for this example. - -.. code-block:: console - - #spark-submit /usr/share/apache-spark/examples/src/main/python/pi.py - Config directory: /usr/share/defaults/spark/ - Pi is roughly 3.134700 - -DARS Usecase example -==================== - -The DARS container is used in conjunction with the Deep Learning Reference Stack container to implement a real world use case. Refer to the `Github Issue Classification`_ Usecase found in the `stacks-usecase`_ repository for a walkthrough. This usecase is implemented using the Scala environment, rather than PySpark. - - -Using Apache Hadoop in DARS -*************************** - -Apache Hadoop is an open source framework allowing for distributed processing of large data sets across clusters of computers using simple programming models. This framework is designed to scale up from a few servers to thousands of machines, each offering local computation and storage. - -Single Node Hadoop Cluster Setup -================================ - -In this mode, all the daemons involved (e.g., the DataNode, NameNode, TaskTracker, JobTracker) run as Java processes on the same machine. This setup is useful for developing and testing Apache Hadoop applications. - -The components of an Apache Hadoop Cluster are described below: - -* NameNode manages HDFS storage. HDFS exposes a filesystem namespace and allows user data to be stored in files. Internally a file is split into one or more blocks and these blocks are stored in a set of DataNodes. -* DataNode is also known as Slave node. It is responsible for storing and managing the data in that node and responds to the NameNode for all filesystem operations. -* JobTracker is a master which creates and runs the job through tasktrackers. It also tracks resource availability and task lifecycle management. -* TaskTracker manages the processing resources on each worker node and send status updates to the JobTracker periodically. - - -Configuration -============= - -#. To setup a single node cluster, run a DARS container with the following flags: - - .. code-block:: bash - - docker run --ulimit nofile=1000000:1000000 -ti --rm --network host clearlinux/stacks-dars-mkl cp -r -n /usr/share/defaults/hadoop/* /etc/hadoop - -#. In the running container, set configuration in the :file:`/etc/hadoop/mapred-site.xml` file - - .. code-block:: xml - - - - mapreduce.framework.name - yarn - - - - yarn.app.mapreduce.am.env - HADOOP_MAPRED_HOME=${HADOOP_HOME} - - - - mapreduce.map.env - HADOOP_MAPRED_HOME=${HADOOP_HOME} - - - - mapreduce.reduce.env - HADOOP_MAPRED_HOME=${HADOOP_HOME} - - - -#. Set up the :file:`/etc/hadoop/yarn-site.xml` as follows - - .. code-block:: xml - - - - yarn.nodemanager.aux-services - mapreduce_shuffle - - - - yarn.nodemanager.auxservices.mapreduce.shuffle.class - org.apache.hadoop.mapred.ShuffleHandler - - - -Start the Apache Hadoop daemons -=============================== - -#. Format the NameNode server using this command: - - .. code-block:: bash - - hdfs namenode -format - -#. Start the Apache Hadoop services - - HDFS Namenode service : - - .. code-block:: bash - - hdfs --daemon start namenode - - - HDFS Datanode service : - - .. code-block:: bash - - hdfs --daemon start datanode - - - Yarn ResourceManager : - - .. code-block:: bash - - yarn --daemon start resourcemanager - - - Yarn NodeManager : - - .. code-block:: bash - - yarn --daemon start nodemanager - - - jobhistory service : - - .. code-block:: bash - - mapred --daemon start historyserver - -#. Verify the nodes are alive with this command: - - - .. code-block:: bash - - yarn node -list 2 - - Your output will look similar to: - - .. code-block:: console - - Total Nodes:1 - Node-Id Node-State Node-Http-Address Number-of-Running-Containers - :43489 RUNNING :8042 0 - - -Example application -=================== - -Apache Hadoop comes packages with a set of example applications. In this example we will show how to use the cluster to calculate Pi. The JAR file containing the compiled class can be found on your running DARS container at :file:`/usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar` - - -.. code-block:: bash - - hadoop jar /usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-$(hadoop version | grep Hadoop | cut -d ' ' -f2).jar pi 16 100 - - -Deploy DARS on Kubernetes\* -*************************** - -Many containerized workloads are deployed in clusters managed by orchestration software like Kubernetes. - -Prerequisites -============= - -* A running Kubernetes cluster at version >= 1.6 with access configured to it using kubectl. -* You must have appropriate permissions to list, create, edit and delete pods in your cluster. -* The service account credentials used by the driver pods must be allowed to create pods, services and configmaps. -* You must have Kubernetes DNS configured in your cluster. - -.. note:: - - To ensure that Kubernetes is correctly installed and configured for |CL|, follow the instructions in :ref:`kubernetes`. - - -#. For this example we will create the following Dockerfile - - .. code-block:: bash - - cat > $(pwd)/Dockerfile << 'EOF' - ARG DERIVED_IMAGE - FROM ${DERIVED_IMAGE} - - RUN mkdir -p /etc/passwd /etc/pam.d /opt/spark/conf /opt/spark/work-dir - - RUN set -ex && \ - rm /bin/sh && \ - ln -sv /bin/bash /bin/sh && \ - touch /etc/pam.d/su \ - echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \ - chgrp root /etc/passwd && chmod ug+rw /etc/passwd - - RUN ln -s /usr/share/apache-spark/jars/ /opt/spark/ && \ - ln -s /usr/share/apache-spark/bin/ /opt/spark/ && \ - ln -s /usr/share/apache-spark/sbin/ /opt/spark/ && \ - ln -s /usr/share/apache-spark/examples/ /opt/spark/ && \ - ln -s /usr/share/apache-spark/kubernetes/tests/ /opt/spark/ && \ - ln -s /usr/share/apache-spark/data/ /opt/spark/ && \ - ln -s /etc/spark/* /opt/spark/conf/ - - COPY entrypoint.sh /opt/ - ENV JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk - ENV PATH="${JAVA_HOME}/bin:${PATH}" - ENV SPARK_HOME /opt/spark - WORKDIR /opt/spark/work-dir - ENTRYPOINT [ "/opt/entrypoint.sh" ] - EOF - - -#. Create the :file:`entrypoint.sh` file. The Dockerfile requires an entrypoint script, to allow spark-submit to interact with the container. - - .. code-block:: bash - - cat > $(pwd)/entrypoint.sh << 'EOF' - #!/bin/bash - # - # Licensed to the Apache Software Foundation (ASF) under one or more - # contributor license agreements. See the NOTICE file distributed with - # this work for additional information regarding copyright ownership. - # The ASF licenses this file to You under the Apache License, Version 2.0 - # (the "License"); you may not use this file except in compliance with - # the License. You may obtain a copy of the License at - # - # http://www.apache.org/licenses/LICENSE-2.0 - # - # Unless required by applicable law or agreed to in writing, software - # distributed under the License is distributed on an "AS IS" BASIS, - # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - # See the License for the specific language governing permissions and - # limitations under the License. - # - - # echo commands to the terminal output - set -ex - - # Check whether there is a passwd entry for the container UID - myuid=$(id -u) - mygid=$(id -g) - # turn off -e for getent because it will return error code in anonymous uid case - set +e - uidentry=$(getent passwd $myuid) - set -e - - # If there is no passwd entry for the container UID, attempt to create one - if [ -z "$uidentry" ] ; then - if [ -w /etc/passwd ] ; then - echo "$myuid:x:$myuid:$mygid:anonymous uid:$SPARK_HOME:/bin/false" >> /etc/passwd - else - echo "Container ENTRYPOINT failed to add passwd entry for anonymous UID" - fi - fi - - SPARK_K8S_CMD="$1" - case "$SPARK_K8S_CMD" in - driver | driver-py | driver-r | executor) - shift 1 - ;; - "") - ;; - *) - echo "Non-spark-on-k8s command provided, proceeding in pass-through mode..." - exec /sbin/tini -s -- "$@" - ;; - esac - - SPARK_CLASSPATH="$SPARK_CLASSPATH:${SPARK_HOME}/jars/*" - env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt - readarray -t SPARK_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt - - if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then - SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH" - fi - - if [ -n "$PYSPARK_FILES" ]; then - PYTHONPATH="$PYTHONPATH:$PYSPARK_FILES" - fi - - PYSPARK_ARGS="" - if [ -n "$PYSPARK_APP_ARGS" ]; then - PYSPARK_ARGS="$PYSPARK_APP_ARGS" - fi - - R_ARGS="" - if [ -n "$R_APP_ARGS" ]; then - R_ARGS="$R_APP_ARGS" - fi - - if [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "2" ]; then - pyv="$(python -V 2>&1)" - export PYTHON_VERSION="${pyv:7}" - export PYSPARK_PYTHON="python" - export PYSPARK_DRIVER_PYTHON="python" - elif [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "3" ]; then - pyv3="$(python3 -V 2>&1)" - export PYTHON_VERSION="${pyv3:7}" - export PYSPARK_PYTHON="python3" - export PYSPARK_DRIVER_PYTHON="python3" - fi - - case "$SPARK_K8S_CMD" in - driver) - CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" - ) - ;; - driver-py) - CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" $PYSPARK_PRIMARY $PYSPARK_ARGS - ) - ;; - driver-r) - CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" $R_PRIMARY $R_ARGS - ) - ;; - executor) - CMD=( - ${JAVA_HOME}/bin/java - "${SPARK_EXECUTOR_JAVA_OPTS[@]}" - -Xms$SPARK_EXECUTOR_MEMORY - -Xmx$SPARK_EXECUTOR_MEMORY - -cp "$SPARK_CLASSPATH" - org.apache.spark.executor.CoarseGrainedExecutorBackend - --driver-url $SPARK_DRIVER_URL - --executor-id $SPARK_EXECUTOR_ID - --cores $SPARK_EXECUTOR_CORES - --app-id $SPARK_APPLICATION_ID - --hostname $SPARK_EXECUTOR_POD_IP - ) - ;; - - *) - echo "Unknown command: $SPARK_K8S_CMD" 1>&2 - exit 1 - esac - - # Execute the container CMD - exec "${CMD[@]}" - EOF - - -#. Make :file:`entrypoint.sh` executable - - .. code-block:: bash - - sudo chmod +x $(pwd)/entrypoint.sh - -#. Build the Docker image, for this example we will use dars_k8s_spark for the name of the image. - - .. code-block:: bash - - docker build . --build-arg DERIVED_IMAGE=clearlinux/stacks-dars-mkl -t dars_k8s_spark - - -#. Verify your built image. Execute the following command looking for the given name dars_k8s_spark - - .. code-block:: bash - - docker images | grep "dars_k8s_spark" - - You should see something like: - - .. code-block:: console - - dars_k8s_spark latest 1fa3278a3421 1 minutes ago 6.56GB - -#. Use a variable to store the image's given name: - - .. code-block:: bash - - DARS_K8S_IMAGE=dars_k8s_spark - - -Configure RBAC -============== - -Create the Spark service account and cluster role binding to allow Spark on Kubernetes to create Executors as required. For this example use the default namespace. - -.. code-block:: bash - - kubectl create serviceaccount spark-serviceaccount --namespace default - kubectl create clusterrolebinding spark-rolebinding --clusterrole=edit --serviceaccount=default:spark-serviceaccount --namespace=default - - -Prepare to Submit the Spark Job -=============================== - -#. Determine the Kubernetes master address: - - .. code-block:: bash - - kubectl cluster-info - - You should see something like: - - .. code-block:: console - - Kubernetes master is running at https://192.168.39.127:8443 - -#. Use a variable to store the master address: - - .. code-block:: bash - - MASTER_ADDRESS='https://192.168.39.127:8443' - -#. Submit the Spark Job on Minikube using the MASTER_ADDRESS and DARS_K8S variables. The driver pod will be called spark-pi-driver. - - .. code-block:: bash - - spark-submit \ - --master k8s://${MASTER_ADDRESS} \ - --deploy-mode cluster \ - --name spark-pi \ - --class org.apache.spark.examples.SparkPi \ - --conf spark.executor.instances=2 \ - --conf spark.kubernetes.container.image=${DARS_K8S_IMAGE} \ - --conf spark.kubernetes.driver.pod.name=spark-pi-driver \ - --conf spark.kubernetes.namespace=default \ - --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark-serviceaccount \ - local:///usr/share/apache-spark/examples/jars/spark-examples_2.12-2.4.0.jar - - -#. Check the Job. Read the logs and look for the Pi result: - - .. code-block:: bash - - kubectl logs spark-pi-driver | grep "Pi is roughly" - - You should see something like: - - .. code-block:: console - - Pi is roughly 3.1418957094785473 - -More information about spark-submit configuration is available in the `running-on-kubernetes`_ documentation. - - -Troubleshooting -*************** - -Dropped or refused connection -============================= - -If Pyspark / Spark-shell warns of a dropped connection exception or Connection refused, check if the `HADOOP_CONF_DIR` environment variable is set. These APIs assume they will use Hadoop Distributed File System. -You can unset `HADOOP_CONF_DIR` and use Spark RDDs, or start Hadoop services and then create your directories and files as required using hdfs. - -It is also possible to change the file system to local without unsetting `HADOOP_CONF_DIR` using one of these commands. - -.. code-block:: bash - - pyspark --conf "spark.hadoop.fs.defaultFS=file:///" - -.. code-block:: bash - - spark-shell --conf "spark.hadoop.fs.defaultFS=file:///" - -Using Spark with proxy settings -=============================== - -There are two ways to work with proxies: - -#. Add the following line to :file:`$SPARK_CONF_DIR/spark-defaults.conf` for both `spark.executor.extraJavaOptions` and `spark.driver.extraJavaOptions` variables: - -.. code-block:: console - - -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= - - - -#. Give the proxies URL and Port as a configuration parameter - -.. code-block:: bash - - pyspark --conf "spark.hadoop.fs.defaultFS=file:///" --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=example.proxy -Dhttp.proxyPort=111 -Dhttps.proxyHost=example.proxy -Dhttps.proxyPort=112" - -.. code-block:: bash - - spark-shell --conf "spark.hadoop.fs.defaultFS=file:///" --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=example.proxy -Dhttp.proxyPort=111 -Dhttps.proxyHost=example.proxy -Dhttps.proxyPort=112" - - -Known issues -============ - -#. There is an exception message `Unrecognized Hadoop major version number: 3.2.0 at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion.` - -This exception can be disregarded because DARS does not use hadoop.hive.shims. Hive binaries installed from Apache on |CL| with JDK11 does not work at the time of this writing. - -#. There is an exception message `Exception in thread "Thread-3" java.lang.ExceptionInInitializerError at org.apache.hadoop.hive.conf.HiveConf` This is related to the same issue with |CL| and JDK11 noted above, and does not affect DARS for the same reason. - -*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* - -.. _Data Analytics Reference Stack: https://github.com/intel/stacks/tree/master/dars/clearlinux - -.. _Docker Hub: https://hub.docker.com/ - -.. _OpenBLAS: http://www.openblas.net/ - -.. _Intel® Math Kernel Library: https://software.intel.com/en-us/mkl - -.. _CentOS: https://www.centos.org/ - -.. _DARS with OpenBLAS: https://hub.docker.com/r/clearlinux/stacks-dars-openblas/ - -.. _DARS with Intel® MKL: https://hub.docker.com/r/clearlinux/stacks-dars-mkl/ - -.. _DARS architecture and performance benchmarks: https://clearlinux.org/stacks/data-analytics-stack-v1 - -.. _DARS Terms of Use: https://clearlinux.org/stacks/data-analytics/terms-of-use - -.. _PySpark getting started guide: https://towardsdatascience.com/how-to-get-started-with-pyspark-1adc142456ec - -.. _Apache Spark documentation: https://spark.apache.org/docs/latest/ - -.. _stacks-usecase: https://github.com/intel/stacks-usecase - -.. _Github Issue Classification: https://github.com/intel/stacks-usecase/tree/master/github-issue-classification - -.. _running-on-kubernetes: https://spark.apache.org/docs/latest/running-on-kubernetes.html#configuration diff --git a/source/guides/stacks/dbrs.rst b/source/guides/stacks/dbrs.rst deleted file mode 100644 index ac3c5f90..00000000 --- a/source/guides/stacks/dbrs.rst +++ /dev/null @@ -1,714 +0,0 @@ -.. _dbrs: - -Database Reference Stack -######################## - -This guide describes the hardware and installation requirements for using the -:abbr:`DBRS (Database Reference Stack)`, along with getting started configuration examples, using |CL-ATTR| as the host system. - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -The Database Reference Stack is integrated, highly-performant, open source, -and optimized for 2nd generation Intel® Xeon® Scalable processors and Intel® -Optane™ persistent memory. This open source community release is part of -an effort to ensure developers have easy access to the features and -functionality of Intel Platforms. - -Stack Features -============== - -Current supported database applications are Apache Cassandra* and Redis*, which -have been enabled for `Intel Optane PMM`_. - -DBRS with Apache Cassandra can be deployed as a standalone container or inside a -Kubernetes* cluster. - -The Redis stack application is enabled for a multinode Kubernetes -environment, using AEP PMem DIMM in fsdax mode for storage. - -Releases -******** - -Refer to the `Database Reference Stack website`_ for information and download links for the different versions and offerings of the stack. - -The release announcement for each release provides more detail about the stack features, as well as benchmark results. - -* `DBRS V2.0`_ release announcement. -* `DBRS V1.0`_ release announcement. - - -.. note:: - - The Database Reference Stack is a collective work, and each piece - of software within the work has its own license. Please see the - `DBRS Terms of Use`_ for more details about licensing and usage of the Database Reference Stack. - - -Hardware Requirements -********************* - -* Intel Xeon Scalable platform with Intel® C620 series chipset -* 2nd Gen Intel Xeon Scalable processor CPU (Intel® Optane™ PMem-enabled stepping). Provides cache & memory control. Intel Optane PMem works only on systems powered by 2nd Generation Intel® Xeon® Platinum or Intel® Xeon® Gold processors. -* BIOS with Reference Code -* Intel Optane PMem - -Hardware configuration used in stacks development -================================================= - -* Intel® Server System R2208WFTZSR -* BIOS with Reference Code - * BIOS ID: SE5C620.86B.0D.01.0438.032620191658 - * BMC Firmware: 1.94.6b42b91d - * Intel Optane PMem Firmware: 1.2.0.5310 -* 2x Intel Xeon Platinum 8268 Processor -* Intel® SSD Data Center Family S5600 Series 960GB 2.5in SATA Drive -* 64 GB RAM - Distributed in 4x 16 GB DDR4 DIMM's -* 2x Intel Optane PMem 256GB Module -* 1-1-1 Layout 8 Intel Optane : 1 RAM ratio - - -.. list-table:: **Table 1. IMC** - :widths: 16,16,16,16,16,16 - :header-rows: 1 - - * - Channel 2 - - Channel 2 - - Channel 1 - - Channel 1 - - Channel 0 - - Channel 0 - - * - Slot 1 - - Slot 0 - - Slot 1 - - Slot 0 - - Slot 1 - - Slot 0 - - * - - - 256 GB DCPMM - - - - 16 GB DRAM - - - - 16 GB DRAM - -Firmware configuration -********************** - -.. important:: - - When updating DCPMM Firmware, all DCPMM parts must be in the same mode (you cannot mix 1LM and 2LM parts). - -The latest firmware download for the Intel® Server Board S2600WF Family is available at the `Intel Download Center`_ - -Firmware Update Steps -===================== - -#. Unzip the contents of the update package and copy all files to the root directory of a removable media (USB flash drive). -#. Insert the USB flash drive to any available USB port on the system to be updated. -#. Boot to EFI shell. -#. Input "fsx(x:0,1,...):" to enter into your usb disk -#. Run "startup.nsh" -#. After update BMC firmware, system BIOS, ME firmware,FD, FRUSDR, system will reboot automatically. - - -If Intel Optane PMem is installed, run startup.nsh a second time after the first reboot to upgrade Intel Optane PMem Firmware: - -* Boot to EFI shell. -* Input "fsx(x:0,1,...):" to enter into your usb disk -* Run "startup.nsh" again to update the corresponding AEP FW. - -.. _dbrs-hardware-configuration: - -Hardware Configuration -********************** - - -Online Resources -================ - -Before going through the configuration steps, we strongly recommend visiting the following resources and wikis to have a broader understanding of what is being done - -* `Quick Start Guide`_ Configure Intel Optane PMem Modules on Linux -* `Managing NVDIMMs`_ -* `Configure, Manage, and Profile`_ Intel Optane PMem Modules - -Optane™ DIMM Configuration -========================== - -The PMem DIMMs can be configured in devdax or fsdax mode. The use case to enable database stack on a kubernetes environment currently only support fsdax mode. - -Configuration Steps -=================== - -.. important:: - - Run the following steps with root privileges (sudo) as shown in the examples - - -#. To configure Optane™ DIMMs for App direct mode run this command - - .. code-block:: bash - - sudo ipmctl create -goal PersistentMemoryType=AppDirect - -#. Verify the Optane™ Configuration by showing the defined region, then reboot the system for your changes to take effect - - .. code-block:: bash - - sudo ipmctl show -region - - -#. Next, list the defined namespaces for the pmem devices in the system. If they are not defined, create them as shown in the following step. - - - .. code-block:: bash - - sudo ndctl list -N - - -#. Create namespaces based on the regions and set mode as fsdax -- use the names of the regions listed in previous step as the –-region parameter (default is region0 and region1; one for each CPU socket) - - .. code-block:: bash - - sudo ndctl create-namespace --region=region0 --mode=fsdax - sudo ndctl create-namespace --region=region1 --mode=fsdax - - -#. Create the filesystem and mount it. We are using /mnt/dax{#} as a convention in this guide to mount our devices - - .. code-block:: bash - - sudo mkfs.ext4 /dev/pmem0 - sudo mount -o dax /dev/pmem0 /mnt/dax0 - sudo mkfs.ext4 /dev/pmem1 - sudo mount -o dax /dev/pmem1 /mnt/dax1 - - -Running DBRS with Apache Cassandra* -*********************************** - -DBRS with Apache Cassandra can be deployed as a standalone container or inside -Kubernetes\*. Instructions for both cases is included here. Note that you can -use the released `Docker image with Apache Cassandra`_ (Docker\* examples below). -These instructions provide a baseline for creating your own container image. -If you are using the released image, skip this section. - -.. important:: - - At the initial release of DBRS, Apache Cassandra is considered to be Engineering Preview release quality and may not be suitable for production release. Please take this into consideration when planning your project. - - - -Build the DBRS with Apache Cassandra container -============================================== - -To build the container with Apache Cassandra, you must build cassandra-pmem, and then build the container using the :command:`docker build` command. We are using |CL| as our container host as well as the OS in the container. - -Build cassandra-pmem -==================== - -.. important:: - - At the initial release of DBRS, the pmem-csi driver is considered to be Engineering Preview release quality and may not be suitable for production release. Please take this into consideration when planning your project. - - -In the `DBRS github repository`_, there is a file called `build-cassandra-pmem.sh`_, which handles all the requirements for compiling cassandra-pmem for Dockerfile usage. The dependencies for this build can be installed with :command:`swupd`. - -.. code-block:: bash - - sudo swupd bundle-add c-basic java-basic devpkg-pmdk pmdk - - -Once installed, we run the script - -.. code-block:: bash - - ./build-cassandra-pmem.sh - - -At the completion of the build you will have a file called :file:`cassandra-pmem-build.tar.gz`. Place this file in the same directory with the Dockerfile to build the Docker image. - -Build the Docker container -========================== - -To build the Docker image, run the Dockerfile in the same directory with the :file:`cassandra-pmem-build.tar.gz` - -.. code-block:: bash - - docker build --force-rm --no-cache -f Dockerfile -t $build_image_name . - - -Once it completes, the Docker image is ready to be used. - -Deploy Apache Cassandra PMEM as a standalone container -====================================================== - -Requirements ------------- - -To deploy Apache Cassandra PMEM, you must meet the following requirements - -* PMEM memory must be configured in `devdax` or `fsdax` mode. The container image is able to handle both modes and depending on the PMEM mode, the mount points inside the container must be different. -* In order to make available `devdax` pmem devices inside the container you must use the `--device` directive. Internally the container always uses :command:`/dev/dax0.0`, so the mapping should be: :command:`--device=/dev/:/dev/dax0.0` -* In a similar fashion for `fsdax` we need the device to be mapped to :command:`/mnt/pmem` inside the container: :command:`--mount type=bind,source=,target=/mnt/pmem` - - -Preparing PMEM for container use --------------------------------- - -The cassandra-pmem image is capable of using both `fsdax` and `devdax`, the necessary steps to configure the PMEM to work with cassandra are documented here. - -.. tabs:: - - .. group-tab:: devdax - - We need to verify the device we want to use is in `devdax` mode - - .. code-block:: bash - - sudo ndctl create-namespace -fe namespace0.0 --mode=devdax - - .. code-block:: console - - { - "dev":"namespace0.0", - "mode":"devdax", - "map":"dev", - "size":"3.94 GiB (4.23 GB)", - "uuid":"cb738cc7-711d-4578-bebf-1f7ba02ca169", - "daxregion":{ - "id":0, - "size":"3.94 GiB (4.23 GB)", - "align":2097152, - "devices":[ - { - "chardev":"dax0.0", - "size":"3.94 GiB (4.23 GB)" - } - ] - }, - "align":2097152 - } - - - If needed, we can reconfigure it using :command:`ndctl create-namespace -fe --mode=devdax`. - - Before using a `devdax` device we need to clear the device: - - .. code-block:: bash - - sudo pmempool rm -vaf /dev/dax0.0 - - - The `jvm.options` configuration for Apache Cassandra should look like the following: - - .. code-block:: console - - -Dpmem_path=/dev/dax0.0 - -Dpool_size=0 - - Where - * pmem_path is the `devdax` device. - * pool_size=0 indicates to use the entire `devdax` device. - - When using the `Docker image with Apache Cassandra`_, the file `jvm.options` is automatically populated. - - .. group-tab:: fsdax - - Verify that the PMEM is in `fsdax` mode - - .. code-block:: bash - - sudo ndctl list -u - - .. code-block:: console - - { - "dev":"namespace0.0", - "mode":"fsdax", - "map":"mem", - "size":"4.00 GiB (4.29 GB)", - "sector_size":512, - "blockdev":"pmem0" - } - - - If for some reason the device is not in `fsdax` mode you can reconfigure the namespace as follows: - - .. code-block:: bash - - sudo `ndctl create-namespace -fe --mode=fsdax` - - - Once the PMEM namespace is configured, you will see a device named :file:`/dev/pmem{0-9}`. We will create a filesystem on that device. The filesystem could be `ext4` or `xfs`, for this example we are going to use `ext4`. - - .. code-block:: bash - - sudo mkfs.ext4 /dev/pmem0 - - .. code-block:: console - - mke2fs 1.45.2 (27-May-2019) - Creating filesystem with 1031680 4k blocks and 258048 inodes - Filesystem UUID: 303c03f5-ac4e-4462-8bf9-bc6b0fae53fe - Superblock backups stored on blocks: - 32768, 98304, 163840, 229376, 294912, 819200, 884736 - - Allocating group tables: done - Writing inode tables: done - Creating journal (16384 blocks): done - Writing superblocks and filesystem accounting information: done - - - Once the filesystem is created, we mount it with the dax option - - .. code-block:: bash - - sudo mount /dev/pmem0 /mnt/pmem -o dax - - - When using `fsdax` mode cassandra-pmem creates a pool file on the pmem mountpoint, so the `jvm.options` configuration should look like the output below: - - .. code-block:: console - - -Dpmem_path=/mnt/pmem/cassandra_pool - -Dpool_size=3221225472 - - - - Where - * `pmem_path` is the path to the pool file, which should include the path itself and the file name - * `pool_size` is the size of the pool file in bytes. If you are using the `Docker image with Apache Cassandra`_ you can pass this value as an environment variable to the container runtime in Gb and the calculation is done automatically. - - Is important to note that when creating the filesystem in the pmem device certain amount of space of the device is used by the filesystem metadata so the pool_size should be smaller than the total pmem namespace size. - - When using the `Docker image with Apache Cassandra`_, the file `jvm.options` is automatically populated with the environment variables `CASSANDRA_PMEM_POOL_NAME` and `CASSANDRA_FSDAX_POOL_SIZE_GB`. - - - -Run the DBRS Container -====================== - -Replace `` in the following commands with the name of the image you are using. - -.. tabs:: - - .. group-tab:: devdax - - In `devdax` mode: - - .. code-block:: bash - - docker run --device=/:/dev/dax0.0 --ulimit nofile=262144:262144 -p 9042:9042 -p 7000:7000 -it --name cassandra-test - - .. group-tab:: fsdax - - In `fsdax` mode: - - .. code-block:: bash - - docker run --mount type=bind,source=/,target=/mnt/pmem --ulimit nofile=262144:262144 -p 9042:9042 -p 7000:7000 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=' --name cassandra-test - - -Container Configuration -======================= - -Using environment variables ---------------------------- - -The container listens on the primary container IP address, but if required, some parameters can be provided as environment variables using `--env`. - -* `CASSANDRA_CLUSTER_NAME` Cassandra cluster name, by default `Cassandra Cluster` -* `CASSANDRA_LISTEN_ADDRESS` Cassandra listen address -* `CASSANDRA_RPC_ADDRESS` Cassandra RPC address -* `CASSANDRA_SEED_ADDRESSES` A comma separated list of hosts in the cluster, if not provided, cassandra is going to run as a single node. -* `CASSANDRA_SNITCH` The snitch type for the cluster, by default it is `SimpleSnitch`, for more complex snitches you can mount your own `cassandra-rackdc.properties` file. -* `LOCAL_JMX` If set to `no` the JMX service will listen on all IP addresses, the default is `yes` and listens just on localhost 127.0.0.1 -* `JVM_OPTS` When set you can pass additional arguments to the JVM for cassandra execution, for example for specifying memory heap sizes `JVM_OPTS=-Xms16G -Xmx16G -Xmn12G` - -When using PMEM in `fsdax` mode, there are some parameters to control the allocation of memory: - - -* `CASSANDRA_FSDAX_POOL_SIZE_GB` The size of the fsdax pool in GB, if it is not specified the pool size is `1` -* `CASSANDRA_PMEM_POOL_NAME` The filename of the pool created in PMEM, by default `cassandra_pool` - -Using custom files ------------------- - -For more complex deployments it is also possible to provide custom `cassandra.yaml` and `jvm.options` files as shown below: - -.. code-block:: bash - - docker run --mount type=bind,source=/,target=/mnt/pmem -it --ulimit nofile=262144:262144 --mount type=bind,source=//cassandra.yaml,target=/workspace/cassandra/conf/cassandra.yaml --mount type=bind,source=/path-to-file>/jvm.options,target=/workspace/cassandra/conf/jvm.options --name cassandra-custom-files - - -Clustering -========== - -For a simple two node cluster using PMEM in `fsdax` mode on both containers: - -Node 1 ------- - -* IP: 172.17.0.2 -* PMEM mountpoint: /mnt/pmem1 - -.. code-block:: bash - - docker run --mount type=bind,source=/mnt/pmem1,target=/mnt/pmem --ulimit nofile=262144:262144 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=2' -e 'CASSANDRA_SEED_ADDRESSES=172.17.0.2:7000,172.17.0.3:7000' --name cassandra-node1 - - -Node 2 ------- - -* IP: 172.17.0.3 -* PMEM mountpoint: /mnt/pmem2 - -.. code-block:: bash - - docker run --mount type=bind,source=/mnt/pmem2,target=/mnt/pmem --ulimit nofile=262144:262144 -it -e 'CASSANDRA_FSDAX_POOL_SIZE_GB=2' -e 'CASSANDRA_SEED_ADDRESSES=172.17.0.2:7000,172.17.0.3:7000' --name cassandra-node2 - - -Once both nodes are running, eventually the gossip is settled and we can use `nodetool` on either container to check cluster status. - -.. code-block:: bash - - docker exec -it bash /workspace/cassandra/bin/nodetool status - - -The output should look similar to this: - -.. code-block:: console - - - Datacenter: datacenter1 - ======================= - Status=Up/Down - |/ State=Normal/Leaving/Joining/Moving - -- Address Load Tokens Owns (effective) Host ID Rack - UN 172.17.0.3 0 bytes 256 100.0% 22387159-8192-41cf-8b6c-8bf0e1049eb7 rack1 - UN 172.17.0.2 0 bytes 256 100.0% 219b56ba-c07c-400b-a018-a5dc20edeb09 rack1 - - - -Persistence -=========== - -By default you can access the data written to Apache Cassandra as long as the container exists. In order to persist the data past that, you can mount volumes or bind mounts on :file:`/workspace/cassandra/data` and :file:`/workspace/cassandra/logs` and in this way the data can still be accessed once the container is deleted. - -Deploy An Apache Cassandra-PMEM cluster on Kubernetes* -****************************************************** - -Many containerized workloads are deployed in clusters and orchestration software like Kubernetes can be useful. We will use the `cassandra-pmem-helm`_ Helm* chart in this example. - -Requirements -============ - -* Kubectl* must be configured to access the Kubernetes Cluster - -* A Kubernetes cluster with `pmem-csi`_ enabled - -* The Kubernetes cluster must have `helm`_ and tiller installed - -* PMEM hardware - -.. important:: - - When selecting the `fsdax` pool file size, it is important to consider that when requesting a volume, certain amount of space is used by the filesystem metadata on that volume and the available space turns out to be less than total amount specified. Taking this into consideration the size of the fsdax pool file should be ~2G less than the total volume size requested. - - -Configuration -============= - -In order to configure the Apache Cassandra PMEM cluster some variables and values are provided. These values are set in :file:`test/cassandra-pmem-helm/values.yaml`, and can be modified according to your specific needs. A summary of those parameters is shown below: - - -* clusterName: The cluster Name set across all deployed nodes -* replicaCount: The number of nodes in the cluster to be deployed -* image.repository: The address of the container registry where the cassandra-pmem image should be pulled -* image.tag: The tag of the image to be pulled during deployment -* image.name: The name of the image to be pulled during deployment -* pmem.containerPmemAllocation: The size of the persistent volume claim to be used as heap, it uses the storage class `pmem-csi-sc-ext4` from pmem-csi The size of the fsdax pool to be created inside the persistent volume claim, in practice it should be `1G` less than pmem.containerPmemAllocation -* pmem.fsdaxPoolSizeInGB: The size of the fsdax pool to be created inside the persistent volume claim, in practice it should be 1G less than pmem.containerPmemAllocation -* enablePersistence: If set to `true`, K8s persistent volumes are deployed to store data and logs -* persistentVolumes.logsVolumeSize: The size of the persistent volume used for storing logs on each node, the default is `4G` -* persistentVolumes.dataVolumeSize: The size of the persistent volume used for storing data on each node, the default is `4G` -* persistentVolumes.logsStorageClass: Storage class used by the logs pvc, by default it uses `pmem-csi-sc-ext4` -* persistentVolumes.dataStorageClass: Storage class used by the data pvc, by default it uses `pmem-csi-sc-ext4` -* provideCustomConfig: If set to `true`, it mounts all the files located on `/files/conf` on `/workspace/cassandra/conf` inside each container in order to provide a way to customize the deployment beyond the options provided here -* exposeJmxPort: When set to `true` it exposes the JMX port as part of the Kubernetes headless service. It should be used together with `enableAdditionalFilesConfigMap` in order to provide authentication files needed for JMX when the remote connections are allowed. When set to `false` only local access through 127.0.0.1 is granted and no additional authentication is needed. -* enableClientToolsPod: If set to `true`, an additional pod independent from the cluster is deployed, this pod contains various Cassandra client tools and mounts test profiles located under `/files/testProfiles` to `/testProfiles` inside the pod. This pod is useful to test and launch benchmarks -* enableAdditionalFilesConfigMap: When set to true, it takes the files located in `/files/additionalFiles` and mount them in `/etc/cassandra` inside the pods, some additional files for cassandra can be stored here, such as JMX auth files -* jvmOpts.enabled: If set to `true` the environment variable `JVM_OPTS` is overridden with the value provided on jvmOpts.value -* jvmOpts.value: Sets the value of the environment variable `JVM_OPTS`, in this way some java runtime configurations can be provided such as RAM heap usage -* resources.enabled: if set to `true`, the resource constraints are set on each pod using the values under resources.requests and resources.limits -* resources.requests.memory: Initial resource allocation for each pod in the cluster -* resources.request.cpu: Initial resource allocation for each pod in the cluster -* resources.limits.memory: Limits for memory allocation for each pod in the cluster -* resources.limits.cpu: Limits for cpu allocation for each pod in the cluster - -Installation -============ - -Once all the configurations are set, to install the chart inside a given Kubernetes cluster you must run: - -.. code-block:: bash - - helm install ./cassandra-pmem-helm - - -Eventually all the given nodes will be shown as running using :command:`kubectl get pods`. - - -Running DBRS with Redis -*********************** - -The Redis stack application is enabled for a multinode Kubernetes environment using Intel Optane DCPMM PMem DIMMs in fsdax mode for storage. - -The source code used for this application can be found in the `Github repository`_ - -The following examples will use the `Docker image with Redis`_. You can also build your own image with Docker by using the :file:`Dockerfile` and running with this command - -.. code-block:: bash - - docker build --force-rm --no-cache -f Dockerfile -t ${DOCKER_IMAGE} . - - - -Single node -=========== - -Prior to starting the container, you will need to have the Intel Optane DCPMM module in fsdax with a file system and mounted in `/mnt/dax0` as shown above. - -Use the following to start the container, replacing ${DOCKER_IMAGE} with the name of the image you are using. - -.. code-block:: bash - - docker run --mount type=bind,source=/mnt/dax0,target=/mnt/pmem0 -i -d --name pmem-redis ${DOCKER_IMAGE} --nvm-maxcapacity 200 --nvm-dir /mnt/pmem0 --nvm-threshold 64 --protected-mode no - - - - -Redis Operator in a Kubernetes cluster -====================================== - -After setting up :ref:`kubernetes` in |CL|, you will need to enable it to support DCPMM using the pmem-cls driver. To install the driver follow the instructions in the `pmem-csi`_ repository. - -We are using source code from the `Redis operator`_ . - -.. note:: - - If you already have a redis-operator, you will need to delete it before installing a new one. - - - - -After installing the operator you are ready to deploy redisfailover instances using a yaml file, like this `example for persistent memory`_. You can download it and change the source of the image to reflect your environment. We have named our yaml `redis-failover.yml` - -To start a redisfailover instance in Kubernetes run the following - -.. code-block:: bash - - kubectl create -f redis-failover.yml - - -.. important:: - - There is a `known issue`_ in which the sentinels do not have enough memory to create the InitContainer. The current workaround is to build the image increasing the limits for the InitContainer memory to 32Mb - -Running DBRS with Memcached -*************************** - -With DBRS V2.0 you can use the DBRS stack with `Memcached`_, a free and open source, high performance, distributed meory object caching system. This stack is ready to use DCPMM in fsdax for storage. The source for this application can be found in the `Memcached`_ repository. - -.. note:: - - The DBRS v2.0 release does not support Redis or Cassandra. - - - -Build the DBRS Memcached image -============================== - -To build the Memcached enabled image, use the Dockerfile with this command: - -.. code-block:: bash - - docker build --force-rm --no-cache -f Dockerfile -t ${DOCKER_IMAGE} . - - -Run DBRS with Memcached as a standalone container -================================================= - -Prior to launching the container, you will need to configure the DCPMM in fsdax mode with a file system, and have it mounted in :file:`/mnt/dax0`. Instructions for configuration can be found in :ref:`dbrs-hardware-configuration`. - -To launch the container run this command: - -.. code-block:: bash - - docker run --mount type=bind,source=/mnt/dax0,target=/mnt/pmem0 -i -d --name pmem-memchached ${DOCKER_IMAGE} -e /mnt/pmem0/memcached.file -m 64 -c 1024 -p 11211 - -where: - -:command:`-m` is the maximum memory limit to use in megabytes -:command:`-e` is the mmap path for external memory (DCPMM storage). For this container the DCPMM sould be mounted inside the container on :file:`/mnt/pmem0` -:command:`-c` is the number of concurrent connections -:command:`-p` is the TCP connection port. - -For more information please refer to this `blog post`_ from `Memcached`_ - - -*Intel, Xeon, Intel Optane, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* - - -.. _Intel Download Center: https://downloadcenter.intel.com/download/28695/Intel-Server-Board-S2600WF-Family-BIOS-and-Firmware-Update-Package-for-UEFI - -.. _Quick Start Guide: https://software.intel.com/en-us/articles/quick-start-guide-configure-intel-optane-dc-persistent-memory-on-linux - -.. _Managing NVDIMMs: https://docs.pmem.io/ndctl-user-guide/managing-nvdimms - -.. _Configure, Manage, and Profile: https://software.intel.com/en-us/articles/configure-manage-and-profile-intel-optane-dc-persistent-memory-modules - -.. _DBRS github repository: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs - -.. _build-cassandra-pmem.sh: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs/cassandra/scripts/ - -.. _cassandra-pmem-helm: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dbrs/cassandra/cassandra-pmem-helm - -.. _helm: https://helm.sh/ - -.. _Github repository: https://github.com/pmem/pmem-redis - -.. _Redis operator: https://github.com/spotahome/redis-operator - -.. _example for persistent memory: https://github.com/spotahome/redis-operator/blob/master/example/redisfailover/pmem.yaml - -.. _known issue: https://github.com/spotahome/redis-operator/issues/176 - -.. _Docker image with Apache Cassandra: https://hub.docker.com/r/clearlinux/stacks-dbrs-cassandra - -.. _Docker image with Redis: https://hub.docker.com/r/clearlinux/stacks-dbrs-redis - -.. _Intel Optane PMM: https://www.intel.com/content/www/us/en/architecture-and-technology/optane-technology/optane-for-data-centers.html - -.. _pmem-csi: https://github.com/intel/pmem-csi/blob/release-0.6/README.md - -.. _DBRS Terms of Use: https://clearlinux.org/stacks/database/terms-of-use - - - -.. _Database Reference Stack website: https://clearlinux.org/stacks/database-reference - -.. _DBRS V1.0: https://clearlinux.org/news-blogs/database-reference-stack-dbrs-v10-now-available - -.. _DBRS V2.0: https://clearlinux.org/blogs-news/database-reference-stack-dbrs-v2-now-available - -.. _Memcached: https://memcached.org - -.. _blog post: https://memcached.org/blog/persistent-memory/ diff --git a/source/guides/stacks/dlrs-inference.rst b/source/guides/stacks/dlrs-inference.rst deleted file mode 100644 index eebcf15d..00000000 --- a/source/guides/stacks/dlrs-inference.rst +++ /dev/null @@ -1,1047 +0,0 @@ -.. _dlrs-inference: - -AI Inference with the Deep Learning Reference Stack -################################################### - -In this guide walk through a solution for using the Deep Learning Reference Stack with a Seldon Core\* platform deployed on Kubernetes\*. Seldon Core simplifies deployment of the models we create and use with the Deep Learning Reference Stack. Use this guide to set up your infrastructure and deploy a benchmarking workload on your Kubernetes cluster. - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -.. figure:: /_figures/stacks/kubeflow-seldon-dlrs-example-diagram.png - :alt: Example diagram with DLRS deployed by Seldon - :width: 800 - - -The solution covered here requires the following software components: - -* `Deep Learning Reference Stack`_ which is a |CL-ATTR| based Docker\* container providing deep learning frameworks and is optimized for Intel® Xeon® Scalable processors. -* `Kubeflow`_ is the machine learning toolkit for Kubernetes that helps with deployment of Seldon Core and Istio components. -* `Seldon Core`_ is a software platform for deploying machine learning models. We use the DLRS container to serve the OpenVino\* framework for inference with the Seldon Core. -* The OpenVino Model Server is included in DLRS and provides the OpenVino framework for inference. From OpenVino, the `OpenVino Toolkit`_ provides improved neural network performance on a variety of Intel processors. For this guide, we converted pre-trained Caffe models into the `Intermediate Representation(IR)`_ of ResNet50 with the OpenVino toolkit. -* `Istio`_ is a traffic manager and performs the load balancing for service requests in the cluster. -* Pre-processing container converts jpeg content into NumPy array -* Post-processing container converts an array of classification probabilities to a human-readable class name. -* Pre and post-processing containers are created using the `Source-to-Image`_ (S2I)toolkit, which builds reproducible container images from source code/ -* `Min.io`_ is used as the distributed object storage for models - - - -Prerequisites -************* - -Although this guide assumes a |CL| host system, it has also been validated with the following software components. - -.. list-table:: **Table 1. Software Component Versions** - :widths: 16,16 - :header-rows: 1 - - * - Component - - Version - - * - DLRS - - 0.4.0 - - * - Docker - - 18.09 - - * - Kubernetes - - 1.15.3 - - * - Source-to-Image - - 1.1.14 - - * - Helm - - 2.14.3 - - * - Kubeflow - - 0.6.1 - - * - Seldon - - 0.3.2 - - * - Rook - - 1.0.5 - - * - Ceph - - 14.2.1-20190430 - - * - Minio - - RELEASE.2019-04-23T23-50-36Z - - * - CentOS - - 7.6 - - * - OpenVINO Toolkit - - 2019 R1.0.1 - - * - MKL-DNN - - 0.19 - -Recommended Hardware -==================== - -We validated this guide on a server with a `2nd Generation Intel Xeon Scalable processor`_, formerly Cascade Lake, and this is recommended to get optimal performance and take advantage of the built in Intel® Deep Learning Boost (Intel® DL Boost) functionality. - -Required Software -================= - -#. :ref:`Install ` |CL| on your host system - - -#. Install the :command:`containers-basic` and :command:`cloud-native-basic` bundles: - - .. code-block:: bash - - sudo swupd bundle-add containers-basic cloud-native-basic - - -#. Start Docker - - Docker is not started upon installation of the :command:`containers-basic` bundle. To start Docker, enter: - - .. code-block:: bash - - sudo systemctl start docker - - -#. Install and configure :ref:`kubernetes`. - - - - - - - -.. note:: - - The Deep Learning Reference Stack was developed to provide the best user experience when executed on a |CL| host. However, as the stack runs in a container environment, you should be able to complete the following sections of this guide on other Linux* distributions, provided they comply with the Docker\* and Kubernetes\* package versions listed above. Look for your distribution documentation on how to update packages and manage Docker services. - - For other systems, please install the following software - - * `Docker 18.09`_ - * `Kubernetes 1.15.3`_ - - -Infrastructure Set-Up -********************* - -Environment -=========== - -Throughout this guide we will refer to the DEPLOY_DIR environment variable. DEPLOY_DIR is a pointer to the current directory with all resources used as the installation directory. Set it as follows - -.. code-block:: bash - - DEPLOY_DIR=`pwd` - -Deployment Tools -================ - -Source-to-Image (S2i) ---------------------- - -S2i is a tool for building artifacts from source and injecting them into Docker images. We use S2i to build the Imagenet transformer. Install it: - -.. code-block:: bash - - wget https://github.com/openshift/source-to-image/releases/download/v1.1.14/source-to-image-v1.1.14-874754de-linux-amd64.tar.gz - tar -zxvf source-to-image-v1.1.14-874754de-linux-amd64.tar.gz - mv -f -t /usr/local/bin/ sti s2i - rm -f source-to-image-v1.1.14-874754de-linux-amd64.tar.gz - chmod +x /usr/local/bin/sti - chmod +x /usr/local/bin/s2i - -kfctl ------ - -`kfctl` is a client used to control and deploy the Kubeflow platform. Install with: - -.. code-block:: bash - - wget https://github.com/kubeflow/kubeflow/releases/download/v0.6.1/kfctl_v0.6.1_linux.tar.gz - tar -zxvf kfctl_v0.6.1_linux.tar.gz - rm -f kfctl_v0.6.1_linux.tar.gz - mv -f kfctl /usr/local/bin/ - chmod +x /usr/local/bin/kfctl - -Minio ------ - -The Minio client is compatible with object cloud storage services. We use it to manage buckets and files stored in Minio storage. Install with: - -.. code-block:: bash - - wget https://dl.min.io/client/mc/release/linux-amd64/mc - mv mc /usr/local/bin/ - chmod +x /usr/local/bin/mc - - -Helm ----- - -Helm is used to deploy components on Kubernetes clusters. Helm is included in the :file:`cloud-native-basic` bundle in |CL| and can be installed with - -.. code-block:: bash - - sudo swupd bundle-add cloud-native-basic - -If you are not using a |CL| host, install with: - -.. code-block:: bash - - wget https://get.helm.sh/helm-v2.14.3-linux-amd64.tar.gz - tar -zxvf helm-v2.14.3-linux-amd64.tar.gz - rm -f helm-v2.14.3-linux-amd64.tar.gz - mv linux-amd64/helm /usr/local/bin/helm - -Regardless of your host OS, initialize Helm as follows: - -.. code-block:: bash - - helm init - kubectl create serviceaccount --namespace kube-system tiller - kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller - kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}' - - -gsutil ------- - -:file:`gsutil` is a client utility to work with Google Cloud\* storage. Follow the instructions to install `gsutil`_ . With the initialized Google Cloud storage command line interface, we will be able to download ResNet50 models, which we will use for model serving. - - -Rook ----- - -Rook.io is used to deploy Minio and Ceph. Clone the GitHub\* repository: - -.. code-block:: bash - - git clone -b release-1.0 https://github.com/rook/rook.git - - -.. todo: ADD CORRECT GITHUB LINK FOR ai-inferencing REPO - -AI Inferencing --------------- - -This guide is based on the code in the IntelSolutionDev Ai Inferencing repository. Clone the repository - -.. code-block:: bash - - git clone https:// - - - - -Platform Backends -================= - -Ceph ----- - -#. Deploy Ceph Rook Operator - - - The Rook Operator is used to deploy the remaining Rook Ceph components. Deploy it: - - .. code-block:: bash - - cd $DEPLOY_DIR - cd rook/cluster/examples/kubernetes/ceph - kubectl create -f common.yaml - kubectl create -f operator.yaml - kubectl -n rook-ceph get pods # wait for rook-ceph-operator pod - -#. Deploy Rook Ceph Cluster - - - The Rook Ceph cluster is used for block storage for all platform components. You will need to modify the :file:`cluster.yaml` for your requirements. For this guide, we will prepare a cluster with 3 mons, and we will store data in :file:`/var/lib/rook` on all nodes. Modify the file: - - .. code-block:: yaml - - apiVersion: ceph.rook.io/v1 - kind: CephCluster - metadata: - name: rook-ceph - namespace: rook-ceph - spec: - cephVersion: - image: ceph/ceph:v14.2.1-20190430 - allowUnsupported: false - dataDirHostPath: /var/lib/rook - mon: - count: 3 - allowMultiplePerNode: false - dashboard: - enabled: true - network: - hostNetwork: false - rbdMirroring: - workers: 0 - annotations: - resources: - storage: - useAllNodes: true - useAllDevices: false - deviceFilter: - location: - config: - directories: - - path: /var/lib/rook - - After modifying the :file:`cluster.yaml`, run: - - .. code-block:: bash - - kubectl create -f cluster.yaml - kubectl -n rook-ceph get pods #wait for osd pods - kubectl create -f toolbox.yaml - kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" - kubectl create -f storageclass.yaml - kubectl patch storageclass rook-ceph-block -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' - - To verify the setup is correct, run: - - .. code-block:: bash - - kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='{.items[0].metadata.name}') ceph status - - The command should return: - - .. code-block:: console - - HEALTH_OK - -#. Troubleshooting - - If you see a warning related to undersized PGs you need to increase the number of PGs using these commands: - - First get number of PGs: - - .. code-block:: bash - - ceph osd pool get replicapool pg_num - - Then double the number of pgs (for example from 300 to 600): - - .. code-block:: bash - - ceph osd pool set replicapool pg_num 600 - ceph osd pool set replicapool pgp_num 600 - -Minio ------ - -The Minio cluster is used as object storage for all components in the platform. Deploy it: - -.. code-block:: bash - - cd $DEPLOY_DIR - cd rook/cluster/examples/kubernetes/minio - kubectl create -f operator.yaml - kubectl -n rook-minio-system get pods # wait for rook-minio-operator pod - kubectl create -f object-store.yaml - -.. note:: - - Minio pods will not start if you are using a proxy in your environment. Please check the proxy settings in the :file:`/etc/kubernetes/manifests/kube-apiserver.yaml`. The `.local,.svc,.nip.io` line should be set to `no_proxy`. - -Docker registry ---------------- - -This Docker registry will be used for all platform components. We will use helm to set up the registry as shown: - -.. code-block:: bash - - cd $DEPLOY_DIR - cd ai-inferencing/infra - helm install --namespace registry --name registry stable/docker-registry -f registry-values.yaml - -Verify the registry setup - -.. code-block:: bash - - REGISTRY_URL=`kubectl get svc -n registry | grep NodePort | awk '{ print $3; }'`.nip.io:5000 - - -Create the Machine Learning Platform -==================================== - -The machine learning platform for this guide is built using the Kubeflow Toolkit from which we use the Seldon-core and Istio components. - -#. Prepare the definition files - - First we will get the configuration file for Istio - - .. code-block:: bash - - cd $DEPLOY_DIR - wget https://raw.githubusercontent.com/kubeflow/kubeflow/v0.6.1/bootstrap/config/kfctl_k8s_istio.yaml - sed -i 's/master.tar.gz/v0.6.1.tar.gz/g' kfctl_k8s_istio.yaml - kfctl init kubeflow --config=$(pwd)/kfctl_k8s_istio.yaml -V - cd kubeflow - kfctl generate all -V - -#. Edit :file:`kustomize/seldon-core-operator/base/statefulset.yaml` to change the version to `0.3.2-SNAPSHOT`. - -#. Edit :file:`kustomize/istio-install/base/istio-noauth.yaml` to change limits for the istio-pilot deployment as shown: - - .. code-block:: yaml - - resources: - limits: - cpu: 1000m - memory: 1000Mi - - This will correct a performance issue which results in istio-pilot causing crashes with multiple Seldon deployments start simultaneously. - - .. note:: - - If istio cannot start because of an OOM (Out of Memory) error, change the limits of all istio-system deployments. The Default settings should be enough for a small cluster (32GB RAM and less). - - - -#. Install the Kubeflow components and wait for all pods in the Kubeflow and istio-system namespace to start. - - .. code-block:: bash - - kfctl apply all -V - -#. Run - - .. code-block:: bash - - kubectl label namespace kubeflow istio-injection=enabled - - kubectl apply -f - < Dockerfile - FROM clearlinux/stacks-dlrs-mkl:v0.4.0 - COPY serve.sh /workspace/scripts/serve.sh - EOF - -#. Create the :file:`serve.sh` file - - .. code-block:: bash - - cat < serve.sh - #!/bin/bash - # temporary workaround - PY_PATH="/usr/local/lib/openvino/inference_engine/:/usr/local/lib" - echo "export PYTHONPATH=\${PY_PATH}" >>/.bashrc - source ~/.bashrc - - # start the model server - cd /ie_serving_py - exec "\$@" - EOF - -#. Make :file:`serve.sh` executable - - .. code-block:: bash - - chmod +x serve.sh - -#. Build the new docker image - - .. code-block:: bash - - REGISTRY_URL=`kubectl get svc -n registry | grep NodePort | awk '{ print $3; }'`.nip.io:5000 - sudo docker build -t ${REGISTRY_URL}/dlrs-mkl-fixed:v0.4.0 . - -#. Upload the image to the registry - - .. code-block:: bash - - sudo docker push ${REGISTRY_URL}/dlrs-mkl-fixed:v0.4.0 - - -Deploy Using Helm with Seldon -============================= - -At this point you are ready to go. Use the Helm chart with Seldon for deployment: - -.. code-block:: bash - - helm install \ - --namespace kubeflow \ - --name seldonovms-server-res \ - --set transformer.image=$REGISTRY_URL/imagenet_transformer:0.1 \ - --set openvino.image=$REGISTRY_URL/dlrs-mkl-fixed:v0.4.0 \ - ai-inferencing/seldon - -Verify that all pods are in the `Running` state: - -.. code-block:: bash - - kubectl -n kubeflow get pods -l version=openvino - -You have now created the inference infrastructure! - - - -Secure Communication -==================== - -You can optionally set up secure communication between the clients and the server. This is not required for completing this guide, but we will walk through it for completeness. - -For this example we will use `10.0.0.1.nip.io` for our domain name. - -#. Clone the repository - - .. code-block:: bash - - git clone https://github.com/nicholasjackson/mtls-go-example - -#. Generate the certificates. - - This script will generate four directories: 1_root, 2_intermediate, 3_application, and 4_client containing the client and server certificates that will be used in the following procedures. When prompted, select `y` for all questions. - - .. code-block:: bash - - cd mtls-go-example - ./generate.sh 10.0.0.1.nip.io password - mkdir 10.0.0.1.nip.io && mv 1_root 2_intermediate 3_application 4_client 10.0.0.1.nip.io - -#. Create a Kubernetes secret to hold the server's certificate and private key. - - We'll use :command:`kubectl` to create the secret istio-ingressgateway-certs in namespace istio-system. The Istio gateway will load the secret automatically. - - .. code-block:: bash - - kubectl create -n istio-system secret tls istio-ingressgateway-certs --key 10.0.0.1.nip.io/3_application/private/10.0.0.1.nip.io.key.pem --cert 10.0.0.1.nip.io/3_application/certs/10.0.0.1.nip.io.cert.pem - -#. Verify that :file:`tls.crt` and :file:`tls.key` have been mounted in the ingress gateway pod - - .. code-block:: bash - - kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-certs - -#. Edit the default kubeflow gateway - - .. code-block:: bash - - kubectl apply -f - < - export INGRESS_ADDRESS= - -#. Build the Docker image: - - .. code-block:: bash - - docker build -t ${REGISTRY_URL}/seldon-ovms-locust-client:0.1 --network=host . - -#. Push the image to the Docker registry - - .. code-block:: bash - - docker push ${REGISTRY_URL}/seldon-ovms-locust-client:0.1 - -#. Change to the :file:`ai-inferencing/clients/locust/helm` directory and modify the number of Lucust slave nodes by editing the :file:`values.yaml` file. Change `slaves_replicas` to the desired number of slave nodes. - -#. Run Locust, modifying this command as your environment requires: - - .. code-block:: bash - - helm helm install --name locust --namespace kubeflow - --set client.image=${REGISTRY_URL}/seldon-ovms-locust-client:0.1 - --set client.ingress=${INGRESS_ADDRESS} - --set client.mount_images_volume.enabled=false - --set client.images_path=./ - ../helm - - Values can be adjusted in the helm command using `--set` as shown in this sample command. Note that `.nip.io` may be necessary when using ingress. - -#. Find the UI port in the output from the helm command: - - .. code-block:: console - - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - locust NodePort 10.110.167.232 8089:XXXXX/TCP 0s - locust-master ClusterIP 10.107.78.16 5557/TCP,5558/TCP 0s - - - -#. On the system running the Kubernetes cluster, open a browser and go to `localhost:XXXXX` where `XXXXX` is the port found above. - -#. Run tests using the UI. - - * In the Locust's landing page you will see 2 fields - Number of users to simulate and Hatch rate. Fill them and press "start swarming" - * Locust should start the test. You can track the number of requests and fails in the "statistics" tab. - - .. figure:: /_figures/stacks/Locust_statistics.png - :alt: Locust statistics - :width: 600 - - * In the "Failures" section you should see the type of errors - there should be only classified errors while running the test. This means that the sent image was classified incorrectly. That's normal behavior - we expect <100% accuracy for this model. - - .. figure:: /_figures/stacks/Locust_failures.png - :alt: Locust failures - :width: 600 - - * You can see some simple charts in the "charts" tab. In "Response Times (ms)" chart, the green line is "Median Response Time", yellow line is "95% percentile". - - .. figure:: /_figures/stacks/Locust_charts.png - :alt: Locust charts - :width: 600 - - * In the Exceptions tab, there might be some exceptions shown. This might happen when tested environments reach their response limit and some requests start to fail. - - .. figure:: /_figures/stacks/Locust_exception.png - :alt: Locust exception - :width: 600 - - - -Performance Tuning -================== - -If you need to maximize the usage of available resources, -it is worth to adjust the threading parameters of inference serving instances. It is not enough to set the OMP_NUM_THREADS environment parameter which defines the number of threads used for inference on the CPU. In this case, the instances will scale across the nodes, but won't scale properly across the available cores on one node. Using the :command:`numactl` program is the solution in this case. :command:`numactl` allows you to run the instance on defined cores and uses memory from the same socket. - -To find out how to assign the cores and memory properly run :command:`numactl -H` which will produce output like this: - -.. code-block:: console - - available: 2 nodes (0-1) - node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 - node 0 size: 195279 MB - node 0 free: 128270 MB - node 1 cpus: 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 - node 1 size: 196608 MB - node 1 free: 119445 MB - node distances: - node 0 1 - 0: 10 21 - 1: 21 10 - - -In this case, the tests are run on Intel® Xeon® Platinum 6260L processor with 2 sockets(nodes) and 24 cores (CPUs) on each socket. -Running the inference serving the application with :command:`numactl --membind=0 --cpubind=0-3` forces the system to use 0,1,2,3 cores and memory located on the same socket (0). To use all available cores there is a need to create more service deployments assigned to the remaining cores. - -The `ai-inferencing` repository contains an example deployment script with 2 cores per instance assignment. - -Automatic CPU and memory binds in Seldon deployment -=================================================== - -The Seldon deployment works by default using one deployment only, that is, only one Seldon deployment should be spawned on one cluster node. When there is only one instance of the deployment, it is not necessary to use :command:`numactl` as all resources can be used by this single deployment. - -In most cases that is far too many resources being used, so this setting is not optimal. Instead, use a mechanism that allows creating more than one deployment per node, and equally spliting CPU and memory banks resources between them, using :command:`numactl`. - -First, it is necessary to set the following Helm values in the :file:`ai-inferencing/seldon/values.yaml` file: - -* `instances` is a number describing how many Seldon deployments and different resources ranges should be prepared (each CPU bind range would be used by only single one deployment) to be used by :file:`numactl` on a single socket. When this variable is set to 1, :file:`numactl` is not used. -* `cpus` should be set to the number of physical CPUs on a single node (without HyperThreading) -* `sockets` should be equal to the number of sockets on a single node and to memory banks number - -Run the benchmark -================= - -There are 2 scripts prepared to automate finding the best configuration -by customizing the number of clients and Seldon instances. - -#. :file:`clients/standalone/scale.sh` - - This is a script created to automatically scale and adjust Seldon instances to the selected configuration (2 or 24 cores per instance). - - It takes the following arguments: - * the number, how many replicas (pods) each Seldon deployment should contain, this number should be equal to the number of the nodes in the cluster - * the number, how many deployments should be created (each node would divide resources between deployments) - - This script is called by :file:`clients/standalone/benchmark.sh` script. - -#. :file:`clients/standalone/benchmark.sh` - - This script is used to run benchmarks with selected configuration. - There are 3 benchmark options to set: - - * number of `nodes` - how many nodes are in the cluster, this will scale Seldon deployments, to have one pod replica for each resource slice on each node. - * list of `instances` values - how many Seldon instances would be started for a particular benchmark - * list of `clients` values - it represents the number of clients to be used in particular benchmark - - It is necessary to customize the file itself to use the selected setup, setting environment variables mentioned below: - - * `SSH_PASSWORD` - password to Kubernetes master host - * `SSH_USER` - user to be used to connect Kubernetes master host - * `SSH_IP` - IP of the Kubernetes master host - * `SCALE_FILE_PATH` - path to downloaded this repository on the Kubernetes master host, for example :file:`/path/to/this/repository/clients/standalone` - * `INGRESS_ADDRESS` - server IP or domain name and port where Istio is exposed - - ssh settings should be set to Kubernetes master host where kubectl is usable. - -.. note:: Before starting :file:`benchmark.sh` script, make sure all standalone client requirements are fulfilled, including installed python requirements and downloaded small sample images set if it is used. - -The output from this file is shown on stdout and saved to file named -:file:`log_n<# nodes>_i<# instances per node>_c<# clients>_.txt`. - -The simplest way to monitor the cores usage is to run `htop` program on each tested node. - -.. figure:: /_figures/stacks/htop.png - :alt: htop output - :width: 600 - -Results -======= - -The test performed on a 2 node cluster with 48 cores per node showed that there are 2 optimal scenarios: - -#. Low latency - 2 instances with 24 cores per instance on each node (4 instances on 2 nodes): - - .. code-block:: console - - 1 (Node 1, socket 0): 'numactl --membind=0 --cpubind=0-23 - 2 (Node 1, socket 1): 'numactl --membind=1 --cpubind=24-46 - 3 (Node 2, socket 0): 'numactl --membind=0 --cpubind=0-23 - 4 (Node 2, socket 1): 'numactl --membind=1 --cpubind=46-47 - - - Inference engine configuration for this case - - .. code-block:: console - - OMP_NUM_THREADS=24 - KMP_SETTINGS=1 - KMP_AFFINITY=granularity=fine,verbose,compact,1,0 - KMP_BLOCKTIME=1 - - -#. High throughput - - 24 instances with 2 cores per instance on each node (48 instances on 2 nodes): - - .. code-block:: console - - 1 (Node 1, socket 0): 'numactl --membind=0 --cpubind=0-1 - 2 (Node 1, socket 0): 'numactl --membind=0 --cpubind=2-3 - ... - 48 (Node 2, socket 1): 'numactl --membind=1 --cpubind=46-47 - - - Inference engine configuration: - - .. code-block:: console - - OMP_NUM_THREADS=2 - KMP_SETTINGS=1 - KMP_AFFINITY=granularity=fine,verbose,compact,1,0 - KMP_BLOCKTIME=1 - - -*Intel, Xeon, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* - - -.. _Deep Learning Reference Stack: https://clearlinux.org/stacks/deep-learning -.. _Kubeflow: https://www.kubeflow.org/ -.. _Seldon Core: https://docs.seldon.io/projects/seldon-core/en/latest/ -.. _OpenVino Toolkit: https://software.intel.com/en-us/openvino-toolkit -.. _Intermediate Representation(IR): https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Converting_Model.html -.. _Istio: https://istio.io/ -.. _Source-to-Image: https://github.com/openshift/source-to-image -.. _Min.io: https://min.io/ -.. _2nd Generation Intel Xeon Scalable processor: https://www.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/cascade-lake/2nd-gen-intel-xeon-scalable-processors.html -.. _Docker 18.09: https://kubernetes.io/docs/setup/production-environment/container-runtimes/ -.. _Kubernetes 1.15.3: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ -.. _gsutil: https://cloud.google.com/storage/docs/gsutil_install#linux -.. _Locust.io: https://locust.io diff --git a/source/guides/stacks/dlrs.rst b/source/guides/stacks/dlrs.rst deleted file mode 100644 index 4387ee2b..00000000 --- a/source/guides/stacks/dlrs.rst +++ /dev/null @@ -1,1173 +0,0 @@ -.. _dlrs: - -Deep Learning Reference Stack -############################# - -This guide gives examples for using the Deep Learning Reference stack to run real-world usecases, as well as benchmarking workloads for TensorFlow\*, -PyTorch\*, and Kubeflow\* in |CL-ATTR|. - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -We created the Deep Learning Reference Stack to help AI developers deliver -the best experience on Intel® architecture. This stack reduces complexity -common with deep learning software components, provides flexibility for -customized solutions, and enables you to quickly prototype and deploy Deep -Learning workloads. Use this guide to run benchmarking workloads on your -solution. - -The latest release of the Deep Learning Reference Stack (`DLRS V6.0`_ ) supports the following features: - -* TensorFlow* 1.15 and TensorFlow* 2.2.0(rc1), an end-to-end open source platform for machine learning (ML). -* PyTorch* 1.4, an open source machine learning framework that accelerates the path from research prototyping to production deployment. -* PyTorch Lightning* which is a lightweight wrapper for PyTorch designed to help researchers set up all the boilerplate state-of-the-art training. -* Transformers* which is a state-of-the-art Natural Language Processing (NLP) library for TensorFlow 2.0 and PyTorch -* Flair*, a PyTorch NLP framework -* OpenVINO™ model server version 2020.1, delivering improved neural network performance on Intel processors, helping unlock cost-effective, real-time vision applications. -* Intel® Deep Learning Boost (Intel® DL Boost) with Intel® Advanced Vector - Extensions 512 (Intel® AVX-512) Vector Neural Network Instruction , designed to - accelerate deep neural network-based algorithms. -* Deep Learning Compilers (TVM* 0.6), an end-to-end compiler stack. - - -.. important:: - - To take advantage of the Intel AVX-512 and VNNI functionality (including the Intel® oneAPI Deep Neural Network Library (oneDNN), found at `oneDNN`_, with the Deep Learning Reference Stack, you must use the following hardware: - - * Intel AVX-512 images require an Intel® Xeon® Scalable processor - * VNNI requires a 2nd generation Intel Xeon Scalable processor - - -Releases -******** - -Refer to the `Deep Learning Reference Stack website`_ for information and download links for the different versions and offerings of the stack. - -* `DLRS V6.0`_ release announcement. -* `DLRS V5.0`_ release announcement. -* `DLRS V4.0`_ release announcement, including benchmark results. -* `DLRS V3.0`_ release announcement, including benchmark results. -* `DLRS V2.0`_ including PyTorch benchmark results. -* `DLRS V1.0`_ including TensorFlow benchmark results. -* `DLRS Release notes`_ on Github\* for the latest release of Deep Learning - Reference Stack. - -.. note:: - - The Deep Learning Reference Stack is a collective work, and each piece of - software within the work has its own license. Please see the `DLRS Terms of Use`_ - for more details about licensing and usage of the Deep Learning Reference Stack. - -Version compatibility -===================== - -We validated the steps in this guide against the following software package versions, unless otherwise stated: - -* |CL| 31290 (Minimum supported version) -* Docker 19.03 -* Kubernetes 1.11.3 -* Go 1.11.12 - -.. note:: - - The Deep Learning Reference Stack was developed to provide the best user experience when executed on a |CL| host. However, as the stack runs in a container environment, you should be able to complete the following sections of this guide on other Linux* distributions, provided they comply with the Docker*, Kubernetes* and Go* package versions listed above. Look for your distribution documentation on how to update packages and manage Docker services. - - -Prerequisites -============= - -* :ref:`Install ` |CL| on your host system -* Add the :command:`containers-basic` bundle -* Add the :command:`cloud-native-basic` bundle - -In |CL|, :command:`containers-basic` includes Docker\*, which is required for -TensorFlow and PyTorch benchmarking. Use the :command:`swupd` utility to -check if :command:`containers-basic` and :command:`cloud-native-basic` are -present: - -.. code-block:: bash - - sudo swupd bundle-list - -To install the :command:`containers-basic` or :command:`cloud-native-basic` -bundles, enter: - -.. code-block:: bash - - sudo swupd bundle-add containers-basic cloud-native-basic - -Docker is not started upon installation of the :command:`containers-basic` -bundle. To start Docker, enter: - -.. code-block:: bash - - sudo systemctl start docker - -To ensure that Kubernetes is correctly installed and configured, follow the -instructions in :ref:`kubernetes`. - -.. warning:: - - Note that although the DLRS images and dockerfiles may be modified for your needs, there are some modifications that may cause unexpected or undesirable results. For example, using the Clear Linux :command:`swupd bundle-add` command to add packages to a Clear Linux based container may overwrite the DLRS core components. Please use care when modifying the contents of the containers. If recaving Errors using the Clear Linux :command:`swupd bundle-add` command try running the Clear Linux :command:`swupd clean` command first. - - -Kubectl -======= - -You can use kubectl to run commands against your Kubernetes cluster. Refer to -the `kubectl overview`_ for details on syntax and operations. Once you have a -working cluster on Kubernetes, use the following YAML script to start a pod with -a simple shell script, and keep the pod open. - -#. Copy this example.yaml script to your system: - - .. code-block:: yaml - - apiVersion: v1 - kind: Pod - metadata: - name: example-pod - labels: - app: ex-pod - spec: - containers: - - name: ex-pod-container - image: sysstacks/dlrs-tensorflow-clearlinux:latest - command: ['/bin/bash', '-c', '--'] - args: [ "while true; do sleep 30; done" ] - -#. Execute the script with kubectl: - - .. code-block:: bash - - kubectl apply –f /example.yaml - - -This script opens a single pod and is helpful to verify your setup is complete and correct. More robust solutions would create a deployment or inject a python script or larger shell script into the container. - - -TensorFlow single and multi-node benchmarks -******************************************* - -This section describes running the `TensorFlow Benchmarks`_ in single node. -For multi-node testing, replicate these steps for each node. These steps -provide a template to run other benchmarks, provided that they can invoke -TensorFlow. - -.. note:: - - Performance test results for the Deep Learning Reference Stack and for this - guide were obtained using `runc` as the runtime. - -#. Download either the `TensorFlow Eigen`_ or the `TensorFlow oneDNN`_ Docker image - from `Docker Hub`_. - -#. Run the image with Docker: - - .. code-block:: bash - - docker run --name --rm -ti bash - - .. note:: - - Launching the Docker image with the :command:`-i` argument starts - interactive mode within the container. Enter the following commands in - the running container. - -#. Clone the benchmark repository in the container: - - .. code-block:: bash - - git clone http://github.com/tensorflow/benchmarks -b cnn_tf_v1.13_compatible - -#. Execute the benchmark script: - - .. code-block:: bash - - python benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --device=cpu --model=resnet50 --data_format=NHWC - -.. note:: - - You can replace the model with one of your choice supported by the - TensorFlow benchmarks. - - If you are using an FP32 based model, it can be converted to an int8 model - using `Intel® AI Quantization Tools for TensorFlow`_. - -PyTorch single and multi-node benchmarks -**************************************** - -This section describes running the `PyTorch benchmarks`_ for Caffe2 in -single node. - -#. Download either the `PyTorch with OpenBLAS`_ or the `PyTorch with Intel - oneDNN`_ Docker image from `Docker Hub`_. - -#. Run the image with Docker: - - .. code-block:: bash - - docker run --name --rm -i -t bash - - .. note:: - - Launching the Docker image with the :command:`-i` argument starts - interactive mode within the container. Enter the following commands in - the running container. - -#. Clone the benchmark repository: - - .. code-block:: bash - - git clone https://github.com/pytorch/pytorch.git - -#. Execute the benchmark script: - - .. code-block:: bash - - cd pytorch/caffe2/python - python convnet_benchmarks.py --batch_size 32 \ - --cpu \ - --model AlexNet - -TensorFlow Training (TFJob) with Kubeflow and DLRS -************************************************** - -.. warning:: - - If you choose the Intel oneDNN image, your platform - must support the Intel AVX-512 instruction set. Otherwise, an - *illegal instruction* error may appear, and you won’t be able to complete this guide. - -A `TFJob`_ is Kubeflow's custom resource used to run TensorFlow training jobs on Kubernetes. This example shows how to use a TFJob within the DLRS container. - -Pre-requisites: - -* A running :ref:`kubernetes` cluster - -#. Deploying Kubeflow with kfctl/kustomize in |CL| - -.. note:: - - This example proposes a Kubeflow installation using kfctl. Please download the `kfctl tarball`_ to complete the following steps - -#. Download, untar and add to your PATH if necessary - - .. code-block:: bash - - KFCTL_URL="https://github.com/kubeflow/kubeflow/releases/download/v0.6.1/kfctl_v0.6.1_linux.tar.gz" - wget -P ${KFCTL_URL} ${KFCTL_PATH} - tar -C ${KFCTL_PATH} -xvf ${KFCTL_PATH}/kfctl_v${kfctl_ver}_linux.tar.gz - export PATH=$PATH:${KFCTL_PATH} - -#. Install Kubeflow resource and TFJob operators - - .. code-block:: bash - - # Env variables needed for your deployment - export KFAPP="" - export CONFIG="https://raw.githubusercontent.com/kubeflow/manifests/master/kfdef/kfctl_k8s_istio.yaml" - - kfctl init ${KFAPP} --config=${CONFIG} -V - cd ${KFAPP} - - # deploy Kubeflow: - kfctl generate k8s -V - kfctl apply k8s -V - -#. List the resources - - Deployment takes around 15 minutes (or more depending on the hardware) to be ready to use. After that you can use kubectl to list all the Kubeflow resources deployed and monitor their status. - - .. code-block:: bash - - kubectl get pods -n kubeflow - -Submitting TFJobs -================= - -We provide `DLRS TFJob`_ examples that use the Deep Learning Reference Stack as the base image for creating the containers to run training workloads in your Kubernetes cluster. - - -Customizing a TFJob -=================== - -A TFJob is a resource with a YAML representation like the one below. Edit to use the DLRS image containing the code to be executed and modify the command for your own training code. - -If you'd like to modify the number and type of replicas, resources, persistent volumes and environment variables, please refer to the `Kubeflow documentation`_ - -.. code-block:: console - - apiVersion: kubeflow.org/v1beta2 - kind: TFJob - metadata: - generateName: tfjob - namespace: kubeflow - spec: - tfReplicaSpecs: - PS: - replicas: 1 - restartPolicy: OnFailure - template: - spec: - containers: - - name: tensorflow - image: dlrs-image - command: - - python - - -m - - trainer.task - - --batch_size=32 - - --training_steps=1000 - Worker: - replicas: 3 - restartPolicy: OnFailure - template: - spec: - containers: - - name: tensorflow - image: dlrs-image - command: - - python - - -m - - trainer.task - - --batch_size=32 - - --training_steps=1000 - Master: - replicas: 1 - restartPolicy: OnFailure - template: - spec: - containers: - - name: tensorflow - image: dlrs-image - command: - - python - - -m - - trainer.task - - --batch_size=32 - - --training_steps=1000 - -Results of running this section -=============================== - -You must parse the logs of the Kubernetes pod to retrieve performance -data. The pods will still exist post-completion and will be in -‘Completed’ state. You can get the logs from any of the pods to inspect the -benchmark results. More information about Kubernetes logging is available -in the Kubernetes `Logging Architecture`_ documentation. - -For more information, please refer to: -* `Distributed TensorFlow`_ -* `TFJobs`_ - - -PyTorch Training (PyTorch Job) with Kubeflow and DLRS -***************************************************** - -A `PyTorch Job`_ is Kubeflow's custom resource used to run PyTorch training jobs on Kubernetes. This example builds on the framework set up in the previous example. - -Pre-requisites: - -* A running :ref:`kubernetes` cluster -* Please follow steps 1 - 5 of the previous example to set up your environment. - - -Submitting PyTorch Jobs -======================= - -We provide `DLRS PytorchJob`_ examples that use the Deep Learning Reference Stack as the base image for creating the container(s) that will run training workloads in your Kubernetes cluster. - -Working with Horovod* and OpenMPI* -********************************** - -`Horovod`_ is a distributed training framework for TensorFlow, Keras, and PyTorch. The `OpenMPI Project`_ is an open source Message Passing Interface implementation. Running Horovod on OpenMPI will let us enable distributed training on DLRS. - -The following deployment uses `Kubeflow OpenMPI instructions`_, meaning you can replace the following variables to have a working Kubernetes cluster with openmpi workers for distributed training. - -To begin, refer to the instructions above to set up a Kubernetes cluster on Clear Linux. You will need to build and push the DLRS docker image with Horovod and OpenMPI enabled, modifying the dockerfile to build your image - -Building the Image -================== - -#. DLRS is part of the `Intel stacks GitHub repository`_. Clone the stacks repository. - - .. code-block:: bash - - git clone https://github.com/intel/stacks.git - -#. Create the ssh-entrypoint.sh script by copying the following into a file in the stacks/dlrs/clearlinux/tensorflow/mkl directory - - .. code-block:: console - - #! /usr/bin/env bash - set -o errexit - - mkdir -p /etc/ssh /var/run/sshd - - # Allow OpenSSH to talk to containers without asking for confirmation - cat << EOF > /etc/ssh/ssh_config - StrictHostKeyChecking no - Port 2022 - UserKnownHostsFile=/dev/null - PasswordAuthentication no - EOF - - /usr/sbin/ssh-keygen -A - -#. Inside the stacks/dlrs/clearlinux/tensorflow/mkl directory, modify the Dockerfile.builder file to add the openssh-server to the container. - - .. code-block:: console - - # update os and add required bundles - RUN swupd bundle-add git curl wget \ - java-basic sysadmin-basic package-utils \ - devpkg-zlib go-basic devpkg-tbb openssh-server - -#. To execute the ssh-entrypoint.sh in the container, add these lines to the Dockerfile.builder file - - .. code-block:: console - - COPY ssh-entrypoint.sh /bin/ssh-entrypoint.sh - RUN chmod +x /bin/ssh-entrypoint.sh - RUN ssh-entrypoint.sh - - .. note:: - - The ssh-entrypoint.sh script will generate ssh host keys for the docker image, but they will be the same every time the image is built. - - -#. Build the container with - - .. code-block:: bash - - make - - .. note:: - - More detail on building the container can be found on the `Intel stacks GitHub repository`_ - -Using the new image with Horovod and OpenMPI -============================================ - -To use the new image we will follow the `Kubeflow OpenMPI instructions`_. You will not need to follow the Installation section, as we have just completed that for the DLRS container. - -#. Generate and deploy Kubeflow's openmpi component. - - .. code-block:: console - - Create a namespace for kubeflow deployment. - kubectl delete namespace kubeflow - NAMESPACE=kubeflow - kubectl create namespace ${NAMESPACE} - - # Generate one-time ssh keys used by Open MPI. - SECRET=openmpi-secret - mkdir -p .tmp - yes | ssh-keygen -N "" -f .tmp/id_rsa -C "" - kubectl delete secret ${SECRET} -n ${NAMESPACE} || true - kubectl create secret generic ${SECRET} -n ${NAMESPACE} --from-file=id_rsa=.tmp/id_rsa --from-file=id_rsa.pub=.tmp/id_rsa.pub --from-file=authorized_keys=.tmp/id_rsa.pub - - # Which version of Kubeflow to use. - # For a list of releases refer to: - # https://github.com/kubeflow/kubeflow/releases - VERSION=master - - # Initialize a ksonnet app. Set the namespace for its default environment. - APP_NAME=openmpi - ks init ${APP_NAME} - cd ${APP_NAME} - ks env set default --namespace ${NAMESPACE} - - # Install Kubeflow components. - ks registry add kubeflow github.com/kubeflow/kubeflow/tree/${VERSION}/kubeflow - ks pkg install kubeflow/openmpi@${VERSION} - - # See the list of supported parameters. - - # Generate openmpi components. - COMPONENT=openmpi - IMAGE= - -#. Run openmpi workers in containers - - .. code-block:: console - - WORKERS= - MEMORY= - GPU=0 - - # We should create a hostfile with the names of each node in the k8s cluster - EXEC="mpiexec --allow-run-as-root -np ${WORKERS} --hostfile /kubeflow/openmpi/assets/hostfile -bind-to none -map-by slot sh -c 'python --device=cpu --data_format=NHWC --model=alexnet --variable_update=horovod --horovod_device=cpu'" - - ks generate openmpi ${COMPONENT} --image ${IMAGE} --secret ${SECRET} --workers ${WORKERS} --gpu ${GPU} --exec "${EXEC}" --memory "${MEMORY}" - - # Deploy to your cluster. - ks apply default - WORKERS= - MEMORY= - GPU=0 - - # We should create a hostfile with the names of each node in the k8s cluster - EXEC="mpiexec --allow-run-as-root -np ${WORKERS} --hostfile /kubeflow/openmpi/assets/hostfile -bind-to none -map-by slot sh -c 'python --device=cpu --data_format=NHWC --model=alexnet --variable_update=horovod --horovod_device=cpu'" - - ks generate openmpi ${COMPONENT} --image ${IMAGE} --secret ${SECRET} --workers ${WORKERS} --gpu ${GPU} --exec "${EXEC}" --memory "${MEMORY}" - - # Deploy to your cluster. - ks apply default - -Using Transformers* for Natural Language Processing -*************************************************** - -The DLRS v5.0 release includes `Transformers`_, a state-of-the-art Natural Language Processing (NLP) library for TensorFlow 2.0 and PyTorch. The library is configured to work within the container environment. - -In this section we use a Jupyter Notebook from inside the container to walk through one of the notebooks shown in the `Transformers`_ repository. - -To run the notebook, you will need to run the Deep Learning Reference Stack, mount it to disk and connect a Jupyter Notebook port. - - -#. Run the DLRS image with Docker: - - .. code-block:: bash - - docker run -it -v ${PWD}:/workspace -p 8888:8888 clearlinux/stacks-pytorch-mkl:latest - - -#. From within the container, navigate to the workspace, and clone the - transformers repository in the container: - - .. code-block:: bash - - cd workspace - git clone https://gist.github.com/16d38f2c9c688963c166c000330a3c11.git - - - -#. Start a Jupyter Notebook that is linked to the exterior port. - Be sure to copy the token from the output of starting Jupyter Notebook. - - .. code-block:: bash - - pip install jupyter --upgrade - jupyter notebook --ip 0.0.0.0 --no-browser --allow-root - -#. To access the Jupyter Notebook, open a browser. - -#. Return to the Terminal where you launched Jupyter Notebook. - Copy one of the URLs that appears after "Or copy and paste on of these URLs." - -#. Paste the URL (with embedded token) into the browser window. - - -The notebook will also be available at the URL of the system serving the notebook. For example if you are running on 192.168.1.10, you will be able to access the notebook from other systems on that subnet by navigating to \http://192.168.1.10:8888 - -From the browser, you will see the following notebooks. - -.. figure:: ../../_figures/stacks/dlrs-transformers-1.png - :scale: 80% - :alt: Transformers Jupyter Notebooks - - Figure 1: Transformers Jupyter Notebooks - - -This example along with the other notebooks show how to get up and running with Transformers. More detail on using Transformers* is available through the `Transformers`_ github repository. - - -Using the OpenVINO™ Model Optimizer -*********************************** - -The OpenVINO™ toolkit has two primary tools for deep learning, the inference engine and the model optimizer. The inference engine is integrated into the Deep Learning Reference Stack. It is better to use the model optimizer after training the model, and before inference begins. This example will explain how to use the model optimizer by going through a test case with a pre-trained TensorFlow model. - -This example uses resources found in the following OpenVINO™ toolkit documentation. - -`Converting a TensorFlow Model`_ - -`Converting TensorFlow Object Detection API Models`_ - -In this example, you will: - -* Download a TensorFlow model -* Clone the Model Optimizer -* Install Prerequisites -* Run the Model Optimizer - -#. Download a TensorFlow model - - We will be using an OpenVINO™ toolkit supported topology with the Model Optimizer. We will use a TensorFlow Inception V2 frozen model. - - Navigate to the `OpenVINO TensorFlow Model page`_. Then scroll down to the second section titled "Supported Frozen Topologies from TensorFlow Object Detection Models Zoo" and download "SSD Inception V2 COCO." - - Unpack the file into your chosen working directory. For example, if the tar file is in your Downloads folder and you have navigated to the directory you want to extract it into, run: - - .. code-block:: bash - - tar -xvf ~/Downloads/ssd_inception_v2_coco_2018_01_28.tar.gz - - -#. Clone the Model Optimizer - - Next we need the model optimizer directory, named `dldt`_. This example assumes the parent directory is on the same level as the model directory, ie: - - .. code-block:: console - - +--Working_Directory - +-- ssd_inception_v2_coco_2018_01_28 - +-- dldt - - - To clone the Model Optimizer, run this from inside the working directory: - - .. code-block:: bash - - git clone https://github.com/opencv/dldt.git - - - If you explore the :file:`dldt` directory, you'll see both the inference engine and the model optimizer. We are only concerned with the model optimizer at this stage. Navigating into the model optimizer folder you'll find several python scripts and text files. These are the scripts you call to run the model optimizer. - - -#. Install Prerequisites for Model Optimizer - - Install the Python packages required to run the model optimizer by running the script dldt/model-optimizer/install_prerequisites/install_prerequisites_tf.sh. - - .. code-block:: bash - - cd dldt/model-optimizer/install_prerequisites/ - ./install_prerequisites_tf.sh - cd ../../.. - - - -#. Run the Model Optimizer - - Running the model optimizer is as simple as calling the appropriate script, however there are many configuration options that are explained in the documentation - - .. code-block:: bash - - python dldt/model-optimizer/mo_tf.py \ - --input_model=ssd_inception_v2_coco_2018_01_28/frozen_inference_graph.pb \ - --tensorflow_use_custom_operations_config dldt/model-optimizer/extensions/front/tf/ssd_v2_support.json \ - --tensorflow_object_detection_api_pipeline_config ssd_inception_v2_coco_2018_01_28/pipeline.config \ - --reverse_input_channels - - - You should now see three files in your working directory, :file:`frozen_inference_graph.bin`, :file:`frozen_inference_graph.mapping`, and :file:`frozen_inference_graph.xml`. These are your new models in the Intermediate Representation (IR) format and they are ready for use in the OpenVINO™ Inference Engine. - - - -Using the OpenVINO™ toolkit Inference Engine -******************************************** - -This example walks through the basic instructions for using the inference engine. - -#. Starting the Model Server - - The process is similar to how we start `Jupter notebooks` on our containers - - Run this command to spin up a OpenVINO™ toolkit model fetched from GCP - - .. code-block:: bash - - docker run -p 8000:8000 stacks-dlrs-mkl:latest bash -c ". /workspace/scripts/serve.sh && ie_serving model --model_name resnet --model_path gs://public-artifacts/intelai_public_models/resnet_50_i8 --port 8000" - - - Once the server is setup, use a :command:`grpc` client to communicate with served model: - - .. code-block:: bash - - git clone https://github.com/IntelAI/OpenVINO-model-server.git - cd OpenVINO-model-server - pip install -q -r OpenVINO-model-server/example_client/client_requirements.txt - pip install --user -q -r OpenVINO-model-server/example_client/client_requirements.txt - cat OpenVINO-model-server/example_client/client_requirements.txt - cd OpenVINO-model-server/example_client - - python jpeg_classification.py --images_list input_images.txt --grpc_address localhost --grpc_port 8000 --input_name data --output_name prob --size 224 --model_name resnet - - - The results of these commands will look like this: - - .. code-block:: console - - start processing: - Model name: resnet - Images list file: input_images.txt - images/airliner.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 97.00 ms; speed 2.00 fps 10.35 - Detected: 404 Should be: 404 - images/arctic-fox.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 16.00 ms; speed 2.00 fps 63.89 - Detected: 279 Should be: 279 - images/bee.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 14.00 ms; speed 2.00 fps 69.82 - Detected: 309 Should be: 309 - images/golden_retriever.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 13.00 ms; speed 2.00 fps 75.22 - Detected: 207 Should be: 207 - images/gorilla.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 11.00 ms; speed 2.00 fps 87.24 - Detected: 366 Should be: 366 - images/magnetic_compass.jpeg (1, 3, 224, 224) ; data range: 0.0 : 247.0 - Processing time: 11.00 ms; speed 2.00 fps 91.07 - Detected: 635 Should be: 635 - images/peacock.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 9.00 ms; speed 2.00 fps 110.1 - Detected: 84 Should be: 84 - images/pelican.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 10.00 ms; speed 2.00 fps 103.63 - Detected: 144 Should be: 144 - images/snail.jpeg (1, 3, 224, 224) ; data range: 0.0 : 248.0 - Processing time: 10.00 ms; speed 2.00 fps 104.33 - Detected: 113 Should be: 113 - images/zebra.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0 - Processing time: 12.00 ms; speed 2.00 fps 83.04 - Detected: 340 Should be: 340 - Overall accuracy= 100.0 % - Average latency= 19.8 ms - - - -Using Seldon and OpenVINO™ model server with the Deep Learning Reference Stack -************************************************************************************* - -`Seldon Core`_ is an open source platform for deploying machine learning models on a Kubernetes cluster. In this section we will walk through using a Seldon server with OpenVINO™ model server. - -Pre-requisites -============== -* A running :ref:`kubernetes` cluster -* An existing Kubeflow deployment -* Helm -* A pre-trained model - -Please refer to: - -* :ref:`kubernetes` -* `Getting Started with Kubeflow`_ -* `Installing Helm`_ - - -.. note:: - - This document was validated with Kubernetes v1.14.8, Kubeflow v0.7, and Helm v3.0.1 - -Prepare the model -================= - -There are several methods to add a model to a Seldon server; we will cover two of them. First a model will be stored in a persistent volume by creating a persistent volume claim and a pod, then copying the model into the pod. Second, a model will be built directly into the base image. Adding a model to a volume is perhaps more traditional in Kubernetes, but some cloud providers have access rules that disallow a private cluster, and adding the model to the image avoids the issue in that scenario. - - -Mount pre-trained models into a persistent volume -------------------------------------------------- - -We will create a small pod to get the model into a volume. - -#. Apply all PV manifests to the cluster - - .. code-block:: bash - - kubectl apply -f storage/pv-volume.yaml - kubectl apply -f storage/model-store-pvc.yaml - kubectl apply -f storage/pv-pod.yaml - -#. Use :command:`kubectl cp` to move the model into the pod, and therefore into the volume - - .. code-block:: bash - - kubectl cp ./ pv-pod:/home - -#. In the running container, fetch your pre-trained models and save them in the :file:`/opt/ml` directory path. - - .. code-block:: bash - - root@hostpath-pvc:/# cd /opt/ml - root@hostpath-pvc:/# # Copy your models here - root@hostpath-pvc:/# # exit - - - -Add the pre-trained model to the image --------------------------------------- - -A custom DLRS image is provided to serve OpenVINO™ model server through Seldon. Add a curl command to download your publicly hosted model and save it in :file:`/opt/ml` in the container filesystem. For example, if you have a model on GCP, use this command: - - .. code-block:: bash - - curl -o "[SAVE_TO_LOCATION]" \ - "https://storage.googleapis.com/storage/v1/b/[BUCKET_NAME]/o/[OBJECT_NAME]?alt=media" - - -Prepare the DLRS image -====================== - -A base image with Seldon and the OpenVINO™ inference engine should be created using the :file:`Dockerfile_openvino_base` dockerfile. - - .. code-block:: bash - - cd docker - docker build -f Dockerfile_openvino_base -t dlrs_openvino_base . - cd .. - - -Deploy the model server -======================= - -Now you're ready to deploy the model server using the Helm chart provided. - - .. code-block:: bash - - cd helm - helm install dlrs-seldon seldon-model-server \ - --namespace kubeflow \ - --set openvino.image=dlrs_openvino_base \ - --set openvino.model.path=/opt/ml \ - --set openvino.model.name= \ - --set openvino.model.input=data \ - --set openvino.model.output=prob - - -This will create your SeldonDeployment - -Extended example with Seldon using Source to Image -================================================== - -`Source to Image (s2i)`_ is a tool to create docker images from source code. - -#. Install source to image (s2i) - - .. code-block:: bash - - cd ${SRC-DIR} - wget https://github.com/openshift/source-to-image/releases/download/v1.1.14/source-to-image-v1.1.14-874754de-linux-amd64.tar.gz - tar xf source-to-image-v1.1.14-874754de-linux-amd64.tar.gz - mv s2i ${BIN_DIR}/s2i && ln -s s2i ${BIN_DIR}/sti - -#. Clone the seldon-core repository - - .. code-block:: bash - - git clone https://github.com/SeldonIO/seldon-core.git ${SRC_DIR}/seldon-core - -#. Create the new image - - Using the DLRS image created above, you can build another image for deploying the Image Transformer component that consumes imagenet classificatin models. - - .. code-block:: bash - - cd ${SRC_DIR}/seldon-core/examples/models/openvino_imagenet_ensemble/resources/transformer/ - s2i -E environment_grpc . dlrs_openvino_base:0.1 imagenet_transformer:0.1 - - Use this newly created image for deploying the Image Transformer component of the `OpenVino Imagenet Pipelines`_ example from Seldon. - - -Use Jupyter Notebook -******************** - -This example uses the `PyTorch with OpenBLAS`_ container image. After it is -downloaded, run the Docker image with :command:`-p` to specify the shared port -between the container and the host. This example uses port 8888. - -.. code-block:: bash - - docker run --name pytorchtest --rm -i -t -p 8888:8888 clearlinux/stacks-pytorch-oss bash - -After you start the container, launch the Jupyter Notebook. This -command is executed inside the container image. - -.. code-block:: bash - - jupyter notebook --ip 0.0.0.0 --no-browser --allow-root - -After the notebook has loaded, you will see output similar to the following: - -.. code-block:: console - - To access the notebook, open this file in a browser: file:///.local/share/jupyter/runtime/nbserver-16-open.html - Or copy and paste one of these URLs: - http://(846e526765e3 or 127.0.0.1):8888/?token=6357dbd072bea7287c5f0b85d31d70df344f5d8843fbfa09 - -From your host system, or any system that can access the host's IP address, -start a web browser with the following. If you are not running the browser on -the host system, replace :command:`127.0.0.1` with the IP address of the host. - -.. code-block:: bash - - http://127.0.0.1:8888/?token=6357dbd072bea7287c5f0b85d31d70df344f5d8843fbfa09 - -Your browser displays the following: - -.. figure:: ../../_figures/stacks/dlrs-fig-1.png - :scale: 50% - :alt: Jupyter Notebook - - Figure 1: Jupyter Notebook - - -To create a new notebook, click :guilabel:`New` and select :guilabel:`Python 3`. - -.. figure:: ../../_figures/stacks/dlrs-fig-2.png - :scale: 50% - :alt: Create a new notebook - - Figure 2: Create a new notebook - -A new, blank notebook is displayed, with a cell ready for input. - -.. figure:: ../../_figures/stacks/dlrs-fig-3.png - :scale: 50% - :alt: New blank notebook - - Figure 3: New blank notebook - -To verify that PyTorch is working, copy the following snippet into the blank -cell, and run the cell. - -.. code-block:: console - - from __future__ import print_function - import torch - x = torch.rand(5, 3) - print(x) - -.. figure:: ../../_figures/stacks/dlrs-fig-4.png - :scale: 50% - :alt: Sample code snippet - - Figure 4: Sample code snippet - -When you run the cell, your output will look something like this: - -.. figure:: ../../_figures/stacks/dlrs-fig-5.png - :scale: 50% - :alt: Code output - - Figure 5: Code output - - -You can continue working in this notebook, or you can download existing -notebooks to take advantage of the Deep Learning Reference Stack's optimized -deep learning frameworks. Refer to `Jupyter Notebook`_ for details. - -Uninstallation -************** - -To uninstall the Deep Learning Reference Stack, you can choose to stop the -container so that it is not using system resources, or you can stop the -container and delete it to free storage space. - -To stop the container, execute the following from your host system: - -#. Find the container's ID - - .. code-block:: bash - - docker container ls - - This will result in output similar to the following: - - .. code-block:: console - - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - e131dc71d339 sysstacks/dlrs-tensorflow-clearlinux "/bin/sh -c 'bash'" 23 seconds ago Up 21 seconds oss - -#. You can then use the ID or container name to stop the container. This example - uses the name "oss": - - .. code-block:: bash - - docker container stop oss - - -#. Verify that the container is not running - - .. code-block:: bash - - docker container ls - - -#. To delete the container from your system you need to know the Image ID: - - .. code-block:: bash - - docker images - - This command results in output similar to the following: - - .. code-block:: console - - REPOSITORY TAG IMAGE ID CREATED SIZE - sysstacks/dlrs-tensorflow-clearlinux latest 82757ec1648a 4 weeks ago 3.43GB - sysstacks/dlrs-tensorflow-clearlinux latest 61c178102228 4 weeks ago 2.76GB - -#. To remove an image use the image ID: - - .. code-block:: bash - - docker rmi 82757ec1648a - - .. code-block:: console - - # docker rmi 827 - Untagged: sysstacks/dlrs-tensorflow-clearlinux:latest - Untagged: sysstacks/dlrs-tensorflow-clearlinux@sha256:381f4b604537b2cb7fb5b583a8a847a50c4ed776f8e677e2354932eb82f18898 - Deleted: sha256:82757ec1648a906c504e50e43df74ad5fc333deee043dbfe6559c86908fac15e - Deleted: sha256:e47ecc039d48409b1c62e5ba874921d7f640243a4c3115bb41b3e1009ecb48e4 - Deleted: sha256:50c212235d3c33a3c035e586ff14359d03895c7bc701bb5dfd62dbe0e91fb486 - - - Note that you can execute the :command:`docker rmi` command using only the first few characters of the image ID, provided they are unique on the system. - -#. Once you have removed the image, you can verify it has been deleted with: - - .. code-block:: bash - - docker images - -Compiling AIXPRT for DLRS -************************* - -To compile AIXPRT for DLRS, you will have to get the community edition of AIXPRT and update the `compile_AIXPRT_source.sh` file. AIXPRT utilizes -build configuration files, so to build AIXPRT in the DLRS image, copy the build files from the base image by adding these commands -to the end of the stacks-dlrs-mkl dockerfile: - - .. code-block:: console - - COPY --from=base /dldt/inference-engine/bin/intel64/Release/ /usr/local/lib/openvino/tools/ - COPY --from=base /dldt/ /dldt/ - COPY ./airxprt/ /workspace/aixprt/ - RUN ./aixprt/install_deps.sh - RUN ./aixprt/install_aixprt.sh - - -AIXPRT requires OpenCV. On |CL|, the OpenCV bundle also installs the DLDT components. To use AIXPRT in the DLRS environment you need to either remove the shared libraries for DLDT from :file:`/usr/lib64` before you run the tests, or ensure that the DLDT components in the :file:`/usr/local/lib` are being used for AIXPRT. This can be achieved using adding LD_LIBRARY_PATH environment variable before testing. - - .. code-block:: bash - - export LD_LIBRARY_PATH=/usr/local/lib - - -The updates to the AIXPRT community edition have been captured in the diff file :file:`compile_AIXPRT_source.sh.patch`. The core of these changes relate to the version of model files(2019_R1) we download from the `OpenCV open model zoo`_ and location of the build files, which in our case is `/dldt`. Please refer to the patch files and make changes as necessary to the compile_AIXPRT_source.sh file as required for your environment. - - -Related topics -************** - -* `TensorFlow Benchmarks`_ -* `PyTorch benchmarks`_ -* `Kubeflow`_ -* :ref:`kubernetes` tutorial -* `Jupyter Notebook`_ - - -*Intel, OpenVINO, Xeon, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* - -.. _TensorFlow: https://www.tensorflow.org/ - -.. _Kubeflow: https://www.kubeflow.org/ - -.. _Docker Hub: https://hub.docker.com/ - -.. _TensorFlow Benchmarks: https://github.com/tensorflow/benchmarks - -.. _PyTorch benchmarks: https://github.com/pytorch/pytorch/blob/master/caffe2/python/convnet_benchmarks.py - -.. _Creating a single control-plane cluster with kubeadm: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/ - -.. _flannel: https://github.com/coreos/flannel - -.. _Getting Started with Kubeflow: https://github.intel.com/verticals/usecases/blob/56717f4642ecd958dc93bbc361c551dfc578d3ed/kubeflow/README.md#getting-started-with-kubeflow - -.. _TensorFlow Eigen: https://hub.docker.com/r/sysstacks/dlrs-tensorflow-clearlinux:v0.6.0-oss - -.. _TensorFlow oneDNN: https://hub.docker.com/r/sysstacks/dlrs-tensorflow2-clearlinux:v0.6.0 - -.. _PyTorch with OpenBLAS: https://hub.docker.com/r/sysstacks/dlrs-pytorch-clearlinux:v0.6.0-oss - -.. _PyTorch with Intel oneDNN: https://hub.docker.com/r/sysstacks/dlrs-pytorch-clearlinux:v0.6.0 - -.. _Intel oneDNN: https://hub.docker.com/r/sysstacks/dlrs-tensorflow-clearlinux - -.. _DLRS V3.0: https://clearlinux.org/stacks/deep-learning-reference-stack-v3 - -.. _DLRS V4.0: https://clearlinux.org/news-blogs/deep-learning-reference-stack-v4 - -.. _DLRS V5.0: https://clearlinux.org/blogs-news/deep-learning-reference-stack-v50-now-available - -.. _DLRS V6.0: https://clearlinux.org/blogs-news/deep-learning-reference-stack-v6-now-available - -.. _dlrs-tfjob: github.com/intel/stacks - -.. _Logging Architecture: https://kubernetes.io/docs/concepts/cluster-administration/logging/ - -.. _DLRS V1.0: https://clearlinux.org/stacks/deep-learning-reference-stack - -.. _DLRS V2.0: https://clearlinux.org/stacks/deep-learning-reference-stack-pytorch - -.. _Jupyter Notebook: https://jupyter.org/ - -.. _kubectl overview: https://kubernetes.io/docs/reference/kubectl/overview/ - -.. _launcher.py: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow - -.. _DLRS Terms of Use: https://clearlinux.org/stacks/deep-learning/terms-of-use - -.. _DLRS Release notes: https://github.com/intel/stacks/tree/master/dlrs - -.. _Seldon Core: https://docs.seldon.io/projects/seldon-core/en/latest/ - -.. _Istio: https://github.com/kubeflow/manifests/blob/master/kfdef/kfctl_k8s_istio.yaml - -.. _Dockerfile_openvino_base: https://github.com/clearlinux/dockerfiles/blob/master/stacks/dlrs/kubeflow/dlrs-seldon/docker/Dockerfile_openvino_base - -.. _TFJob: https://www.kubeflow.org/docs/components/tftraining - -.. _Arrikto: https://www.kubeflow.org/docs/started/k8s/kfctl-existing-arrikto/ - -.. _kfctl tarball: https://github.com/kubeflow/kubeflow/releases/download/v0.6.1/kfctl_v0.6.1_linux.tar.gz - -.. _MetalLB: https://metallb.universe.tf/ - -.. _Kubeflow documentation: https://www.kubeflow.org/docs/components/tftraining/#what-is-tfjob - -.. _Distributed TensorFlow: https://www.tensorflow.org/deploy/distributed -.. _TFJobs: https://www.kubeflow.org/docs/components/tftraining/ - -.. _Intel® AI Quantization Tools for TensorFlow: https://github.com/IntelAI/tools/blob/master/tensorflow_quantization/README.md#quantization-tools - -.. _OpenCV open model zoo: https://github.com/opencv/open_model_zoo - -.. _PyTorch Job: https://www.kubeflow.org/docs/components/pytorch/ - -.. _Converting a TensorFlow Model: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html - -.. _Converting TensorFlow Object Detection API Models: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html - -.. _OpenVINO TensorFlow Model page: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html - -.. _dldt: https://github.com/opencv/dldt - -.. _DLRS TFJob: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob - -.. _DLRS PytorchJob: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-pytorchjob - -.. _Installing Helm: https://helm.sh/docs/intro/install/ - -.. _OpenVino Imagenet Pipelines: https://docs.seldon.io/projects/seldon-core/en/stable/examples/openvino_ensemble.html - -.. _Source to Image (s2i): https://docs.seldon.io/projects/seldon-core/en/latest/wrappers/s2i.html - -.. _Deep Learning Reference Stack website: https://clearlinux.org/stacks/deep-learning - -.. _Horovod: https://github.com/horovod/horovod - -.. _OpenMPI Project: https://www.open-mpi.org - -.. _Kubeflow OpenMPI instructions: https://github.com/kubeflow/mpi-operator/blob/master/README.md - -.. _Intel stacks GitHub repository: https://github.com/intel/stacks.git - -.. _Transformers: https://github.com/huggingface/transformers - -.. _oneDNN: https://github.com/oneapi-src/oneDNN diff --git a/source/guides/stacks/greengrass.rst b/source/guides/stacks/greengrass.rst deleted file mode 100644 index 4feb21c6..00000000 --- a/source/guides/stacks/greengrass.rst +++ /dev/null @@ -1,430 +0,0 @@ -.. _greengrass: - -Enable AWS Greengrass\* and OpenVINO™ toolkit -############################################# - -This guide explains how to enable AWS Greengrass\* and OpenVINO™ toolkit. -Specifically, the guide demonstrates how to: - -* Set up the Intel edge device with |CL-ATTR| -* Install the OpenVINO™ toolkit and Amazon Web Services\* (AWS\*) - Greengrass\* software stacks -* Use AWS Greengrass\* and AWS Lambda\* to deploy the FaaS samples from - the cloud - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -Hardware accelerated Function-as-a-Service (FaaS) enables cloud developers to -deploy inference functionalities [1] on Intel IoT edge devices with -accelerators (CPU, Integrated GPU, Intel® FPGA, and Intel® Movidius™ -technology). These functions provide a great developer experience and -seamless migration of visual analytics from cloud to edge in a secure manner -using a containerized environment. Hardware-accelerated FaaS provides the -best-in-class performance by accessing optimized deep learning libraries on -Intel IoT edge devices with accelerators. - -Supported platforms -******************* - -* Operating System: |CL| latest release -* Hardware: Intel® Core™ processors (that support inference on CPU only) - -Sample description -================== - -The AWS Greengrass samples are located at `Edge-Analytics-FaaS`_. This -guide uses the 1.0 version of the source code. - -|CL| provides the following AWS Greengrass samples: - -* `greengrass_classification_sample.py`_ - - This AWS Greengrass sample classifies a video stream using classification - networks such as AlexNet and GoogLeNet and publishes top-10 results on AWS\* - IoT Cloud every second. - -* `greengrass_object_detection_sample_ssd.py`_ - - This AWS Greengrass sample detects objects in a video stream and - classifies them using single-shot multi-box detection (SSD) networks such - as SSD Squeezenet, SSD Mobilenet, and SSD300. This sample publishes - detection outputs such as class label, class confidence, and bounding box - coordinates on AWS IoT Cloud every second. - - -Install the OS on the edge device -********************************* - -Start with a clean installation of |CL| on a new system, using the -:ref:`bare-metal-install-desktop`, found in :ref:`get-started`. - -Create user accounts -==================== - -After |CL| is installed, create two user accounts. Create an administrative -user in |CL| and create a user account for the Greengrass services to use ( -see Greengrass user below). - -#. Create a new user and set a password for that user. Enter the following - commands as ``root``: - - .. code-block:: bash - - useradd - passwd - -#. Next, enable the :command:`sudo` command for your new . Add - to the `wheel` group: - - .. code-block:: bash - - usermod -G wheel -a - -#. Create a :file:`/etc/fstab` file. - - .. code-block:: bash - - touch /etc/fstab - - .. note:: - - By default, |CL| does not create an :file:`/etc/fstab` file. - You must create this file before the Greengrass service runs. - -Add required bundles -==================== - -Use the :command:`swupd` software updater utility to add the prerequisite bundles -for the OpenVINO software stack: - -.. code-block:: bash - - swupd bundle-add os-clr-on-clr desktop-autostart computer-vision-basic - -.. note:: - - Learn more about how to :ref:`swupd-guide`. - -The :command:`computer-vision-basic` bundle installs the OpenVINO™ toolkit, -and the sample models optimized for Intel edge platforms. - -.. _convert-dl-models: - -Convert deep learning models -============================ - -Locate sample models --------------------- - -There are two types of provided models that can be used in conjunction with -AWS Greengrass for this guide: classification or object detection. - -To complete this guide using an image classification model, -download the BVLC AlexNet model files `bvlc_alexnet.caffemodel`_ and -`deploy.prototxt`_ to the default model_location at -:file:`/usr/share/openvino/models`. Any custom pre-trained classification models -can be used with the classification sample. - -For object detection, the sample models optimized for Intel edge platforms -are included with the computer-vision-basic bundle installation at -:file:`/usr/share/openvino/models`. These models are provided as an example; -you may also use a custom SSD model with the Greengrass object detection sample. - -Run model optimizer -------------------- - -Follow the instructions in the `Model Optimizer Developer Guide`_ for converting -deep learning models to Intermediate Representation using Model Optimizer. To -optimize either of the sample models described above, run one of the following commands. - -For classification using BVLC AlexNet model: - -.. code-block:: bash - - python3 mo.py --framework caffe --input_model - /bvlc_alexnet.caffemodel --input_proto - /deploy.prototxt --data_type --output_dir - --input_shape [1,3,227,227] - -For object detection using SqueezeNetSSD-5Class model: - -.. code-block:: bash - - python3 mo.py --framework caffe --input_model - /'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.caffemodel - --input_proto /'SqueezeNet 5-Class detection'/SqueezeNetSSD-5Class.prototxt - --data_type --output_dir - -In these examples: - -* `` is :file:`/usr/share/openvino/models`. - -* `` is FP32 or FP16, depending on target device. - -* `` is the directory where the Intermediate Representation - (IR) is stored. IR contains .xml format corresponding to the network - structure and .bin format corresponding to weights. This .xml file should be - passed to :command:``. - -* In the BVLC AlexNet model, the prototxt defines the input shape with - batch size 10 by default. In order to use any other batch size, the - entire input shape must be provided as an argument to the model - optimizer. For example, to use batch size 1, you must provide: - `--input_shape [1,3,227,227]` - - -Configure AWS Greengrass group -****************************** - -For each Intel edge platform, you must create a new AWS Greengrass group -and install AWS Greengrass core software to establish the connection between -cloud and edge. - -#. To create an AWS Greengrass group, follow the instructions in - `Configure AWS IoT Greengrass on AWS IoT`_. - -#. To install and configure AWS Greengrass core on edge platform, follow - the instructions in `Start AWS Greengrass on the Core Device`_. In - step 8(b), download the x86_64 Ubuntu\* configuration of the AWS Greengrass - core software. - - .. note:: - - You do not need to run the :file:`cgroupfs-mount.sh` script in step #6 - of Module 1 of the `AWS Greengrass Developer Guide`_ because this is - enabled already in |CL|. - -#. Be sure to download both the security resources and the AWS Greengrass - core software. - - .. note:: - - Security certificates are linked to your AWS account. - - -Create and package Lambda function -********************************** - -#. Complete steps 1-4 of the AWS Greengrass guide at - `Create and Package a Lambda Function`_. - - .. note:: - - This creates the tarball needed to create the AWS Greengrass - environment on the edge device. - - -#. In step 5, replace :file:`greengrassHelloWorld.py` with the classification or - object detection Greengrass sample from `Edge-Analytics-Faas`_: - - * Classification: `greengrass_classification_sample.py`_ - - * Object Detection: `greengrass_object_detection_sample_ssd.py`_ - -#. Zip the selected Greengrass sample with the extracted Greengrass SDK folders - from the previous step into :file:`greengrass_sample_python_lambda.zip`. - - The zip should contain: - - * greengrasssdk - - * greengrass classification or object detection sample - - For example: - - .. code-block:: bash - - zip -r greengrass_lambda.zip greengrasssdk - greengrass_object_detection_sample_ssd.py - -#. Return to the AWS documentation section called - `Create and Package a Lambda Function`_ and complete the procedure. - - .. note:: - - In step 9(a) of the AWS documentation, while uploading the zip file, - make sure to name the handler to one of the following, depending on the - AWS Greengrass sample you are using: - - * greengrass_object_detection_sample_ssd.function_handler - * greengrass_classification_sample.function_handler - - -Configure Lambda function -************************* - -After creating the Greengrass group and the Lambda function, start -configuring the Lambda function for AWS Greengrass. - -#. Follow steps 1-8 in `Configure the Lambda Function for AWS IoT Greengrass`_ - in the AWS documentation. - -#. In addition to the details mentioned in step 8, change the Memory limit - to 2048 MB to accommodate large input video streams. - -#. Add the following environment variables as key-value pairs when editing - the Lambda configuration and click on update: - - .. list-table:: **Table 1. Environment variables: Lambda configuration** - :widths: 20 80 - :header-rows: 1 - - * - Key - - Value - * - PARAM_MODEL_XML - - /, where is user specified and - contains IR.xml, the Intermediate Representation file from the - OpenVINO™ Model Optimizer. - For this guide, should be set to '/usr/share/openvino/models' - or one of its subdirectories. - * - PARAM_INPUT_SOURCE - - /input.webm to be specified by user. Holds both input and - output data. For webcam, set PARAM_INPUT_SOURCE to ‘/dev/video0’ - * - PARAM_DEVICE - - "CPU" - * - PARAM_CPU_EXTENSION_PATH - - /usr/lib64/libcpu_extension.so - * - PARAM_OUTPUT_DIRECTORY - - to be specified by user. Holds both input and output - data - * - PARAM_NUM_TOP_RESULTS - - User specified for classification sample. - (e.g. 1 for top-1 result, 5 for top-5 results) - -#. Add subscription to subscribe, or publish messages from AWS Greengrass - Lambda function by completing the procedure in `Configure the Lambda Function for AWS IoT Greengrass`_. - - .. note:: - - The optional topic filter field is the topic mentioned inside the Lambda function. In this guide, sample topics include the following: - :command:`openvino/ssd` or :command:`openvino/classification` - -Add local resources -=================== - -Refer to the AWS documentation `Access Local Resources with Lambda Functions and Connectors`_ -for details about local resources and access privileges. - -The following table describes the local resources needed for the CPU: - -.. list-table:: **Local resources** - :widths: 20, 20, 20, 20 - :header-rows: 1 - - * - Name - - Resource type - - Local path - - Access - - * - ModelDir - - Volume - - to be specified by user - - Read-Only - - * - Webcam - - Device - - /dev/video0 - - Read-Only - - * - DataDir - - Volume - - to be specified by user. Holds both input and output - data. - - Read and Write - -Deploy Lambda function -********************** - -Refer to the AWS documentation `Deploy Cloud Configurations to an AWS IoT Greengrass Core Device`_ for instructions on how to deploy the lambda function to AWS -Greengrass core device. Select *Deployments* on the group page and follow the instructions. - -Output consumption -================== - -There are four options available for output consumption. These options are -used to report, stream, upload, or store inference output at an interval -defined by the variable :command:`reporting_interval` in the AWS Greengrass samples. - -#. IoT cloud output: - - This option is enabled by default in the AWS Greengrass samples using the - :command:`enable_iot_cloud_output` variable. You can use it to verify the lambda - running on the edge device. It enables publishing messages to IoT cloud - using the subscription topic specified in the lambda. (For example, topics - may include :command:`openvino/classification` for classification and :command:`openvino/ssd` - for object detection samples.) For classification, top-1 result with class - label are published to IoT cloud. For SSD object detection, detection - results such as bounding box coordinates of objects, class label, and - class confidence are published. - - Refer to the AWS documentation - `Verify the Lambda Function Is Running on the Device`_ for instructions on - how to view the output on IoT cloud. - -#. Kinesis streaming: - - This option enables inference output to be streamed from the edge device - to cloud using Kinesis [3] streams when :command:`enable_kinesis_output` is set - to True. The edge devices act as data producers and continually push - processed data to the cloud. You must set up and specify - Kinesis stream name, Kinesis shard, and AWS region in the AWS Greengrass - samples. - -#. Cloud storage using AWS S3 bucket: - - When the :command:`enable_s3_jpeg_output` variable is set to True, it enables - uploading and storing processed frames (in jpeg format) in an AWS S3 - bucket. You must set up and specify the S3 bucket name in the AWS - Greengrass samples to store the JPEG images. The images are named using the - timestamp and uploaded to S3. - -#. Local storage: - - When the :command:`enable_s3_jpeg_output` variable is set to True, it enables - storing processed frames (in jpeg format) on the edge device. The images - are named using the timestamp and stored in a directory specified by - :command:`PARAM_OUTPUT_DIRECTORY`. - -References -********** - -#. AWS Greengrass: https://aws.amazon.com/greengrass/ -#. AWS Lambda: https://aws.amazon.com/lambda/ -#. AWS Kinesis: https://aws.amazon.com/kinesis/ - - -*Intel, OpenVINO, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* - - -.. _Edge-Analytics-FaaS: https://github.com/intel/Edge-Analytics-FaaS/tree/v1.0/AWS%20Greengrass - -.. _bvlc_alexnet.caffemodel: http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel - -.. _deploy.prototxt: https://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/deploy.prototxt - -.. _greengrass_classification_sample.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_classification_sample.py - -.. _greengrass_object_detection_sample_ssd.py: https://github.com/intel/Edge-Analytics-FaaS/blob/v1.0/AWS%20Greengrass/greengrass_object_detection_sample_ssd.py - -.. _Model Optimizer Developer Guide: https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer - -.. _AWS Greengrass Developer Guide: https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html - -.. _Configure AWS IoT Greengrass on AWS IoT: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-config.html - -.. _Start AWS Greengrass on the Core Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/gg-device-start.html - -.. _Configure the Lambda Function for AWS IoT Greengrass: https://docs.aws.amazon.com/greengrass/latest/developerguide/config-lambda.html - -.. _Access Local Resources with Lambda Functions and Connectors: https://docs.aws.amazon.com/greengrass/latest/developerguide/access-local-resources.html - -.. _Deploy Cloud Configurations to an AWS IoT Greengrass Core Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/configs-core.html - -.. _Verify the Lambda Function Is Running on the Device: https://docs.aws.amazon.com/greengrass/latest/developerguide/lambda-check.html - -.. _Create and Package a Lambda Function: https://docs.aws.amazon.com/greengrass/latest/developerguide/create-lambda.html diff --git a/source/guides/stacks/hpcrs.rst b/source/guides/stacks/hpcrs.rst deleted file mode 100644 index ebf5b640..00000000 --- a/source/guides/stacks/hpcrs.rst +++ /dev/null @@ -1,242 +0,0 @@ -.. _hpcrs: - -High Performance Computing Reference Stack -########################################## - -This guide gives examples for converting Docker* containers, such as those provided by the :ref:`dlrs` into Singularity* containers suited for HPC, and then walking through a multi-node benchmarking example with TensorFlow*. - -.. contents:: - :local: - :depth: 1 - - -Overview -******** - -The High Performance Computing Reference Stack (HPCRS) meets the needs of deploying HPC and AI workloads on the same system. This software solution reduces the complexities associated with integrating software components for High Performance Computing (HPC) Platforms. `Singularity`_ is an open source container platform to package entire scientific workflows, software and libraries, and even data. - - -Installing Singularity -********************** -The installation instructions are for Linux* systems, and have been enabled for installation on |CL-ATTR|. - -.. note:: - - The steps for installation can also be found on the `Singularity quick-start`_ https://sylabs.io/guides/3.0/user-guide/quick_start.html#quick-installation - -#. Install Go*. - - This guide requires version 1.13 of Go, for compatibility with Singularity v3.0.0. Please use these steps to ensure the correct version of Go is installed: - - .. code-block:: bash - - $ export VERSION=1.13 OS=linux ARCH=amd64 && \ - wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \ - sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ - rm go$VERSION.$OS-$ARCH.tar.gz - -#. Setup the environment for Go. - - .. code-block:: bash - - echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \ - echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \ - source ~/.bashrc - -#. Install :command:`dep` for dependency resolution with Singularity v3.0.0. - - .. code-block:: bash - - go get -u github.com/golang/dep/cmd/dep - -#. Download Singularity. - - .. code-block:: bash - - go get -d github.com/sylabs/singularity - - - .. note:: - - Go will complain that there are no Go files, but it will still download the Singularity source code to the appropriate directory within the $GOPATH. - -#. Checkout version 3.0.0 of Singularity. - - .. code-block:: bash - - export VERSION=v3.0.3 # or another tag or branch if you like && \ - cd $GOPATH/src/github.com/sylabs/singularity && \ - git fetch && \ - git checkout $VERSION # omit this command to install the latest bleeding edge code from master - -#. Build Singularity. - - Singularity uses a custom build system called makeit. mconfig is called to generate a Makefile and then make is used to compile and install. - The devpkg-openssl, devpkg-util-linux package may be required and can be installed using the :command:`sudo swupd bundle-add `. - - .. code-block:: bash - - ./mconfig && \ - make -C ./builddir && \ - sudo make -C ./builddir install - - -#. Configure bash completion (optional). - - To enjoy bash completion with Singularity commands and options, source the bash completion file. Add this command to your ~/.bashrc file so that bash completion continues to work in new shells - - .. code-block:: bash - - . /usr/local/etc/bash_completion.d/singularity - - -Converting Docker images to Singularity Images -********************************************** - -#. Download d2s. - - :command:`d2s` os an open source tool to convert Docker images to - Singularity images. You can use the script in the location where it is - downloaded, or install it using the included :file:`setup.py` file with - the :command:`python setup.py install` - - - .. code-block:: bash - - git clone https://github.com/intel/stacks.git - cd stacks/hpcrs/d2s - -#. List local Docker images. - - .. code-block:: bash - - python d2s.py --list_docker_images - - Your output can appear like this: - - .. code-block:: console - - ============================== - Docker images present locally - ============================== - ID NAME - 0: clearlinux/stacks-dlrs-mkl - 1: clearlinux/stacks-dlrs_2-mkl - ============================== - -#. Convert to Singularity images. - - To convert the Docker images to Singularity images, use the :command:`d2s` - script with the ID numbers of the images you wish to convert. - We strongly recommend using one of the :file:`clearlinux/stacks-dlrs-mkl` - or :file:`sysstacks/stacks-dlrs-mkl` based images for this guide. Other - images may be incompatible with expected configuration or filesystem - options. - - .. code-block:: bash - - python d2s.py --convert_docker_images - -#. Use the Singularity image. - - To use the container shell to run workloads, launch the image and you - will be dropped into the shell. The Singularity image name will be the - same as the name of the Docker image, with slashes converted to - underscores. - - .. code-block:: bash - - singularity shell - - Using the example output above, after conversion you could launch the clearlinux/stacks-dlrs-mkl Singularity image with - :command:`singularity shell clearlinux_stacks-dlrs-mkl` - -Execute a multi-node benchmark on an HPC cluster -************************************************ -The following example was executed on an Intel(r) Xeon(r) Processor-based -HPC infrastructure. The following steps may need to be adjusted for -different environments. See this `Intel Whitepaper`_ for more information. - -Running a ResNet50 workload multi-node --------------------------------------- - -#. Download the TensorFlow benchmark. - - .. code-block:: bash - - git clone http://github.com/tensorflow/benchmarks -b cnn_tf_v1.13_compatible - -#. Copy the Singularity image and the benchmark files to the HPC cluster - environment. - -#. Install OpenMPI* if needed. - - .. note:: - - If the HPC host does not have OpenMPI installed, install a custom - local version in the user's home directory. This version must be the - same as the version installed in the DLRS container. Follow the steps - for `building OpenMPI`_ from their documentation. - -#. Adjust PATH variables. - - Include the OpenMPI install locations in the PATH and LD_LIBRARY_PATH - environment variables. - - .. code-block:: bash - - export PATH="$PATH:/bin" - export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/lib/" - -#. Execute the TF benchmark script on single or multiple nodes using OpenMPI - through the :command:`mpirun` command. Replace variables in {} braces to - reflect your environment. - - .. code-block:: bash - - mpirun --np ${NUM_COPIES} \ - -bind-to none \ - -map-by slot \ - --display-map \ - -host ${HOSTNAMES} \ - --report-bindings \ - --oversubscribe \ - -x LD_LIBRARY_PATH \ - -x PATH \ - -x HOROVOD_FUSION_THRESHOLD \ - -x OMP_NUM_THREADS=${OMP_NUM_THREADS} \ - singularity exec ${PATH_TO_SING_IMAGE} \ - python ${PATH_TO_TF_BENCH}/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py \ - --batch_size=128 \ - --model=resnet50 \ - --num_intra_threads=${NUM_INTRA_THREADS} \ - --num_inter_threads=${NUM_INTER_THREADS} \ - --data_format=NHWC \ - --device=cpu \ - --variable_update=horovod \ - --horovod_device=cpu - - .. note:: - - Refer to the `DLRS script`_ for recommended values for setting environment variables in the :command:`mpirun` command. - - .. note:: - - You may see an error regarding a missing library while executing the DLRS container. - “tensorflow.python.framework.errors_impl.NotFoundError: libnuma.so.1: cannot open shared object file: No such file or directory” - - A workaround for this error is to bind the path to the library from the host. - - .. code-block:: bash - - --bind /usr/lib64/libnuma.so.1:/usr/lib64/libnuma.so.1 - -.. _Singularity: https://sylabs.io/ - -.. _Singularity quick-start: https://sylabs.io/guides/3.0/user-guide/quick_start.html#quick-installation - -.. _Intel Whitepaper: https://www.intel.com/content/www/us/en/artificial-intelligence/solutions/best-known-methods-for-scaling-deep-learning-with-tensorflow-on-xeon-processor-based-clusters.html - -.. _building OpenMPI: https://www.open-mpi.org/faq/?category=building#easy-build - -.. _DLRS script: https://github.com/intel/stacks/blob/master/dlrs/clearlinux/tensorflow_2/mkl/scripts/set_env.sh diff --git a/source/guides/stacks/mers.rst b/source/guides/stacks/mers.rst deleted file mode 100644 index d1ac448e..00000000 --- a/source/guides/stacks/mers.rst +++ /dev/null @@ -1,564 +0,0 @@ -.. _mers: - -Media Reference Stack -##################### - -The Media Reference Stack (MeRS) is a highly optimized software stack for -Intel® Architecture Processors (the CPU) and Intel® Processor Graphics (the -GPU) to enable media prioritized workloads, such as transcoding and analytics. - -This guide explains how to use the pre-built |MERS| container image, build -your own |MERS| container image, and use the reference stack. - -.. contents:: - :local: - :depth: 1 - -Overview -******** - -Developers face challenges due to the complexity of software integration for -media tasks that require investing time and engineering effort. -For example: - - * Finding the balance between quality and performance. - * Understanding available standard-compliant encoders. - * Optimizing across the hardware-software stack for efficiency. - -|MERS| abstracts away the complexity of integrating multiple software -components and specifically tunes them for Intel platforms. |MERS| enables -media and visual cloud developers to deliver experiences using a simple -containerized solution. - - -Releases -******** - -Refer to the `System Stacks for Linux* OS repository -`_ for information and download links for the -different versions and offerings of the stack. - -* MeRS V0.2.0 release announcement including media processing on GPU and - analytics on CPU. - -* MeRS V0.1.0 including media processing and analytics CPU. - -* `MeRS Release notes on Github* - `_ for the - latest release of Deep Learning Reference Stack - - -Prerequisites -============= - -|MERS| can run on any host system that supports Docker\*. This guide uses -|CL-ATTR| as the host system. - -- To install |CL| on a host system, see how to - :ref:`install Clear Linux* OS from the live desktop - `. - -- To install Docker* on a |CL| host system, see - the :ref:`instructions for installing Docker* `. - -.. important:: - - For optimal media analytics performance, a processor with Vector Neural - Network Instructions (VNNI) should be used. VNNI is an extension of Intel® - Advanced Vector Extensions 512 (Intel® AVX-512) and is available starting - with the 2nd generation of Intel® Xeon® Scalable processors, providing AI - inference acceleration. - -Stack features -============== - -The |MERS| provides a `pre-built Docker image available on DockerHub -`_, which includes -instructions on building the image from source. |MERS| is open-sourced to -make sure developers have easy access to the source code and are able to -customize it. |MERS| is built using the latest *clearlinux/os-core* Docker -image and aims to support the latest |CL| version. - -|MERS| provides the following libraries and drivers: - -.. list-table:: - :widths: 15 85 - - * - SVT-HEVC - - Scalable Video Technology for HEVC encoding, also known as H.265 - * - SVT-AV1 - - Scalable Video Technology for AV1 encoding - * - x264 - - x264 for H.264/MPEG-4 AVC encoding - * - dav1d - - `dav1d `_ for AV1 decoding - * - libVA - - `VAAPI (Video Acceleration API) open-source library (LibVA), - `_ which provides access to graphics - hardware acceleration capabilities. - * - media-driver - - `Intel® Media Driver for VAAPI `_ - for supporting hardware acceleration on Intel® Gen graphics hardware - platforms. - * - gmmlib - - `Intel® Graphics Memory Management Library - `_ provides device specific and buffer - management for the Intel® Graphics Compute Runtime for oneAPI Level Zero - and OpenCL™ Driver and the Intel Media Driver for VAAPI. - -Components of the |MERS| include: - -* |CL| as a base for performance and security. - -* `OpenVINO™ toolkit - `_ for inference. - -* `FFmpeg* `_ with plugins for: - - - `Scalable Video Technology (SVT) - `_ - -* `GStreamer* `_ with plugins for: - - - `Scalable Video - Technology (SVT) `_ - - `OpenVINO™ toolkit - `_ - - `VAAPI `_ - -* `Intel® Media SDK `_ - -.. note:: - - The |MERS| is validated on 11th generation Intel Processor Graphics and - newer. Older generations should work but are not tested against. - -.. note:: - - The pre-built |MERS| container image configures FFmpeg without certain - elements (specific encoder, decoder, muxer, etc.) that you may require. If - you require changes to FFmpeg we suggest starting at - :ref:`building-the-mers-container-image`. - -.. note:: - - The Media Reference Stack is a collective work, and each piece of software - within the work has its own license. Please see the `MeRS Terms of Use - `_ for more details about - licensing and usage of the Media Reference Stack. - - -Get the pre-built |MERS| container image -**************************************** - -Pre-built |MERS| Docker images are available on DockerHub* at -https://hub.docker.com/r/sysstacks/mers-clearlinux - - -To use the |MERS|: - -#. Pull the image directly from `Docker Hub - `_. - - .. code-block:: bash - - docker pull sysstacks/mers-clearlinux - - .. note :: - - The |MERS| docker image is large in size and will take some time to - download depending on your Internet connection. - - If you are on a network with outbound proxies, be sure to configure - Docker to allow access. See the `Docker service proxy - `_ and - `Docker client proxy - `_ - documentation for more details. - -#. Once you have downloaded the image, run it using the following command: - - .. code-block:: bash - - docker run -it sysstacks/mers-clearlinux - - This will launch the image and drop you into a bash shell inside the - container. GStreamer and FFmpeg programs are installed in the container - image and accessible in the default $PATH. Use these programs as you would - outside of |MERS|. - - Paths to media files and video devices, such as cameras, can be shared from - the host to the container with the :command:`--volume` switch `using Docker - volumes `_. - -.. _building-the-mers-container-image: - -Build the |MERS| container image from source -******************************************** - -If you choose to build your own MeRS container image, you can optionally add -customizations as needed. The :file:`Dockerfile` for the MeRS is available on -`GitHub `_ and can be used -as a reference when creating your own container image. - -#. The |MERS| image is part of the dockerfiles repository inside the |CL| - organization on GitHub. Clone the :file:`stacks` repository. - - .. code-block:: bash - - git clone https://github.com/intel/stacks.git - -#. Navigate to the :file:`stacks/mers/clearlinux` directory which contains - the Dockerfile for the |MERS|. - - .. code-block:: bash - - cd ./stacks/mers/clearlinux - -#. Use the :command:`docker build` command with the :file:`Dockerfile` to - build the MeRS container image. - - .. code-block:: bash - - docker build --no-cache -t sysstacks/mers-clearlinux . - -Use the |MERS| container image -****************************** - -This section shows examples of how the |MERS| container image can be used to -process media files. - -The models and video source can be substituted from your use-case. Some -publicly licensed sample videos are available at `sample-videos repository -`_ for testing. - - -Media Transcoding -================= - -The examples below show transcoding using the GPU or CPU for processing. - -#. On the host system, setup a workspace for data and models: - - .. code:: bash - - mkdir ~/ffmpeg - mkdir ~/ffmpeg/input - mkdir ~/ffmpeg/output - -#. Copy a video file to :file:`~/ffmpeg/input`. - - .. code:: bash - - cp ~/ffmpeg/input - -#. Run the *sysstacks/mers-clearlinux* Docker image, allowing shared access to - the workspace on the host: - - .. code:: bash - - docker run -it \ - --volume ~/ffmpeg:/home/mers-user:ro \ - --device=/dev/dri \ - --env QSV_DEVICE=/dev/dri/renderD128 \ - sysstacks/mers-clearlinux:latest - - .. note:: - - The :command:`--device` parameter and the **GSV_DEVICE** environment - variable allow shared access to the GPU on the host system. The values - needed may be different depending on host's graphics configuration. - - After running the :command:`docker run` command, you enter a bash shell - inside the container. - -#. From the container shell, you can run FFmpeg and - GStreamer commands against the videos in :file:`/home/mers-user/input` as - you would normally outside of |MERS|. - - Some sample commands are provided for reference. - - For more information on using the *FFmpeg* commands, refer to the `FFmpeg - documentation `_. - - For more information on using the *GStreamer* commands, refer to the - `GStreamer documentation - `_. - - -Example: Transcoding using GPU -------------------------------- - -The examples below show transcoding using the GPU for processing. - - -Using a FFmpeg to transcode raw content to SVT-HEVC and mp4: - -.. code:: bash - - ffmpeg -y -vaapi_device /dev/dri/renderD128 -f rawvideo -video_size 320x240 -r 30 -i -vf 'format=nv12, hwupload' -c:v h264_vaapi -y - -Using a GStreamer to transcode H264 to H265: - -.. code:: bash - - gst-launch-1.0 filesrc location= ! h264parse ! vaapih264dec ! vaapih265enc rate-control=cbr bitrate=5000 ! video/x-h265,profile=main ! h265parse ! filesink location= - -|MERS| builds FFmpeg with `HWAccel -`_ enabled which supports VAAPI. -Refer to the `FFmpeg wiki on VAAPI -`_ and `GStreamer with Media-SDK -wiki -`_ -for more usage examples and compatibility information. - - -Example: Transcoding using CPU ------------------------------- - -The example below shows transcoding of raw yuv420 content to SVT-HEVC and mp4, -using the CPU for processing. - -.. code:: bash - - ffmpeg -f rawvideo -vcodec rawvideo -s 320x240 -r 30 -pix_fmt yuv420p -i -c:v libsvt_hevc -y - -Additional generic examples of FFmpeg commands can be found in the -`OpenVisualCloud repository -`_ -and used for reference with |MERS|. - - -Media Analytics -=============== - -This example shows how to perform analytics and inferences with GStreamer -using the CPU for processing. - -The steps here are referenced from the `gst-video-analytics Getting Started -Guide `_ except simply -substituting the *gst-video-analytics* docker image for the -*sysstacks/mers-clearlinux* image. - -The example below shows how to use the |MERS| container image to perform video -with object detection and attributes recognition of a video using GStreamer -using pre-trained models and sample video files. - -#. On the host system, setup a workspace for data and models: - - .. code:: bash - - mkdir ~/gva - mkdir ~/gva/data - mkdir ~/gva/data/models - mkdir ~/gva/data/models/intel - mkdir ~/gva/data/models/common - mkdir ~/gva/data/video - -#. Clone the opencv/gst-video-analytics repository into the workspace: - - .. code:: bash - - git clone https://github.com/opencv/gst-video-analytics ~/gva/gst-video-analytics - cd ~/gva/gst-video-analytics - git submodule init - git submodule update - -#. Clone the Open Model Zoo repository into the workspace: - - .. code:: bash - - git clone https://github.com/opencv/open_model_zoo.git ~/gva/open_model_zoo - -#. Use the Model Downloader tool of Open Model Zoo to download ready to use - pre-trained models in IR format. - - .. note:: - - If you are on a network with outbound proxies, you will need to - configure set environment variables with the proxy server. - Refer to the documentation on :ref:`proxy` for detailed steps. - - On |CL| systems you will need the *python-extras* bundle. - Use :command:`sudo swupd bundle-add python-extras` for the downloader script to work. - - .. code:: bash - - cd ~/gva/open_model_zoo/tools/downloader - python3 downloader.py --list ~/gva/gst-video-analytics/samples/model_downloader_configs/intel_models_for_samples.LST -o ~/gva/data/models/intel - - -#. Copy a video file in h264 or mp4 format to :file:`~/gva/data/video`. Any - video with cars, pedestrians, human bodies, and/or human faces can be used. - - .. code:: bash - - git clone https://github.com/intel-iot-devkit/sample-videos.git ~/gva/data/video - - This example simply clones all the video files from the `sample-videos - repsoitory `_. - -#. From a desktop terminal, allow local access to the X host display. - - .. code:: bash - - xhost local:root - - export DATA_PATH=~/gva/data - export GVA_PATH=~/gva/gst-video-analytics - export MODELS_PATH=~/gva/data/models - export INTEL_MODELS_PATH=~/gva/data/models/intel - export VIDEO_EXAMPLES_PATH=~/gva/data/video - -#. Run the *sysstacks/mers-clearlinux* docker image, allowing shared access - to the X server and workspace on the host: - - .. code:: bash - - docker run -it --runtime=runc --net=host \ - -v ~/.Xauthority:/root/.Xauthority \ - -v /tmp/.X11-unix:/tmp/.X11-unix \ - -e DISPLAY=$DISPLAY \ - -e HTTP_PROXY=$HTTP_PROXY \ - -e HTTPS_PROXY=$HTTPS_PROXY \ - -e http_proxy=$http_proxy \ - -e https_proxy=$https_proxy \ - -v $GVA_PATH:/home/mers-user/gst-video-analytics \ - -v $INTEL_MODELS_PATH:/home/mers-user/intel_models \ - -v $MODELS_PATH:/home/mers-user/models \ - -v $VIDEO_EXAMPLES_PATH:/home/mers-user/video-examples \ - -e MODELS_PATH=/home/mers-user/intel_models:/home/mers-user/models \ - -e VIDEO_EXAMPLES_DIR=/home/mers-user/video-examples \ - sysstacks/mers-clearlinux:latest - - .. note:: - - In the :command:`docker run` command above: - - - :command:`--runtime=runc` specifies the container runtime to be - *runc* for this container. It is needed for correct interaction with X - server. - - - :command:`--net=host` provides host network access to the container. - It is needed for correct interaction with X server. - - - Files :file:`~/.Xauthority` and :file:`/tmp/.X11-unix` mapped to the - container are needed to ensure smooth authentication with X server. - - - :command:`-v` instances are needed to map host system directories - inside the Docker container. - - - :command:`-e` instances set the Docker container environment - variables. Some examples need these variables set correctly in order - to operate correctly. Proxy variables are needed if host is behind a - firewall. - - - After running the :command:`docker run` command, it will drop you into a - bash shell inside the container. - -#. From the container shell, run a sample analytics program in - :file:`~/gva/gst-video-analytics/samples` against your video source. - - Below are sample analytics that can be run against the sample videos. - Choose one to run: - - - Samples with *face detection and classification*: - - .. code:: bash - - ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/face-demographics-walking-and-pause.mp4 - ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/face-demographics-walking.mp4 - ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-female-and-male.mp4 - ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-male.mp4 - ./gst-video-analytics/samples/shell/face_detection_and_classification.sh $VIDEO_EXAMPLES_DIR/head-pose-face-detection-female.mp4 - - When running, a video with object detection and attributes recognition - (bounding boxes around faces with recognized attributes) should be - played. - - .. figure:: /_figures/stacks/mers-fig-1.png - :scale: 60% - :align: center - :alt: Face detection with the Clear Linux* OS Media Reference Stack - - Figure 1: Screenshot of |MERS| running face detection with GSTreamer - and OpenVINO. - - - Sample with *vehicle detection*: - - .. code:: bash - - ./gst-video-analytics/samples/shell/vehicle_detection_2sources_cpu.sh $VIDEO_EXAMPLES_DIR/car-detection.mp4 - - When running, a video with object detection and attributes recognition - (bounding boxes around vehicles with recognized attributes) should be - played. - - .. figure:: /_figures/stacks/mers-fig-2.png - :scale: 60% - :align: center - :alt: Vehicle detection with the Clear Linux* OS Media Reference Stack - - Figure 2: Screenshot of |MERS| running vehicle detection with - GSTreamer and OpenVINO. - - - Sample with *FPS measurement*: - - .. code:: bash - - ./gst-video-analytics/samples/shell/console_measure_fps_cpu.sh $VIDEO_EXAMPLES_DIR/bolt-detection.mp4 - - -Add AOM support -*************** - -The current version of |MERS| does not include the `Alliance for Open Media -`_ Video Codec (AOM). AOM can be built from source on an -individual basis. - -To add AOM support to the |MERS| image: - - -#. The following programs are needed to add AOM support to |MERS|: **docker, - git, patch**. On |CL| these can be installed with the commands below. For - other operating systems, install the appropriate packages. - - .. code:: bash - - sudo swupd bundle-add containers-basic dev-utils - - -#. Clone the Intel Stacks repository from GitHub. - - .. code:: bash - - git clone https://github.com/intel/stacks.git - -#. Navigate to the directory for the |MERS| image. - - .. code:: bash - - cd stacks/mers/clearlinux/ - -#. Apply the patch to the :file:`Dockerfile`. - - .. code:: bash - - patch -p1 < aom-patches/stacks-mers-v2-include-aom.diff - -#. Use the :command:`docker build` command to build a local copy of the - MeRS container image tagged as *aom*. - - .. code-block:: bash - - docker build --no-cache -t sysstacks/mers-clearlinux:aom . - -Once the build has completed successfully, the local image can be used -following the same steps in this tutorial by substituting the image name with -*sysstacks/mers-clearlinux:aom*. - - -*Intel, Xeon, OpenVINO, and the Intel logo are trademarks of Intel -Corporation or its subsidiaries. OpenCL and the OpenCL logo are trademarks of -Apple Inc. used by permission by Khronos.*