clearlinux/clear-linux-documentation
2
0
Fork 0
mirror of https://github.com/clearlinux/clear-linux-documentation.git synced 2026-08-26 18:36:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Restores old DRLS content for translation team QA. (#760)

Browse Source 
Signed-off-by: Kevin Putnam <kevin.putnam@intel.com>
This commit is contained in:
Kevin Putnam
2019-08-30 10:52:27 -07:00
committed by michael vincerra
parent 8830c289e9
commit 7a9c102925
7 changed files with 15 additions and 463 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/guides/index.rst
+1 -1
View File
@@ -55,4 +55,4 @@ Stacks
:glob:
stacks/*
dlrs2
stacks/dlrs/*
source/guides/stacks/dlrs.rst → source/guides/stacks/dlrs/dlrs.rst
+14 -462
View File
@@ -3,8 +3,8 @@
Deep Learning Reference Stack
#############################
This guide gives examples for using the Deep Learning Reference stack to run real-world usecases, as well as benchmarking workloads for TensorFlow\*,
PyTorch\*, and Kubeflow\* in |CL-ATTR|.
This guide describes how to run benchmarking workloads for TensorFlow\*,
PyTorch\*, and Kubeflow in |CL-ATTR| using the Deep Learning Reference Stack.
.. contents::
:local:
@@ -35,7 +35,7 @@ The Deep Learning Reference Stack is available in the following versions:
.. important::
To take advantage of the Intel® AVX-512 and VNNI functionality (including the MKL-DNN releases) with the Deep
To take advantage of the Intel® AVX-512 and VNNI functionality with the Deep
Learning Reference Stack, you must use the following hardware:
* Intel® AVX-512 images require an Intel® Xeon® Scalable Platform
@@ -44,8 +44,7 @@ The Deep Learning Reference Stack is available in the following versions:
Stack features
==============
* `DLRS V4.0`_ release announcement, including benchmark results.
* `DLRS V3.0`_ release announcement.
* `DLRS V3.0`_ release announcement.
* Deep Learning Reference Stack v2.0 including current
`PyTorch benchmark`_.
* Deep Learning Reference Stack v1.0 including current
@@ -102,6 +101,7 @@ We validated these steps against the following software package versions:
* Kubernetes 1.11.3
* Go 1.11.12
.. note::
The Deep Learning Reference Stack was developed to provide the best user
@@ -366,8 +366,8 @@ Run a TFJob
This replicates and deploys three test setups in your Kubernetes cluster.
Results of running this section
===============================
Results of running this guide
=============================
You must parse the logs of the Kubernetes pod to retrieve performance
data. The pods will still exist post-completion and will be in
@@ -375,392 +375,6 @@ data. The pods will still exist post-completion and will be in
benchmark results. More information about Kubernetes logging is available
in the Kubernetes `Logging Architecture`_ documentation.
TensorFlow Training (TFJob) with Kubeflow and DLRS
**************************************************
A `TFJob`_ is Kubeflow's custom resource used to run TensorFlow training jobs on Kubernetes. This example shows how to use a TFJob within the DLRS container.
Pre-requisites:
* A running :ref:`kubernetes` cluster
#. Deploying Kubeflow with kfctl/kustomize in |CL|
.. note::
This example proposes a Kubeflow installation with the binary kfctl maintained by `Arrikto`_. Please download the `kfctl tarball`_ to complete the following steps
#. Download, untar and add to your PATH if necessary
.. code-block:: bash
KFCTL_URL="https://github.com/kubeflow/kubeflow/releases/download/v0.6.1/kfctl_v0.6.1_linux.tar.gz"
wget -P ${KFCTL_URL} ${KFCTL_PATH}
tar -C ${KFCTL_PATH} -xvf ${KFCTL_PATH}/kfctl_v${kfctl_ver}_linux.tar.gz
export PATH=$PATH:${KFCTL_PATH}
#. Install `MetalLB`_
.. code-block:: bash
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.1/manifests/metallb.yaml
#. Install Kubeflow resource and TFJob operators
.. code-block:: bash
# Env variables needed for your deployment
export KFAPP="<your choice of application directory name>"
export CONFIG="https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_existing_arrikto.yaml"
kfctl init ${KFAPP} --config=${CONFIG} -V
cd ${KFAPP}
# deploy Kubeflow:
kfctl generate k8s -V
kfctl apply k8s -V
#. List the resources
Deployment takes around 15 minutes (or more depending on the hardware) to be ready to use. After that you can use kubectl to list all the Kubeflow resources deployed and monitor their status.
.. code-block:: bash
kubectl get pods -n kubeflow
Submitting TFJobs
=================
We provide several `DLRS TFJob`_ examples that use the Deep Learning Reference Stack as the base image for creating the containers to run training workloads in your Kubernetes cluster.
Customizing a TFJob
===================
A TFJob is a resource with a YAML representation like the one below. Edit to use the DLRS image containing the code to be executed and modify the command for your own training code.
If you'd like to modify the number and type of replicas, resources, persistent volumes and environment variables, please refer to the `Kubeflow documentation`_
.. code-block:: console
apiVersion: kubeflow.org/v1beta2
kind: TFJob
metadata:
generateName: tfjob
namespace: kubeflow
spec:
tfReplicaSpecs:
PS:
replicas: 1
restartPolicy: OnFailure
template:
spec:
containers:
- name: tensorflow
image: dlrs-image
command:
- python
- -m
- trainer.task
- --batch_size=32
- --training_steps=1000
Worker:
replicas: 3
restartPolicy: OnFailure
template:
spec:
containers:
- name: tensorflow
image: dlrs-image
command:
- python
- -m
- trainer.task
- --batch_size=32
- --training_steps=1000
Master:
replicas: 1
restartPolicy: OnFailure
template:
spec:
containers:
- name: tensorflow
image: dlrs-image
command:
- python
- -m
- trainer.task
- --batch_size=32
- --training_steps=1000
For more information, please refer to:
* `Distributed TensorFlow`_
* `TFJobs`_
PyTorch Training (PyTorch Job) with Kubeflow and DLRS
*****************************************************
A `PyTorch Job`_ is Kubeflow's custom resource used to run PyTorch training jobs on Kubernetes. This example builds on the framework set up in the previous example.
Pre-requisites:
* A running :ref:`kubernetes` cluster
* Please follow steps 1 - 5 of the previous example to set up your environment.
Submitting PyTorch Jobs
=======================
We provide several `DLRS PytorchJob`_ examples that use the Deep Learning Reference Stack as the base image for creating the container(s) that will run training workloads in your Kubernetes cluster.
Select one form the list below:
Using Kubeflow Seldon and OpenVINO* with the Deep Learning Reference Stack
**************************************************************************
`Seldon Core`_ is an open source platform for deploying machine learning models on a Kubernetes cluster. Seldon Core is supported in the `DLRS V4.0`_ release.
Pre-requisites
==============
* A running :ref:`kubernetes` cluster
.. note::
Instead of using Arrikto's configuration manifest as shown in the preceeding example, you should use the manifest provided by `Istio`_, for this example, as Seldon deployments depend on it.
#. Install deployment tools
.. code-block:: bash
INSTALL_DIR=$HOME/install_dir
BIN_DIR=${INSTALL_DIR}/bin
SRC_DIR=${INSTALL_DIR}/source
export PATH=${BIN_DIR}:$PATH
mkdir -p ${BIN_DIR} && mkdir ${SRC_DIR}
cd ${SRC_DIR}
#. Install Helm*
.. code-block:: bash
wget https://get.helm.sh/helm-v2.14.3-linux-amd64.tar.gz && tar xf helm-v2.14.3-linux-amd64.tar.gz
mv linux-amd64/helm ${BIN_DIR}/helm
#. Clean the environment
.. code-block:: bash
rm -rf ${SRC_DIR}/*
#. Prepare the DLRS image
The DLRS base image needs to be rebuilt with the `Dockerfile_openvino_base`_ to add Seldon and the OpenVINO inference engine.
.. code-block:: bash
docker build -f Dockerfile_openvino_base -t dlrs_openvino_base:0.1 .
#. Mount pre-trained models into a persistent volume
This will also apply all PV manifests to the cluster
.. code-block:: bash
kubectl apply -f storage/pv-volume.yaml
kubectl apply -f storage/model-store-pvc.yaml
kubectl apply -f storage/pv-pod.yaml
#. Start a shell for the container used as pv:
.. code-block:: bash
kubectl exec -it hostpath-pvc -- /bin/bash
#. Save pre-trained models
Now that you're inside the running container, fetch your pre-trained models and save them at `/opt/ml`
.. code-block:: bash
root@hostpath-pvc:/# cd /opt/ml
root@hostpath-pvc:/# # Copy your models here
root@hostpath-pvc:/# # exit
#. Deploy the model server
Now you're ready to deploy the model server using the Helm chart provided.
.. code-block:: bash
helm install -- name=seldonov-model-server \
--namespace kubeflow \
--set openvino.image=dlrs_openvino_base:0.1 \
--set openvino.model.path=/opt/ml/<models_directory> \
--set openvino.model.name=<model_name> \
--set openvino.model.input=data \
--set openvino.model.output=prob
dlrs-seldon/helm/seldon-model-server
Using the Intel® OpenVINO Model Optimizer
*****************************************
The Intel OpenVINO toolkit has two primary tools for deep learning, the inference engine and the model optimizer. The inference engine is integrated into the Deep Learning Reference Stack. It is better to use the model optimizer after training the model, and before inference begins. This example will explain how to use the model optimizer by going through a test case with a pre-trained TensorFlow model.
This example uses resources found in the following OpenVino Toolkit documentation.
`Converting a TensorFlow Model`_
`Converting TensorFlow Object Detection API Models`_
In this example, you will:
* Download a TensorFlow model
* Clone the Model Optimizer
* Install Prerequisites
* Run the Model Optimizer
#. Download a TensorFlow model
We will be using an OpenVINO supported topology with the Model Optimizer. We will use a TensorFlow Inception V2 frozen model.
Navigate to the `OpenVINO TensorFlow Model page`_. Then scroll down to the second section titled "Supported Frozen Topologies from TensorFlow Object Detection Models Zoo" and download "SSD Inception V2 COCO."
Unpack the file into your chosen working directory. For example, if the tar file is in your Downloads folder and you have navigated to the directory you want to extract it into, run:
.. code-block:: bash
tar -xvf ~/Downloads/ssd_inception_v2_coco_2018_01_28.tar.gz
#. Clone the Model Optimizer
Next we need the model optimizer directory, named `dldt`_. This example assumes the parent directory is on the same level as the model directory, ie:
.. code-block:: console
+--Working_Directory
+-- ssd_inception_v2_coco_2018_01_28
+-- dldt
To clone the Model Optimizer, run this from inside the working directory:
.. code-block:: bash
git clone https://github.com/opencv/dldt.git
If you explore the :file:`dldt` directory, you'll see both the inference engine and the model optimizer. We are only concerned with the model optimizer at this stage. Navigating into the model optimizer folder you'll find several python scripts and text files. These are the scripts you call to run the model optimizer.
#. Install Prerequisites for Model Optimizer
Install the Python packages required to run the model optimizer by running the script dldt/model-optimizer/install_prerequisites/install_prerequisites_tf.sh.
.. code-block:: bash
cd dldt/model-optimizer/install_prerequisites/
./install_prerequisites_tf.sh
cd ../../..
#. Run the Model Optimizer
Running the model optimizer is as simple as calling the appropriate script, however there are many configuration options that are explainedin the documentation
.. code-block:: bash
python dldt/model-optimizer/mo_tf.py \
--input_model=ssd_inception_v2_coco_2018_01_28/frozen_inference_graph.pb \
--tensorflow_use_custom_operations_config dldt/model-optimizer/extensions/front/tf/ssd_v2_support.json \
--tensorflow_object_detection_api_pipeline_config ssd_inception_v2_coco_2018_01_28/pipeline.config \
--reverse_input_channels
You should now see three files in your working directory, :file:`frozen_inference_graph.bin`, :file:`frozen_inference_graph.mapping`, and :file:`frozen_inference_graph.xml`. These are your new models in the Intermediate Representation (IR) format and they are ready for use in the OpenVINO Inference Engine.
Using the OpenVino Inference Engine
***********************************
This example walks through the basic instructions for using the inference engine.
#. Starting the Model Server
The process is similar to how we start `Jupter notebooks` on our containers
Run this command to spin up a OpenVino model fetched from GCP
.. code-block:: bash
docker run -p 8000:8000 stacks-tensorflow-mkl:latest bash -c ". /workspace/scripts/serve.sh && ie_serving model --model_name resnet --model_path gs://intelai_public_models/resnet_50_i8 --port 8000"
Once the server is setup, use a :command:`grpc` client to communicate with served model:
.. code-block:: bash
git clone https://github.com/IntelAI/OpenVINO-model-server.git
cd OpenVINO-model-server
pip install -q -r OpenVINO-model-server/example_client/client_requirements.txt
pip install --user -q -r OpenVINO-model-server/example_client/client_requirements.txt
cat OpenVINO-model-server/example_client/client_requirements.txt
cd OpenVINO-model-server/example_client
python jpeg_classification.py --images_list input_images.txt --grpc_address localhost --grpc_port 8000 --input_name data --output_name prob --size 224 --model_name resnet
The results of these commands will look like this:
.. code-block:: console
start processing:
Model name: resnet
Images list file: input_images.txt
images/airliner.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 97.00 ms; speed 2.00 fps 10.35
Detected: 404 Should be: 404
images/arctic-fox.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 16.00 ms; speed 2.00 fps 63.89
Detected: 279 Should be: 279
images/bee.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 14.00 ms; speed 2.00 fps 69.82
Detected: 309 Should be: 309
images/golden_retriever.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 13.00 ms; speed 2.00 fps 75.22
Detected: 207 Should be: 207
images/gorilla.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 11.00 ms; speed 2.00 fps 87.24
Detected: 366 Should be: 366
images/magnetic_compass.jpeg (1, 3, 224, 224) ; data range: 0.0 : 247.0
Processing time: 11.00 ms; speed 2.00 fps 91.07
Detected: 635 Should be: 635
images/peacock.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 9.00 ms; speed 2.00 fps 110.1
Detected: 84 Should be: 84
images/pelican.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 10.00 ms; speed 2.00 fps 103.63
Detected: 144 Should be: 144
images/snail.jpeg (1, 3, 224, 224) ; data range: 0.0 : 248.0
Processing time: 10.00 ms; speed 2.00 fps 104.33
Detected: 113 Should be: 113
images/zebra.jpeg (1, 3, 224, 224) ; data range: 0.0 : 255.0
Processing time: 12.00 ms; speed 2.00 fps 83.04
Detected: 340 Should be: 340
Overall accuracy= 100.0 %
Average latency= 19.8 ms
Use Jupyter Notebook
********************
@@ -797,8 +411,8 @@ the host system, replace :command:`127.0.0.1` with the IP address of the host.
Your browser displays the following:
.. figure:: ../../_figures/stacks/dlrs-fig-1.png
:scale: 50%
.. figure:: figures/dlrs-fig-1.png
:scale: 50 %
:alt: Jupyter Notebook
Figure 1: :guilabel:`Jupyter Notebook`
@@ -806,7 +420,7 @@ Figure 1: :guilabel:`Jupyter Notebook`
To create a new notebook, click :guilabel:`New` and select :guilabel:`Python 3`.
.. figure:: ../../_figures/stacks/dlrs-fig-2.png
.. figure:: figures/dlrs-fig-2.png
:scale: 50%
:alt: Create a new notebook
@@ -814,7 +428,7 @@ Figure 2: Create a new notebook
A new, blank notebook is displayed, with a cell ready for input.
.. figure:: ../../_figures/stacks/dlrs-fig-3.png
.. figure:: figures/dlrs-fig-3.png
:scale: 50%
:alt: New blank notebook
@@ -829,13 +443,13 @@ cell, and run the cell.
x = torch.rand(5, 3)
print(x)
.. figure:: ../../_figures/stacks/dlrs-fig-4.png
.. figure:: figures/dlrs-fig-4.png
:scale: 50%
:alt: Sample code snippet
When you run the cell, your output will look something like this:
.. figure:: ../../_figures/stacks/dlrs-fig-5.png
.. figure:: figures/dlrs-fig-5.png
:scale: 50%
:alt: code output
@@ -918,31 +532,6 @@ To stop the container, execute the following from your host system:
docker images
Compiling AIXPRT with OpenMP on DLRS
************************************
To compile AIXPRT for DLRS, you will have to get the community edition of AIXPRT and update the `compile_AIXPRT_source.sh` file.AIXPRT utilizes
build configuration files, so to build AIXPRT on the image, copy, the build files from the base image, this can be done by adding these commands
to the end of the stacks-tensorflow-mkl dockerfile:
.. code-block:: console
COPY --from=base /dldt/inference-engine/bin/intel64/Release/ /usr/local/lib/openvino/tools/
COPY --from=base /dldt/ /dldt/
COPY ./airxprt/ /workspace/aixprt/
RUN ./aixprt/install_deps.sh
RUN ./aixprt/install_aixprt.sh
AIXPRT requires OpenCV. On |CL|, the OpenCV bundle also installs the DLDT components. To use AIXPRT in the DLRS environment you need to either remove the shared libraries for DLDT from :file:`/usr/lib64` before you run the tests, or ensure that the DLDT components in the :file:`/usr/local/lib` are being used for AIXPRT. This can be achieved using adding LD_LIBRARY_PATH environment variable before testing.
.. code-block:: bash
export LD_LIBRARY_PATH=/usr/local/lib
The updates to the AIXPRT community edition have been captured in the diff file :file:`compile_AIXPRT_source.sh.patch`. The core of these changes relate to the version of model files(2019_R1) we download from the `OpenCV open model zoo`_ and location of the build files, which in our case is `/dldt`. Please refer to the patch files and make changes as necessary to the compile_AIXPRT_source.sh file as required for your environment.
Related topics
**************
@@ -982,8 +571,6 @@ Related topics
.. _DLRS V3.0: https://clearlinux.org/stacks/deep-learning-reference-stack-v3
.. _DLRS V4.0: https://clearlinux.org/news-blogs/deep-learning-reference-stack-v4
.. _dlrs-tfjob: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
.. _Logging Architecture: https://kubernetes.io/docs/concepts/cluster-administration/logging/
@@ -1002,39 +589,4 @@ Related topics
.. _DLRS Release notes: https://github.com/clearlinux/dockerfiles/blob/master/stacks/dlrs/releasenote.md
.. _Seldon Core: https://docs.seldon.io/projects/seldon-core/en/latest/
.. _Istio: https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml
.. _Dockerfile_openvino_base: https://github.com/clearlinux/dockerfiles/blob/master/stacks/dlrs/kubeflow/dlrs-seldon/docker/Dockerfile_openvino_base
.. _TFJob: https://www.kubeflow.org/docs/components/tftraining
.. _Arrikto: https://www.kubeflow.org/docs/started/k8s/kfctl-existing-arrikto/
.. _kfctl tarball: https://github.com/kubeflow/kubeflow/releases/download/v0.6.1/kfctl_v0.6.1_linux.tar.gz
.. _MetalLB: https://metallb.universe.tf/
.. _Kubeflow documentation: https://www.kubeflow.org/docs/components/tftraining/#what-is-tfjob
.. _Distributed TensorFlow: https://www.tensorflow.org/deploy/distributed
.. _TFJobs: https://www.kubeflow.org/docs/components/tftraining/
.. _Intel® quantization tools: https://github.com/IntelAI/tools/blob/master/tensorflow_quantization/README.md#quantization-tools
.. _OpenCV open model zoo: https://github.com/opencv/open_model_zoo
.. _PyTorch Job: https://www.kubeflow.org/docs/components/pytorch/
.. _Converting a TensorFlow Model: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html
.. _Converting TensorFlow Object Detection API Models: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html
.. _OpenVINO TensorFlow Model page: https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html
.. _dldt: https://github.com/opencv/dldt
.. _DLRS TFJob: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-tfjob
.. _DLRS PytorchJob: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow/dlrs-pytorchjob
.. _Intel® quantization tools: https://github.com/IntelAI/tools/blob/master/tensorflow_quantization/README.md#quantization-tools
source/guides/stacks/dlrs/figures/dlrs-fig-1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

source/guides/stacks/dlrs/figures/dlrs-fig-2.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

source/guides/stacks/dlrs/figures/dlrs-fig-3.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

source/guides/stacks/dlrs/figures/dlrs-fig-4.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 67 KiB

source/guides/stacks/dlrs/figures/dlrs-fig-5.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 81 KiB