Feeding Tensorflow from GPU

About

Reference guides:

• Supercharging Object Detection in Video: from Glacial to Lightning Speed
• Setup
• First App
• Optimizing Decoding and Graph Feeding

Target projects:

• tensorflow-object-detection-cpp - contains a sample model, labels and a test video
• fast_od - contains feeding Tensorflow from GPU

Setup

Kubuntu 18.04, CUDA 10.0 (10.0 is a requirement for Java version of TF 1.15 in Maven CDN):

mkdir -p ~/git/tf_cv_cu
cd ~/git/tf_cv_cu

# Download section
git clone https://github.com/fierval/tensorflow-object-detection-cpp.git
git clone https://github.com/fierval/fast_od.git
wget https://github.com/Kitware/CMake/releases/download/v3.16.2/cmake-3.16.2.tar.gz
wget https://github.com/bazelbuild/bazel/releases/download/0.25.2/bazel_0.25.2-linux-x86_64.deb
wget https://github.com/tensorflow/tensorflow/archive/v1.15.0.tar.gz -O tensorflow-1.15.0.tar.gz
wget https://github.com/opencv/opencv/archive/3.4.9.tar.gz -O opencv-3.4.9.tar.gz
wget https://github.com/opencv/opencv_contrib/archive/3.4.9.tar.gz -O opencv_contrib-3.4.9.tar.gz

# Unpack archives
tar xzvf cmake-3.16.2.tar.gz
tar xzvf tensorflow-1.15.0.tar.gz
tar xzvf opencv-3.4.9.tar.gz
tar xzvf opencv_contrib-3.4.9.tar.gz

# Build & update cmake
sudo apt install libssl-dev
sudo apt install qt4-default
cd cmake-3.16.2
./bootstrap -qt-gui
make -j8
sudo make install

cmake -version
cmake-gui -version

# Install bazel
sudo dpkg -i bazel_0.25.2-linux-x86_64.deb
bazel version

# Build TF
cd ~/git/tf_cv_cu/tensorflow-1.15.0/
tensorflow/contrib/makefile/download_dependencies.sh
./configure
# pick all defaults except select yes for CUDA support
bazel build //tensorflow:libtensorflow_cc.so

# Build OpenCV
sudo apt install libgtkglext1 libgtkglext1-dev
sudo apt install build-essential
sudo apt install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libdc1394-22-dev
# sudo apt install libjasper-dev # missing in Kubuntu 18.04

sudo cp ~/git/tf_cv_cu/fast_od/docker/nvcuvid/dynlink_* /usr/local/cuda/include

cd ~/git/tf_cv_cu/opencv-3.4.9
mkdir build; cd build
cmake -D WITH_CUDA=ON \
      -D OPENCV_EXTRA_MODULES_PATH=~/git/tf_cv_cu/opencv_contrib-3.4.9/modules \
      -D BUILD_PERF_TESTS=OFF \
      -D BUILD_TESTS=OFF \
      -D BUILD_opencv_cudacodec=ON \
      -D WITH_NVCUVID=ON \
      -D WITH_GTK_2_X=ON \
      ..
make -j8
sudo make install
sudo ldconfig