C++ TensorFlow: A Journey

Flow those Tensors!

One of my projects required doing some model inferencing from C++.

So, it’s time to delve into the world of linking TensorFlow into a C++ CMake based project.

One way to do it is to build your C++ project within the TensorFlow build tree, using Google’s build tool Bazel. I first tried this, but because of the existing dependencies in the CMake project, it involves changing these to Bazel build format. It also builds a 100MB binary for a tiny project, so that’s just silly.

So a better approach is to build a shared library of TensorFlow and link to it.

Following the instructions here is a good start: https://github.com/cjweeks/tensorflow-cmake

Adding the libtensorflow_all.so target (with the linkopts line removed on mac) to the tensorflow/BUILD file is key here, along with copying the built files to the system directories.

# Added build rule
cc_binary(
    name = "libtensorflow_all.so",
    linkshared = 1,
    deps = [
        "//tensorflow/core:framework_internal",
        "//tensorflow/core:tensorflow",
        "//tensorflow/cc:cc_ops",
        "//tensorflow/cc:client_session",
        "//tensorflow/cc:scope",
        "//tensorflow/c:c_api",
    ],
)

I ran into trouble with Protocol Buffers (and got distracted, by the way, did you know that the guy who wrote Protocol Buffers built a LAN-party house?). TensorFlow 1.4.0, the latest as of this writing, requires Protocol Buffers 3.4, but I had 3.5 installed. You’d think this would be no problem, having a better version, but I kept running into this error:

/usr/local/include/google/tensorflow/tensorflow/core/framework/tensor.pb.h:19:2:
This file was generated by an older version of protoc which is
incompatible with your Protocol Buffer headers. Please
regenerate this file with a newer version of protoc.

So after some investigating and trying to get it to work with Protocol Buffers 3.5 (which usually is a fruitless task, trying to get some software to use a different version of a dependency than it needs), downgrading is in order, so a build and install of Protocol Buffers 3.4 fixed it:

wget https://github.com/google/protobuf/releases/download/v3.4.0/protobuf-cpp-3.4.0.zip
unzip protobuf-cpp-3.4.0.zip
./configure
make
sudo make install

Sweet, it now now longer complains of the Protocol Buffers version, but is complaining about nsync.h missing. Looks like we need to install NSync as well. Not the 90’s band, the Google mutex library.

git clone https://github.com/google/nsync.git
cmake .
make
sudo make install

Ok, now we’re rolling. Time to build TensorFlow:

bazel clean
bazel build tensorflow:libtensorflow_all.so

And add that library to our CMake CMakeLists.txt:

target_link_libraries(mpc ipopt z ssl uv uWS tensorflow_all)

Hooray! Now we can build our CMake C++ project.

Get Tom Jacobs’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Alright! The individual source files compile. But come link time, we get the nice error:

Undefined symbols for architecture x86_64:
“tensorflow::TensorShapeRep::DestructorOutOfLine()”, referenced from:
tensorflow::Input::Initializer::Initializer<float, void>(std::initializer_list<float> const&) in main.cpp.o
“tensorflow::TensorShapeBase<tensorflow::TensorShape>::TensorShapeBase(tensorflow::gtl::ArraySlice<long long>)”, referenced from:
tensorflow::Input::Initializer::Initializer<float, void>(std::initializer_list<float> const&) in main.cpp.o
“tensorflow::TfCheckOpHelperOutOfLine(tensorflow::Status const&, char const*)”, referenced from:
_main in main.cpp.o
….

And so on. Looks like we have a few missing symbols. Adding dynamic_lookup to the CMAKE_CXX_FLAGS in CMakeLists.txt gets the project building and linking:

set(CXX_FLAGS “-Wall -undefined dynamic_lookup”)
set(CMAKE_CXX_FLAGS “${CXX_FLAGS}”)

But of course it fails when run.

dyld: Symbol not found: __ZN10tensorflow10DEVICE_CPUE
Referenced from: /usr/local/lib/libtensorflow_all.so
Expected in: flat namespace
in /usr/local/lib/libtensorflow_all.so
Abort trap: 6

Hm, it may have something to do with Eigen. This comment fills me with hope:

So I didn’t realise that you can’t actually link with Tensorflow as a normal C++ library from an external tool. You have to use Bazel and your code has to be inside the Tensorflow repo. It does tell you this in the C++ tutorial but it is very unexpected and I hadn’t read that.

Anyway, using Bazel resolved the errors about Eigen. It seems like it downloads it automatically somehow.

Ok, back up a bit.

Taking a look at the tensorflow/BUILD file, it looks like there’s a very similar .so build target to the one we added, with a few more dependencies:

tf_cc_shared_object(
name = “libtensorflow_cc.so”,
….

Let’s try building that already existing shared library, and linking our program against that.

bazel build tensorflow:libtensorflow_cc.so
cp bazel-bin/tensorflow/libtensorflow_cc.so /usr/local/lib/
bazel build tensorflow:libtensorflow_framework.so
cp bazel-bin/tensorflow/libtensorflow_framework.so /usr/local/lib/

And link:

target_link_libraries(mpc ipopt z ssl uv uWS tensorflow_cc)

Look at that!

make
— Configuring done
— Generating done
— Build files have been written
[ 33%] Linking CXX executable mpc
[100%] Built target mpc

./mpc
2017–11–28 18:04:44.426134: I /Users/thomasjacobs/Desktop/CarND-MPC-Project/src/main.cpp:31]

19
-3

It builds and runs! Of course - the target we added was outdated, and the TensorFlow developers have already added a C++ shared library target.

Now onto loading up a pre-trained Keras h5 model file and making some predictions, let’s hope the new top level Keras API is also linked in there…

It looks like not, so we may have to re-define the model and load the weights with some IO ops. https://www.tensorflow.org/api_docs/cc/group/io-ops