TensorFlow-Keras Model, Train in Python, Use in C++

Despite many advantages of Python such as simple syntax, easy-to-read code and large number of available 3rd-party packages, when it comes to embedded systems and real-time applications, C++ is undoubtedly one of the most efficient and powerful programming languages that dominates the realm of these systems.

However, complex syntax of C++ makes it difficult to implement high level algorithm such as Machine Learning algorithms. For example building a small CNN network and it’s training pipeline can go so complex that no developer prefers doing such task the in C++ from the scratch.

In this post, we’ll learn how to transfer and use CNN models that is trained in Python with Keras to C++ using a light weight header-only library named frugally-deep. I found no good tutorial on the web teaching this library and its usage in details. So, here we are going to explore it step by step.

First we train a simple CNN network in Python on a 2000 Cat & Dog image dataset. No matter how much accuracy we achieve, we just save the model to use it in the c++ code :

	import cv2
	import glob
	from sklearn.neighbors import KNeighborsClassifier
	import numpy as np
	from sklearn.preprocessing import LabelEncoder
	from tensorflow.keras.utils import to_categorical
	from joblib import dump, load
	from sklearn.model_selection import train_test_split
	import tensorflow as tf
	from tensorflow.keras import models
	from tensorflow.keras import layers
	import os
	data=[]
	labels=[]
	for item in glob.glob("Dataset/*/*"):
	img = cv2.imread(item,cv2.IMREAD_GRAYSCALE) # read images with one channel grayscale
	r_img= cv2.resize(img,(128,128)) # resize to 128x128
	r_img = np.expand_dims(r_img, axis = -1)
	data.append(r_img) # add resized image to dataset list
	label = item.split("\\")[1]
	labels.append(label) #add image label to dataset list
	#preprocess
	le = LabelEncoder()
	labels = le.fit_transform(labels)
	labels = to_categorical(labels)
	data = np.array(data)/255 # Normalize channel between 0 to 1
	#split test and train randomly
	x_train, x_test, y_train, y_test = train_test_split(data,labels,test_size=0.2)
	#train
	net= models.Sequential(
	[
	layers.Conv2D(32,(3,3),strides=(1,1),activation="relu",input_shape=(128,128,1)),
	layers.Conv2D(32,(3,3),strides=(1,1),activation="relu"),
	layers.BatchNormalization(),
	layers.MaxPool2D((3,3)),
	layers.Conv2D(64,(5,5),strides=(1,1),activation="relu"),
	layers.Conv2D(64,(5,5),strides=(1,1),activation="relu"),
	layers.BatchNormalization(),
	layers.AvgPool2D((3,3)),
	layers.Dropout(0.75),
	layers.Flatten(),
	layers.Dense(64,activation="relu"),
	layers.Dense(16,activation="relu"),
	layers.Dense(2,activation="softmax")
	]
	)
	print(net.summary())
	net.compile(optimizer="SGD", loss="binary_crossentropy",metrics=["accuracy"])
	H = net.fit(x_train,y_train,batch_size=32, epochs=24, validation_data=(x_test,y_test))
	net.save("CatDogNew.h5") # Save the model

view raw CatDogCNN.py hosted with ❤ by GitHub

After running the above code , we will have our model “CatDogNew.h5” file saved in project folder.

Before we start the next step we need to download the library from its repository on gihub. After download , find “keras_export” directory in the main folder of the library, and open command prompt from this directory :

Now, we should convert the model to json file using convert_model.py . Run the following command in the command line to build model in json format :

python convert_model.py path/to/your/model/CatDogNew.h5 CatDogNew.json

Running the above command will yield “CatDogNew.json” in the keras_export folder.

In this step which is so important, we should prepare our Visual Studio project to use the frugally-deep library. I use Visual Studio 2019, however there is no big difference between this version and its old ones. First, Create new Console App and choose a name for it.

After creating the Project, you should have one cpp file opened with a main function containing “hello world” output. Now, It’s time to include library header file in project. Frugally-deep depends on 3 other libraries which they are also header-only and can be included without any pain. The libraries are :

• Eigen
• fplus
• nlohmann

I gathered all of them in a zip file that you can download it with just a click here. I added the frugally-deep header files to the zip file so you don’t need to get it from source code downloaded from github.

Extract zip file and move the include folder to a place that is easy to address later. The include folder must contains all libraries like below :

Open project properties from Project menu, in the left side panel click on VC++ Directories, then in right side edit the Include Directories row and insert the path to the already extracted include folder containing 4 libraries in the field :

As the the frugally-deep is a header-only library, it need no more configuration in project properties. So, we can go to the project’s main file and past following codes in it :

	#include <iostream>
	#include <fdeep/fdeep.hpp>
	#include <opencv2/core/core.hpp>
	#include <opencv2/highgui/highgui.hpp>
	#include <opencv2/imgproc.hpp>
	#include <filesystem>
	#include <exception>
	using namespace stdext;
	namespace fs = std::filesystem;
	int main()
	{
	std::string test_image_folder_path = "d:/catdog/"; // path to folder containing test images
	const auto mymodel = fdeep::load_model("D:/CatDogNew.json"); // load the converted model
	for (const auto& entry : fs::directory_iterator(test_image_folder_path)) {
	try
	{
	std::string image_path = entry.path().string(); // get images path one by one
	const cv::Mat image = cv::imread(image_path, cv::IMREAD_GRAYSCALE); // read the image in single channel grayscale mode
	cv::Mat resizedImg;
	cv::resize(image, resizedImg, cv::Size(128, 128)); // resize image to the 128x128 (Model input dimension)
	imshow("Display Window", resizedImg); // just show the image
	// convert cv::MAT to fdeep::tensor
	const auto input = fdeep::tensor_from_bytes(resizedImg.ptr(),
	static_cast<std::size_t>(resizedImg.rows),
	static_cast<std::size_t>(resizedImg.cols),
	static_cast<std::size_t>(resizedImg.channels()),
	0.0f, 1.0f);
	auto result = mymodel.predict({ input }); // predict the image's label and ouput a 1x2 tensor containing each class probability
	std::cout << fdeep::show_tensors(result) << std::endl; // print the tensor
	cv::waitKey();
	}
	catch (const std::exception& e)
	{
	std::cout << e.what() << '\n';
	}
	}
	}

view raw kerasModelInCpp.cpp hosted with ❤ by GitHub

let’s see what happens in important parts of the code, line by line:

Line 1 to 8 is doing include jobs. First of all, we need to have OpenCV installed, to install and use OpenCV in c++ please see this post. Then we load the JSON model in line 16. We have accomplished half of the job, but the important part of the code is still remained, reading image and applying preprocess. We must feed the model with (128,128) single channel image that is normalized to [0-1] for each pixel. In line 21 we read the image in grayscale mode, then resize it in line 24. The normalization part still remained. We will do it when we are converting the image matrix to tensor. The tensor_from_bytes function in frugally-deep library does it for us in line 28. It takes 6 input parameters, first one is the image matrix pointer, the second and third parameters are the image height and image width. Fourth parameter is the number of channels. And very important last parameters are intended to re-scale the channel values from range [0, 255] to [low, high] in which the low and high must be specified with fifth and sixth parameters.

Finally we pass the input tensor to predict function, maybe you ask why we give it within {}, the answer is : frugally-deep supports models with multiple inputs , so we give each input in {}, separated by comma. The predict function output a tensor array that each element shows the corresponding class probability.

Congratulations! You have made your way to the end of this short journey. Our CNN model was trained in python, converted, and then used in C++.

Although using frugally-deep in your c++ project can be funny, you should know its pros and cons :

Pros

• Performance: Since it’s a C++ library, it can offer better performance compared to running the same model in Python.
• Deployment: Useful for deploying models in environments where Python is not an option (e.g., embedded systems).

Cons

• Complex Models: There might be limitations in terms of the types of models and layers supported compared to running directly in Keras with TensorFlow.
• Updates: Keeping up with the latest changes and features in Keras and TensorFlow might require updates to frugally-deep.

Thank you so much for reading this post!