July 11, 2024
tensorflow keras c plus plus

TensorFlow-Keras Model, Train in Python, Use in C++

Despite many advantages of Python such as simple syntax, easy-to-read code and large number of available 3rd-party packages, when it comes to embedded systems and real-time applications, C++ is undoubtedly one of the most efficient and powerful programming languages that dominates the realm of these systems.

Embedding System
Top 10 languages that used in embedded projects (Reference)

However, complex syntax of C++ makes it difficult to implement high level algorithm such as Machine Learning algorithms. For example building a small CNN network and it’s training pipeline can go so complex that no developer prefers doing such task the in C++ from the scratch.

In this post, we’ll learn how to transfer and use CNN models that is trained in Python with Keras to C++ using a light weight header-only library named frugally-deep. I found no good tutorial on the web teaching this library and its usage in details. So, here we are going to explore it step by step.

First we train a simple CNN network in Python on a 2000 Cat & Dog image dataset. No matter how much accuracy we achieve, we just save the model to use it in the c++ code :

import cv2
import glob
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.utils import to_categorical
from joblib import dump, load
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras import models
from tensorflow.keras import layers
import os
for item in glob.glob("Dataset/*/*"):
img = cv2.imread(item,cv2.IMREAD_GRAYSCALE) # read images with one channel grayscale
r_img= cv2.resize(img,(128,128)) # resize to 128x128
r_img = np.expand_dims(r_img, axis = -1)
data.append(r_img) # add resized image to dataset list
label = item.split("\\")[1]
labels.append(label) #add image label to dataset list
le = LabelEncoder()
labels = le.fit_transform(labels)
labels = to_categorical(labels)
data = np.array(data)/255 # Normalize channel between 0 to 1
#split test and train randomly
x_train, x_test, y_train, y_test = train_test_split(data,labels,test_size=0.2)
net= models.Sequential(
net.compile(optimizer="SGD", loss="binary_crossentropy",metrics=["accuracy"])
H = net.fit(x_train,y_train,batch_size=32, epochs=24, validation_data=(x_test,y_test))
net.save("CatDogNew.h5") # Save the model
view raw CatDogCNN.py hosted with ❤ by GitHub

After running the above code , we will have our model “CatDogNew.h5” file saved in project folder.

Before we start the next step we need to download the library from its repository on gihub. After download , find “keras_export” directory in the main folder of the library, and open command prompt from this directory :

keras model export to json
open command prompt in keras_export folder

Now, we should convert the model to json file using convert_model.py . Run the following command in the command line to build model in json format :

python convert_model.py path/to/your/model/CatDogNew.h5 CatDogNew.json

Running the above command will yield “CatDogNew.json” in the keras_export folder.

In this step which is so important, we should prepare our Visual Studio project to use the frugally-deep library. I use Visual Studio 2019, however there is no big difference between this version and its old ones. First, Create new Console App and choose a name for it.

keras model to cpp json
Create new Console App project

After creating the Project, you should have one cpp file opened with a main function containing “hello world” output. Now, It’s time to include library header file in project. Frugally-deep depends on 3 other libraries which they are also header-only and can be included without any pain. The libraries are :

  • Eigen
  • fplus
  • nlohmann

I gathered all of them in a zip file that you can download it with just a click here. I added the frugally-deep header files to the zip file so you don’t need to get it from source code downloaded from github.

Extract zip file and move the include folder to a place that is easy to address later. The include folder must contains all libraries like below :

tensorflow keras c plus c++
the extracted folder should contain 4 libraries

Open project properties from Project menu, in the left side panel click on VC++ Directories, then in right side edit the Include Directories row and insert the path to the already extracted include folder containing 4 libraries in the field :

keras model to cpp
Add extracted include folder address to project

As the the frugally-deep is a header-only library, it need no more configuration in project properties. So, we can go to the project’s main file and past following codes in it :

#include <iostream>
#include <fdeep/fdeep.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <filesystem>
#include <exception>
using namespace stdext;
namespace fs = std::filesystem;
int main()
std::string test_image_folder_path = "d:/catdog/"; // path to folder containing test images
const auto mymodel = fdeep::load_model("D:/CatDogNew.json"); // load the converted model
for (const auto& entry : fs::directory_iterator(test_image_folder_path)) {
std::string image_path = entry.path().string(); // get images path one by one
const cv::Mat image = cv::imread(image_path, cv::IMREAD_GRAYSCALE); // read the image in single channel grayscale mode
cv::Mat resizedImg;
cv::resize(image, resizedImg, cv::Size(128, 128)); // resize image to the 128x128 (Model input dimension)
imshow("Display Window", resizedImg); // just show the image
// convert cv::MAT to fdeep::tensor
const auto input = fdeep::tensor_from_bytes(resizedImg.ptr(),
0.0f, 1.0f);
auto result = mymodel.predict({ input }); // predict the image's label and ouput a 1x2 tensor containing each class probability
std::cout << fdeep::show_tensors(result) << std::endl; // print the tensor
catch (const std::exception& e)
std::cout << e.what() << '\n';

let’s see what happens in important parts of the code, line by line:

Line 1 to 8 is doing include jobs. First of all, we need to have OpenCV installed, to install and use OpenCV in c++ please see this post. Then we load the JSON model in line 16. We have accomplished half of the job, but the important part of the code is still remained, reading image and applying preprocess. We must feed the model with (128,128) single channel image that is normalized to [0-1] for each pixel. In line 21 we read the image in grayscale mode, then resize it in line 24. The normalization part still remained. We will do it when we are converting the image matrix to tensor. The tensor_from_bytes function in frugally-deep library does it for us in line 28. It takes 6 input parameters, first one is the image matrix pointer, the second and third parameters are the image height and image width. Fourth parameter is the number of channels. And very important last parameters are intended to re-scale the channel values from range [0, 255] to [low, high] in which the low and high must be specified with fifth and sixth parameters.

Finally we pass the input tensor to predict function, maybe you ask why we give it within {}, the answer is : frugally-deep supports models with multiple inputs , so we give each input in {}, separated by comma. The predict function output a tensor array that each element shows the corresponding class probability.

tensorflow keras c plus c++
Final Output that predicted the Cat with 58% of confidence

Congratulations! You have made your way to the end of this short journey. Our CNN model was trained in python, converted, and then used in C++.

Although using frugally-deep in your c++ project can be funny, you should know its pros and cons :


  • Performance: Since it’s a C++ library, it can offer better performance compared to running the same model in Python.
  • Deployment: Useful for deploying models in environments where Python is not an option (e.g., embedded systems).


  • Complex Models: There might be limitations in terms of the types of models and layers supported compared to running directly in Keras with TensorFlow.
  • Updates: Keeping up with the latest changes and features in Keras and TensorFlow might require updates to frugally-deep.

Thank you so much for reading this post!

13 thoughts on “TensorFlow-Keras Model, Train in Python, Use in C++

  1. That’s wonderful for me learning about relationship between Python and C++.
    Thank you very much.
    If you can display code work with RGB image, that will more helpful for me and other readers.

  2. First of all, thank you for posting a good article.
    I want to ask you any problem.
    What is causing the error when model load ?

    model full_model() <– model.hpp

  3. Hi,
    Is it faster than running the model with Python ?
    If so, do you have any idea on how much time do you gain from doing that ?

    Jean-Luc moosh

    1. There are several tools and libraries that allow you to export and run Pytorch models in C++. However, I didn’t use none of them:
      – TorchScript
      – ONNX
      – LibTorch

Leave a Reply

Your email address will not be published. Required fields are marked *