Category "inference"

Use multiple images for batch inference cppflow C++

I'm trying to use cppflow library in windows 10 x64 machine in VS2019 C++. I want to inference my model for batch of images (vector <cv::Mat> ). I write a

How to speed up Tensorflow 2 keras model for inference?

So there's a big update nowadays, moving from TensorFlow 1.X to 2.X. In TF 1.X I got use to a pipeline which helped me to push my keras model to production. Th

OnnxRuntime vs OnnxRuntime+OpenVinoEP inference time difference

I'm trying to accelerate my model's performance by converting it to OnnxRuntime. However, I'm getting weird results, when trying to measure inference time. Whil