Category "inference"

Pytorch models deployment to sagemaker

I have trained my yolov5 model, and have weights.pt, now I need to deploy it using sagemaker, for that I need to create an endpoint. I'm following this tutoriel

Use multiple images for batch inference cppflow C++

I'm trying to use cppflow library in windows 10 x64 machine in VS2019 C++. I want to inference my model for batch of images (vector <cv::Mat> ). I write a

How to speed up Tensorflow 2 keras model for inference?

So there's a big update nowadays, moving from TensorFlow 1.X to 2.X. In TF 1.X I got use to a pipeline which helped me to push my keras model to production. Th

OnnxRuntime vs OnnxRuntime+OpenVinoEP inference time difference

I'm trying to accelerate my model's performance by converting it to OnnxRuntime. However, I'm getting weird results, when trying to measure inference time. Whil