'What is the best approach for storing multiple vectors per person for face recognition
I want to make a face recognition for employees as work. I already have system that gets image from cameras and outputs face embeddings (128-dimensional vectors). So my next step, as far as I understand, is to compare these embeddings with the one stored somewhere in database and find one with nearest distance.
The problem is that I want to enable machine learning for this. Initially, on like every tutorial, only one photo of employee is used to create a reference embedding. But what if a want to store multiple embeddings for one person? For example, maybe this person came with glasses, or slightly changed appearance so that my system no longer recognises it. I want to be able to associate multiple embeddings with one person or another, creating a collection of embeddings for each employee, I think this would improve recognition system. And if in future my system will show me that there's unknown person, I could tell it that this embedding corresponds to specific person.
Is there any database that can store (maybe as array) or associate multiple vectors per person? I've looked into Milvus, FAISS, but didn't find anything about that.
I use Python 3.9 with OpenCV3, Tensorflow and Keras for creating embeddings.
Solution 1:[1]
Maybe you can store one id for one person with different vectors in milvus
Solution 2:[2]
If your embeddings come from different dimensions of a person, such as a face and a voiceprint. Then it makes sense to store two vector fields in milvus, one for the face vector and one for the voiceprint vector.
Solution 3:[3]
1- you can store many embeddings for a person. when you have a face to compare, then you will compare it to the many images of each person. then, find the average of the similarity and decide they are same person or different.
2- if you have many facial images for a person, then you will find embeddings for each photo then find an average embedding. suppose that you have 10 images for a person, you find 128-d embeddings for all of 10 photos. thereafter, you will find the average of each dimension and finally you will have one 128-d embedding.
i recommend you to store your embeddings in spotify annoy, facebook faiss, nmslib or elasticsearch. you can find implementations of deepface library for python with those vector databases with a basic google search.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | coolflower |
| Solution 2 | Jack |
| Solution 3 | johncasey |