How much embeddings needed to have good accuracy in search

I am using milvus for my face recognition application. I have created 75 embedding. one person having 15 embeddings after augmentation. I am using IVF_FLAT index type with metric type IP (now previously i used L2 but did not get accuracy difference) nlist 256 and nprob 32 . The embedding distance between the target image to other embeddings are coming very less which is nearly 0.25 to 0.30 but with its own image embedding is very high nearly 0.90 to 0.99.

I did tweaking with parameters nprobe but not getting good accuracy for searching. please suggest what can i do?