I have a data set of 800K and 1.2 million from two e-commerce marketplaces. Each product has attributes such as product name, price, specifications, product URLs, product descriptions, etc. I’m trying to implement a product-matching system that compares products and gives me products that are a match and also similar.
- Website A has iPhone 14, 256 GB black color phone, Website B has iPhone 14, 256 GB black color phone - then the system should return a match that says “same product”.
- Website A has iPhone 14, 256 GB black color phone, and Website B has iPhone 14, 256 GB white color phone. It should return the response “similar products, but not same”.
- Website A has iPhone 14, 256 GB black color phone, and Website B has iPhone 13, 256 GB black color phone. It should return " different products".
I’m researching on different ways to implement this and came across Milvus as a potential option. I’d appreciate it If anyone can give guidance or their thoughts on how to use Milvus for this use case.