Equitable Distribution of Suppliers in Ecommerce Platform

Hi team, I have an ecommerce use case where we have suppliers selling products. We have many different types of suppliers, some can have 90k different products to sell and others can have only 10 products. They usually have products in several categories. The problem of equitable distribution arises when searching products with "multi supplier queries", that is to say where multiple suppliers are supposed to be found in the results (e.g product_price < 50$), which is very often the case on big ecommerce platforms. One important rule for us is that a supplier cannot be "promoted" (kind of) just because he has 150 times more products then the others.
Until migrating to Typesense, here is what we did :
1. We calculate several scores for a product based on metrics like : its rating, its competitiveness, its order growth, is the supplier reactive, and so on, to have a global score.
2. This global score is used to sort products, to propose best scored products to clients according to their search queries.
3. For each search query, we find products that can match from every supplier concerned by the query. Then we distribute each product on a 20 products page where each supplier matched can have maximum 3 products displayed per page. All results are sorted by their product_score.
I am currently trying to reproduce this behavior in Typesense with the grouping parameters and the pagination parameters. I use a small dataset of 9946 products for tests.
My problem is that for a global query, I only find 40 documents out of 9946. With pages of 20 products, the third page is empty (big problem). I should have approximately 500 pages available, with maximum 3 products, for each supplier, per page of 20 products.
1. Am I doing something wrong or is there a parameter that I misunderstood ?
2. If the supplier distribution is not equitable, e.g supplier A have 9900 products in the results, and supplier B has 46, does the grouping parameters allow to have 10 products for supplier A and B on the first 4 pages, then the 5th page has 14 products of supplier A and 6 products of supplier B, and finally all remaining pages have supplier A products
3. More generally, this business rule is a common rule for a ecommerce platform with multiple suppliers proposing their products. Is there a better way to handle an equitable distribution ?
I give the supplier repartition of my test dataset and the query screenshot in the thread

> My problem is that for a global query, I only find 40 documents out of 9946.
IIRC, found parameter in a group query counts the number of groups, not the number of documents within each group

Hi Jason thank you for your quick response.
Can you explain why starting at page 15 I do not have any results ?

It sounds like there are only 40 unique suppliers in your dataset. So the pagination is based on the group rather than the individual items within the group

Yes I just understood this 5mins ago, my bad
Is it possible to base pagination on items within the group whereas the group itself ?

No it’s not possible, but you can calculate the pages you’re talking about based on the group_limit parameter you’ve set

Ok!
Do you have any suggestion/use case that would help me with the third and last question in the initial message of this thread ?

> More generally, this business rule is a common rule for a ecommerce platform with multiple suppliers proposing their products. Is there a better way to handle an equitable distribution ?
group_by would be the best way to achieve what you’re looking to do

How should I implement it to fit my use case ? Which is to have X suppliers maximum per page of 20 products for a query

When you group_by, you can set the per_page parameter which controls the total groups per page. So in your example, you’d set per_page to X, and then set group_limit to whatever 20 / X evaluates to

I'm sorry if I wasn't clear but I don't think this works for my use case. Like you said the pagination is based on the groups and not on items per group, so I won't be able to have the same supplier on several pages for equitable partition

For instance, imagine a search query with the following (non realistic) results :
Supplier A : 100 documents
Supplier B : 100 documents
Supplier C : 100 documents
Supplier D : 100 documents
Supplier E : 100 documents

My use case needs the following rule :
Maximum 3 products per supplier on each page (20 products per page)

So the results should follow this repartition:
Page 1 : 4 products for each supplier
Page 2 : 4 products for each supplier
Page 3 : 4 products for each supplier
etc.

This style of dynamic group sizes is not possible. You might be able to do this in two queries:

1. Do the query without a group_by, but facet on the supplier_id, which will give you the total number of suppliers.
2. In the next query, based the total number of suppliers obtained from query 1, set the value of group_limit dynamically