Could 'rerank-count' be a percentage instead of absolute?

[yuvalshimron]

Is your feature request related to a problem? Please describe.
When the number of eligible documents per query is very unpredictable it's very difficult to choose a good number for 'rerank-count'. Instead, a percentage (e.g. rerank-ratio: 0.1) could make better sense in some use cases. This is also true when the eligible documents fluctuate a lot during the day even for the same query.

Describe the solution you'd like
Specifying 'rerank-ratio' as a value in (0, 1], as an alternative to an absolute value that is given in 'rerank-count'.

[bratseth]

A percentage of the matched docs?

[yuvalshimron]

I guess there are two options:

1. a percentage of the matched docs (which we ranked in first phase)
2. a percentage of the docs after first phase (could in theory be less if there is a rank score drop limit, right?)

[bratseth]

Yes, you are right it should be counted after any phase 1 dropping. We also want to make it global, not per node, to simplify scaling.

But second phase may be the most expensive part of query evaluation. Do you really want it to be 1000x more expensive for a query which matches a thousand times as many documents?

I think you also want it to be a (>100 global) percentage of the number of hits to return, regardless of the number of matches.

[yuvalshimron]

I understand what you are saying, but even if the intension is to only penalize the 'heavy' queries, it's very hard to come up with the right re-rank-count value and make sure it stays the right value over time.

For example, if the avg eligible docs fluctuates a lot during the day, this tuning probably needs to happen during peak time (when avg eligible docs is high) which might not be easy to do. It's also likely that this number changes over time and impacts among others how autoscale (or someone) scales the system up or down (while the re-rank count stays the same and doesn't adjust to the new state). I just think a percentage gives a more predictable boost, but I understand it doesn't necessarily make sense to penalize 'small' queries as well.

By the way - if today the total-matched-docs <= rerank-count do we skip first-phase? If so then it might not be that of an issue to use a constant number, but if we still pay a big static cost then that's also why I would have preferred to penalize all queries.

[bratseth]

Yes, what you are saying makes sense, I just think we need something a bit more complicated which clamps the cost increase with many matches and the quality decrease with few.

We always evaluate first-phase because it happens during matching, before the total match count is known.