Ranking
Diversity without wrecking relevance
Maximal marginal relevance with a λ you can drag — the point where six nearly identical results become six useful ones, and the point just past it where the list stops answering the question.
Sort by relevance, take the top six, ship it. The results are all excellent and the list is nearly useless, because five of them say the same thing.
This is not a bug in the ranker. It is the ranker working exactly as specified: the documents scoring highest against a query are, very often, the documents most similar to each other. Relevance is not a diverse objective, and nothing in will ever complain.
The greedy fix
Maximal marginal relevance builds the result list one slot at a time. At each step it picks the candidate with the best trade between being relevant and being different from what is already in the list:
is the candidate pool, is what you have chosen so far, and slides between the two objectives. At the second term vanishes and you get the original ranking back, unchanged. At relevance is ignored entirely and it just hunts for whatever is most unlike the current set.
Note the , not a sum or a mean. A candidate is penalized by its similarity to the single nearest thing already selected — one near-duplicate is enough to disqualify it, no matter how different it is from the rest.
Drag λ
Fourteen candidates for “how do I combine two rankers”, spread across five topics. Six slots. The right-hand column is the pure-relevance ranking for comparison; anything MMR pulled in that relevance alone would have missed is marked.
Where the interesting region is
Start at λ = 1. Five of the six results are about rank fusion, and four of those are near-restatements of each other. Every one is a good answer. The list is still bad.
Slide down to about 0.7. One or two fusion articles drop out and evaluation and dense retrieval appear. Mean relevance has barely moved — you gave up a few hundredths — and the list now answers the question from more than one angle. For most surfaces this region is where you want to live.
Keep going to 0.3. Infrastructure and cold-start results push in. These are genuinely less relevant to someone asking how to combine two rankers, and the list is starting to feel like it did not understand the question.
At λ = 0 the query has been abandoned completely. The six results are simply the six most mutually different documents in the pool.
The curve at the top sweeps λ across its whole range so you can see the trade rather than infer it. Coming down from λ = 1, diversity rises steeply while mean relevance falls barely at all — you are buying a lot of variety very cheaply. Somewhere around 0.6 the two lines cross and the trade inverts. That knee is the parameter you actually want, and it moves with the query set, which is why it belongs in an experiment rather than a constant.
When not to do this
Diversification is a bet that you might have misunderstood the query. That bet is worth taking when intent is ambiguous, when the surface is a feed the user will scan, or when the cost of showing one irrelevant result is low.
It is a bad bet when the user knows exactly what they want. Someone searching for a specific error message, an order number, a person’s name — for them, six near-identical results are the right answer, and every slot you spend on variety is a slot spent on something they did not ask for. Known-item search should not be diversified.
The useful signal is usually query specificity: short, broad, ambiguous queries want more diversity, long and specific ones want less. Some systems make a function of that rather than a constant.
Two things this does not solve
MMR can only diversify what candidate generation gave it. If your retrieval stage returned fourteen documents and eleven are about rank fusion, no value of will produce a balanced list — it can only pick the least-similar three from a pool that was never diverse. Real diversity work happens further upstream, in how candidates are retrieved, and it is considerably more expensive than a reranking pass.
Your relevance metrics cannot see the improvement. NDCG will go down when you diversify, because you deliberately demoted relevant documents. That is not a regression, but the metric cannot tell the difference. If you are going to diversify, you need a diversity-aware metric (α-NDCG, subtopic recall) or an online experiment measuring something further downstream — otherwise your offline evaluation will faithfully report that you made the product worse.