Test against judged results
Each evaluation query has relevance judgments for candidate pages. After a ranking or index change, Argand reruns the set and compares the new scores with the current release.
Search quality
How Argand tests its results.
The beta is imperfect. These are the tests used to find regressions, measure improvements, and decide whether a new index can ship.
The target
Put the useful source first.
The first result should be the page that best answers the query.
That is a target, not a claim that every beta query already succeeds. The rest of this page explains how Argand measures the gap between the target and the current results.
How we check
Three checks run before release.
The automated gate compares each candidate with the current release. Human review covers queries and failure modes that a public benchmark does not represent.
Measure the top of the list
MRR measures how often the first relevant result appears near the top. NDCG@10 measures the quality and order of the first ten results. Both matter to the release gate.
Block material regressions
A candidate that exceeds the allowed regression tolerance is rejected automatically. The tolerance and the current benchmark results are published on the Benchmarks page.
The proof
The public benchmarks can be rerun.
Argand is tested on public collections of queries, documents, and relevance judgments. The current medical, scientific, and financial results are compared with the same reference engine on the same data.
The datasets and scoring method are public. The Benchmarks page records the commands and configuration needed to reproduce the reported scores, along with what cannot yet be checked outside the project.
See the real scores and re-run them yourselfWhere it's weak
Known weaknesses in the beta.
Long conversational queries remain less reliable than short, specific ones. The index also has uneven coverage across languages and commercial sites. Reported failures and privacy-limited aggregates become regression cases where possible. The Privacy page describes the data boundary; the Roadmap lists the larger gaps.
What we won't do
Constraints on how quality improves.
- Sources remain the destination. A result opens the publisher's page. Argand does not generate a substitute answer from that page.
- Ranking does not depend on a behavioural profile. Argand uses the query, the index, and explicit settings. Anonymous searches are not joined into a personal history.
- Spam handling is part of relevance. Duplicate, deceptive, and low-value pages are detected and penalized instead of being treated as equally valid candidates.
Want the deeper dives? See the benchmarks, how little energy a search costs on Energy, and the why on About.