Introduce a SEARCH_AFTER index pagination type
Current Paginated interface only allows to restart a query from
a given offset. Update it to also allow restarting queries using
a searchAfter key. searchAfter can help with performance of queries
such as [1] where the visibility is being queried for an account
that cannot see most/all changes. [1] finishes in ~45 mins with
OFFSET and ~10 mins with SEARCH_AFTER when the site has around
1M abandoned changes.
Index config has been updated to add a paginationType entry with
OFFSET and SEARCH_AFTER as supported values. The default is set
to OFFSET to keep the current pagination type unchanged.
Gerrit's Lucene index implementation has been updated to support
this new pagination type with search-after[2]. Elasticsearch
index implementation has also been updated to paginate using
search-after[3] and can be further improved to use PIT[4]. Lucene
and Elasticsearch tests have been updated to run with both pagination
types. Also, Elasticsearch tests now use the official docker image
as it can handle running tests with multiple index config suites.
Note that, searchAfter will not impact using offsets in Gerrit
query APIs.
[1] gerrit query 'status:abandoned visibleto:guest'
[2] https://lucene.apache.org/core/6_6_5/core/org/apache/lucene/search/IndexSearcher.html#search-org.apache.lucene.search.Query-int-org.apache.lucene.search.Sort-boolean-boolean-
[3] https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after
[4] https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html
Release-Notes: Index searches now support search-after pagination
Change-Id: If3f8d914d5fd3f350c026cf099f08cf9a13f2195
diff --git a/Documentation/config-gerrit.txt b/Documentation/config-gerrit.txt
index 9dea08e..121ac8f 100644
--- a/Documentation/config-gerrit.txt
+++ b/Documentation/config-gerrit.txt
@@ -3117,6 +3117,25 @@
+
Defaults to true.
+[[index.paginationType]]index.paginationType::
++
+The pagination type to use when index queries are repeated to
+obtain the next set of results. Supported values are:
++
+* `OFFSET`
++
+Index queries are repeated with a non-zero offset to obtain the
+next set of results.
++
+* `SEARCH_AFTER`
++
+Index queries are repeated using a search-after object. This type
+is supported for Lucene and ElasticSearch. Other index backends can
+provide their custom implementations for search-after. Note that,
+`SEARCH_AFTER` does not impact using offsets in Gerrit query APIs.
++
+Defaults to `OFFSET`.
+
[[index.maxLimit]]index.maxLimit::
+
Maximum limit to allow for search queries. Requesting results above this
