commit 76fed530a0fdc114ecfde3fec003d360e7ace671
author Dyrone Teng <dyroneteng@gmail.com> Wed Mar 18 11:20:03 2020 +0800
committer Dyrone Teng <dyroneteng@gmail.com> Wed Mar 18 15:32:26 2020 +0800
tree 042ece9a721c7705f5e327466768493f0efcefe9
parent c5243ea8f5fd916a31f97bbcabc0a84bcff3eea3

Support 'max_result_window' config for Elasticsearch indexes

Gerrit supports Elasticsearch as the index for search API. This is very
convenient for users to choose the index type they want. But when there
exists a large number of documents in the index of Elasticsearch, some
relative functions such as the front pages and APIs will become
unusable.

This is because of the default Elasticsearch 'max_result_window' setting
being 10000. When the query result (from + size) is greater than it, the
query will fail.

When gerrit executes a query on the index (such as the user is searching
on the Gerrit front pages, which makes a request to the Gerrit
backend API), then Gerrit will execute an HTTP request to the
Elasticsearch API to query the relative data and return the data back
to the front user to view the data they want.

But if the result set that exists in the index is greater than the
default value 10000, Gerrit will encounter this issue because
Elasticsearch API will return an error response to tip the invoker
that the result window is exceeded with the query.

There are three different solutions to the problem:

    1. Support 'max_result_window' config for Elasticsearch indexes. It
is the simplest solution and the common solution for multiple releases
of Gerrit (all Elasticsearch versions supported by Gerrit support this
setting).

    2. Use Elasticsearch Scroll Query API instead. This way decreases
the costs of query performance, but brings other problems. Elasticsearch
does not support traditional pagination by using scroll API and has
some limitation with the '_scoll_id'; it is not recommended to use it
for real-time user requests.

    3. Use Elasticsearch Search After API instead. It's the best way in
Elasticsearch to solve the costly deep pagination. But, it only supports
Elasticsearch versions greater than 6.2, while Gerrit still supports
5.6.

[1] https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-scroll
[2] https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-search-after

Bug: Issue 12444
Change-Id: I39da7d1d50df7bbe9dc88411632bb029c77f9f36