tree 8c7966fc6997f1a4695ba66798001659a47fbdc1
parent 1539ef83faa632f08d947a67e2c017dc049e1813
author Stefan Hengl <stefan.hengl@gmail.com> 1603466420 +0200
committer Han-Wen Nienhuys <hanwen@google.com> 1605803001 +0000

Use newline index for regexp content searches

For content searches trying to match multiple terms on the same line, we
check whether the matches of the individual terms intersect before
calling the regex engine. If they don't intersect, we skip the document.

This optimisation is useful whenever terms of the query appear often in
the same document but rarely on the same line.

The table below shows latencies of search queries of master vs.
this change on a set of medium-sized repos. All latencies are
reported in ms.

+-------------------------+-------------+-----------------+-----------+
| Query                   | Master (ms) | Patchset 7 (ms) | Delta (%) |
+-------------------------+-------------+-----------------+-----------+
| (func).*?(rtoorg)       | 21.43       | 2.67            | 87.56     |
+-------------------------+-------------+-----------------+-----------+
| (func).*?(bool).*?(bar) | 723.60      | 170.1           | 76.49     |
+-------------------------+-------------+-----------------+-----------+
| (func).*?(bool).*?      | 1320        | 186.2           | 85.89     |
| (int32).*?(error)       |             |                 |           |
+-------------------------+-------------+-----------------+-----------+
| (func).*?(return)       | 236.97      | 73.73           | 68.88     |
+-------------------------+-------------+-----------------+-----------+
| (config).*?(override)   | 827.90      | 328.67          | 60.30     |
+-------------------------+-------------+-----------------+-----------+

Change-Id: Ic4e4c3e7a4ad61ec56efada35a70aa14d0766ff7