commit f9579994763988b068c0f9b1f7a278ff66b780e3
author David Ostrovsky <david@ostrovsky.org> Thu Jan 24 21:11:54 2019 +0100
committer David Ostrovsky <david@ostrovsky.org> Wed Oct 16 10:26:23 2019 +0200
tree a1bf1040a95fe5a5dadf62ce3d8c8070588bc1ab
parent 0fb1fd1936249f1d5f2758077a12243746f15d50

Migrate change index to use dimensional numeric types

Lucene 6.x deprecated IntField and replaced it with IntPoint that is
using different backend storage: [1]. Instead of continuing to
represent numeric data using a structure specifically designed and
tuned for text, the Bkd implementation introduced the first flexible
tree structure designed specifically for indexing discrete numeric
points: [1]. While the new data types are mostly the drop in
replacement for old IntField and LongField types, new type cannot be
used for document id types.

The previous migration from Lucene 5 to Lucene 6 switched to using
deprecated LegacyIntField type. In the next Lucene release 7, this
class and friends were extracted from Lucene distribution and moved
for one release to Apache Solr library. So theoretically we could
still use Apache Solr dependency by adding this dependency to Gerrit
and continue to use the old/deprecated/removed data types for one
major Lucene release.

We prefer forward migration strategy and switch to using string field
type as document id for account, change and groups indexes. Only change
index is handled in this commit. Other indexes are handled in follow-up
changes.

To support online migration, legacy numeric field types are still used
in the old index schema version, but new dimensional point field types
are used in new schema version. Old integer document id field type is
replaced with string type id in new change index schema. Therefore, in
different code paths it must be decided whether the legacy number field
types or the new dimensional point field types should be used depending
on the currently used index schema version. To support this logic, new
attribute is added to the index schema class: useLegacyNumericFields.

While this approach temporarily complicates the code, it can be removed
when a next gerrit version is released. Until then the deprecated type
classes are still used.

Non id fields are replaced with new IntPoint and LongPoint fields so
that we do not use any deprecated and removed features in Lucene and
could easily upgrade to the next major Lucene release without relying
on third party dependency (Apache Solr).

One side effect of this change is that ChangeQueryBuilder in the
AbandonUtil must be used with Guice provider. The reason for that is
because index collection must be accessed to retrieve schema instance,
to detect the useLegacyNumericFields attribute. Given that AbandonUtil
is bound in singleton scope, index collection is only provided when
multiversion index module is started. When the support for legacy
numeric field is removed in later gerrit releases this change can be
reverted.

[1] https://users.cs.duke.edu/~pankaj/publications/papers/bkd-sstd.pdf

Bug: Issue 11643
Change-Id: Icbc80d8a775a6ffea97e99717b24d3e8cacaee14