Merge branch 'stable-3.5'
* stable-3.5:
Update git submodules
Remove jackson-* libraries from Gerrit
Tolerate null values for insertions/deletions
GitProtocolV2IT: Match trace output from newer git
Remove ES leftovers
Set version to 3.5.1-SNAPSHOT
Set version to 3.5.0.1
Set version to 3.5.1-SNAPSHOT
Set version to 3.5.0
Revert "Increase hover delay for token highlight"
Remove unused variable
Disable logging cache stats for new sites
init: Invoke reindex only once.
reindex: Print cache statistics only once at the end of program
Special char warning
Indicate soft hyphen in diff
Set version to 3.5.0-SNAPSHOT
Set version to 3.5.0-rc4
Skip external ids migration for accounts with case insensitive SHA1
Handle duplicate external ids when in migration mode
Run DefaultIndexBindingIT only for index.type=fake
Update replication plugin to latest master
Update git submodules
Update git submodules
Update git submodules
Fix listening on ENTER shortcut in <gr-file-list>
Increase hover delay for token highlight
Introduce a pluggable indexing libModule type
Rename libModule types adding the _TYPE suffix
Update git submodules
Add Thomas and Patrick to the lists of maintainers
Update git submodules
Document new diff caches in config-gerrit
LuceneChangeIndex: remove redundant ChangeData lookup
Remove redundant type parameter for TypeLiteral<>
Set version to 3.5.0-SNAPSHOT
Set version to 3.5.0-rc3
Set version to 3.4.3-SNAPSHOT
Set version to 3.4.2
Set version to 3.3.9-SNAPSHOT
Set version to 3.3.8
Set version to 3.2.15-SNAPSHOT
Set version to 3.2.14
Add documentation for the external id notes case insensitivity migration
Add online migration to migrate external IDs to work case insensitively
Set version to 2.16.29-SNAPSHOT
Set version to 2.16.28
Document userNameCaseInsensitive defaults to true for new sites
Remove support for ElasticSearch indexes
Use JGit 5.1.15.202012011955-r javadoc to resume release build
Remove reference to deprecated metrics-reporter-elasticsearch
Bazel: Remove deprecated bazel-toolchain dependency
Avoid lucene index deletes during offline reindexing
Fix mismerge of I359578a
Remove bazel workaround
Fix group suggestions
Fix group suggestions
Update replication plugin to latest master
Display cache stats after reindex operation
Fix AllProjectsIndexer to avoid duplicate reindex work
Rename ProjectCache.evict() to ProjectCache.evictAndReindex()
AllChangesIndexer: Parallelize project slice creation
AllChangesIndexer: Avoid scanning for change refs in each slice
Fix tests compilation
Set version to 3.5.0-SNAPSHOT
Set version to 3.5.0-rc2
VersionMetaData: Don't close passed in RevWalk
Allow context-dependent group suggestions in gr-permission
Allow context-dependent group suggestions in gr-permission
Fix disabling repo configs if you do not have permissions to edit
Move AutoFlush to 'gerrit.server.index.options' package
AccessIT: Add tests for when group appears twice for same rule
Add p param to QueryGroupsParams
Add import to gr-registration-dialog_test
Add migration mode for case insensitive external IDs
Prevent infinite loops with GWT UI and HTTP auth
Fix BatchMetaDataUpdate to not close passed in object reader
AllChangesIndexer: skip creating slices for projects with no changes
Bump auto-value-gson to 1.3.1
fixup!: Disable auto flushing during offline Lucene indexing
Don't retain body in RevWalk for Change meta reachability check
Move verification of GetChange meta-ref id to the API layer
Eclipse settings: configure build to use JavaSE-11
Add missing "--migrate-draft-to" flag on init doc
AllChangesIndexer: Show total failed changes
Fix buggy Index-Interactive Executor
reindex: Use thread count specified on command line
Disable auto flushing during offline Lucene indexing
Set version to 3.5.0-SNAPSHOT
Set version to 3.5.0-rc1
Avoid creating loose objects in Schema 146
Avoid creating loose objects in schema 167
Stop using experiment flag for Token Highlighting
Upgrade lit for plugins from 2.0.0-rc.3 to 2.0.2
Fix schemas to use the appropriate ObjectInserter
Add REST endpoint for commits included in refs
Update schemas 115,119 to create a desired initial commit
Create initial commit in schema 146 only when necessary
Avoid creating loose objects in schemas 115 and 119
Use JGit's RevWalk.getMergedInto instead of IncludedInResolver
Fix position of titles for related changes
Toggling theme reloads the page
Only force the comment re-creation workaround for Safari
doc: remove fragment from dashboard examples
Update JGit to 60b81c5a9280e44fa48d533a61f915382b2b9ce2
Ignore patchset level comments when computing unresolved count
Do not pop up reviewer suggestions before typing a character
Fix Enter shortcut for comments within file-list
Ignore patchset level comments when computing unresolved count
Parallelize inserts into accountPatchReviewDb in schema 127
Set version to 3.5.0-SNAPSHOT
Set version to 3.5.0-rc0
Avoid listing repositories multiple times during schema migrations
Parallelize Schema 108
Avoid creating loose objects in Schema 154
Parallelize Schema 130 and 131
Avoid creating loose objects in Schema 144
Avoid creating loose objects in Schema 139
Avoid creating loose objects in Schema 124
Fix gr-tooltip test
Update schemas 115,139,144 to ignore entries not in the 'accounts' table
Upgrade JGit to 60b81c5a9280e44fa48d533a61f915382b2b9ce2
Set version to 3.3.8-SNAPSHOT
Set version to 3.3.7
Reuse the same Repository when listing changes by status
Reuse the already opened Repository for refs filtering
Load an arbitrary version of change notes
Set version to 3.2.14-SNAPSHOT
Set version to 3.2.13
Doc: Fix asciidoctor warning
Reuse the already opened Repository in ReceiveCommit
Change bouncycastle urls
Add utf-8 support in the license generator
Allow loading change notes from an existing Repository
DRY out set config of Git protocol v2 in tests
Unify getRefs() and getRefsByPrefix() in permission-aware refdb
rest-api-projects: Fix typo referring to /groups/
Log the result of git-upload-pack command in httpd_log
Fix serialization of AllUsersName and AllProjectsName
Ignore the Rule-Name key in submit record footers
Fix DynamicOptions to invoke listeners registered to BeanParseListener
Update git submodules
Update jgit to 84707715108a65a366ef35f2ae04aabecd0b35f6
Add PluginServletContext#getVirtualServerName
Introduce "--reindex-threads" option
ListAccess: Fix incorrect behavior when group appears twice for same rule
Allow moving of merge commits
Bump Bazel version to 4.2.0
Introduce flag '--migrate-draft-to'
PolyGerrit: replace event.path with event.composedPath
PolyGerrit: replace event.path with event.composedPath
OutgoingEmail: Handle null accountId in getUserNameEmailFor(Account.Id)
NoteDbMigrator: Make shuffling project slices optional
NoteDbMigrator: Make GC on repositories optional
OutgoingEmail: Handle null accountId in getNameEmailFor(Account.Id) method
OnlineNoteDbMigration: allow per-project migration
Update jgit to 5.1.16.202106041830-r
NoteDbMigrator: Warm PostgreSQL DB before migration
doc: document how to get flat html doc files
Change-Id: Icb42ed158b658771aa9251815f13a652b1f53a66
diff --git a/.bazelrc b/.bazelrc
index 3f9335c..db1fd57 100644
--- a/.bazelrc
+++ b/.bazelrc
@@ -13,10 +13,6 @@
build --announce_rc
-# Workaround Bazel worker crash (remove after upgrading to 4.1.0)
-# https://github.com/bazelbuild/bazel/issues/13333
-build --experimental_worker_multiplex=false
-
test --build_tests_only
test --test_output=all
test --java_toolchain=//tools:error_prone_warnings_toolchain_java11
diff --git a/.bazelversion b/.bazelversion
index fcdb2e1..6aba2b2 100644
--- a/.bazelversion
+++ b/.bazelversion
@@ -1 +1 @@
-4.0.0
+4.2.0
diff --git a/.settings/org.eclipse.jdt.core.prefs b/.settings/org.eclipse.jdt.core.prefs
index 8efbd11..09ce63b 100644
--- a/.settings/org.eclipse.jdt.core.prefs
+++ b/.settings/org.eclipse.jdt.core.prefs
@@ -10,9 +10,9 @@
org.eclipse.jdt.core.compiler.annotation.nullanalysis=disabled
org.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode=enabled
org.eclipse.jdt.core.compiler.codegen.methodParameters=do not generate
-org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.8
+org.eclipse.jdt.core.compiler.codegen.targetPlatform=11
org.eclipse.jdt.core.compiler.codegen.unusedLocal=preserve
-org.eclipse.jdt.core.compiler.compliance=1.8
+org.eclipse.jdt.core.compiler.compliance=11
org.eclipse.jdt.core.compiler.debug.lineNumber=generate
org.eclipse.jdt.core.compiler.debug.localVariable=generate
org.eclipse.jdt.core.compiler.debug.sourceFile=generate
@@ -29,6 +29,7 @@
org.eclipse.jdt.core.compiler.problem.deprecationWhenOverridingDeprecatedMethod=disabled
org.eclipse.jdt.core.compiler.problem.discouragedReference=warning
org.eclipse.jdt.core.compiler.problem.emptyStatement=warning
+org.eclipse.jdt.core.compiler.problem.enablePreviewFeatures=disabled
org.eclipse.jdt.core.compiler.problem.enumIdentifier=error
org.eclipse.jdt.core.compiler.problem.explicitlyClosedAutoCloseable=warning
org.eclipse.jdt.core.compiler.problem.fallthroughCase=warning
@@ -87,6 +88,7 @@
org.eclipse.jdt.core.compiler.problem.redundantSuperinterface=ignore
org.eclipse.jdt.core.compiler.problem.reportMethodCanBePotentiallyStatic=ignore
org.eclipse.jdt.core.compiler.problem.reportMethodCanBeStatic=ignore
+org.eclipse.jdt.core.compiler.problem.reportPreviewFeatures=warning
org.eclipse.jdt.core.compiler.problem.specialParameterHidingField=disabled
org.eclipse.jdt.core.compiler.problem.staticAccessReceiver=warning
org.eclipse.jdt.core.compiler.problem.suppressOptionalErrors=disabled
@@ -126,4 +128,5 @@
org.eclipse.jdt.core.compiler.problem.unusedWarningToken=warning
org.eclipse.jdt.core.compiler.problem.varargsArgumentNeedCast=warning
org.eclipse.jdt.core.compiler.processAnnotations=enabled
-org.eclipse.jdt.core.compiler.source=1.8
+org.eclipse.jdt.core.compiler.release=enabled
+org.eclipse.jdt.core.compiler.source=11
diff --git a/Documentation/backend_licenses.txt b/Documentation/backend_licenses.txt
index 3113682..586406f 100755
--- a/Documentation/backend_licenses.txt
+++ b/Documentation/backend_licenses.txt
@@ -61,11 +61,8 @@
* guice:guice-library
* guice:guice-servlet
* guice:javax_inject
-* httpcomponents:httpasyncclient
* httpcomponents:httpclient
* httpcomponents:httpcore
-* httpcomponents:httpcore-nio
-* jackson:jackson-core
* jetty:http
* jetty:io
* jetty:jmx
@@ -1113,22 +1110,6 @@
----
-[[elasticsearch]]
-elasticsearch
-
-* elasticsearch-rest-client:elasticsearch-rest-client
-
-[[elasticsearch_license]]
-----
-Elasticsearch
-Copyright 2009-2015 Elasticsearch
-
-This product includes software developed by The Apache Software
-Foundation (http://www.apache.org/).
-
-----
-
-
[[flexmark]]
flexmark
diff --git a/Documentation/backup.txt b/Documentation/backup.txt
index dd47035..9139e71 100644
--- a/Documentation/backup.txt
+++ b/Documentation/backup.txt
@@ -45,10 +45,6 @@
It can be recomputed from primary data in the git repositories but
reindexing may take a long time hence backing up the index makes sense
for production installations.
-+
-If you have chosen to use _Elastic Search_ for indexing,
-refer to its
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html[backup documentation,role=external,window=_blank].
[#optional-backup-cache]
Caches::
diff --git a/Documentation/cmd-index.txt b/Documentation/cmd-index.txt
index ab341e8..ce3a024 100644
--- a/Documentation/cmd-index.txt
+++ b/Documentation/cmd-index.txt
@@ -157,6 +157,9 @@
link:cmd-ls-user-refs.html[gerrit ls-user-refs]::
Lists refs visible for a specified user.
+link:cmd-migrate-externalids-to-insensitive.html[gerrit migrate-externalids-to-insensitive]::
+ Migrate external-ids to case insensitive.
+
link:cmd-plugin-install.html[gerrit plugin add]::
Alias for 'gerrit plugin install'.
diff --git a/Documentation/cmd-migrate-externalids-to-insensitive.txt b/Documentation/cmd-migrate-externalids-to-insensitive.txt
new file mode 100644
index 0000000..b023089
--- /dev/null
+++ b/Documentation/cmd-migrate-externalids-to-insensitive.txt
@@ -0,0 +1,44 @@
+= gerrit migrate-externalids-to-insensitive
+
+== NAME
+gerrit migrate-externalids-to-insensitive - Migrate external-ids to case insensitive.
+
+== SYNOPSIS
+[verse]
+--
+_ssh_ -p <port> <host> _gerrit migrate-externalids-to-insensitive_
+--
+
+== DESCRIPTION
+This command allows to trigger online conversion of `username` and
+`gerrit` external IDs to be handled case insensitively. This is done by
+recomputing the name of the note from the sha1 sum of the all lowercase
+external ID key, instead of preserving the key capitalization.
+
+The command requires link:#auth.userNameCaseInsensitive[auth.userNameCaseInsensitive] and
+link:#auth.userNameCaseInsensitiveMigrationMode[auth.userNameCaseInsensitiveMigrationMode] to
+be set to true to perform the migration.
+
+After the successful migration
+link:#auth.userNameCaseInsensitiveMigrationMode[auth.userNameCaseInsensitiveMigrationMode] is
+set to false.
+
+== ACCESS
+Caller must be a member of the privileged 'Administrators' group.
+
+== SCRIPTING
+This command is intended to be used in scripts.
+
+== EXAMPLES
+Start the online external ids migration:
+
+----
+$ ssh -p 29418 review.example.com gerrit migrate-externalids-to-insensitive
+----
+
+GERRIT
+------
+Part of link:index.html[Gerrit Code Review]
+
+SEARCHBOX
+---------
diff --git a/Documentation/config-accounts.txt b/Documentation/config-accounts.txt
index 7a7cef2..aca9591 100644
--- a/Documentation/config-accounts.txt
+++ b/Documentation/config-accounts.txt
@@ -111,8 +111,8 @@
Accessing the account data in Git is not fast enough for account
queries, since it requires accessing all user branches and parsing
all files in each of them. To overcome this Gerrit has a secondary
-index for accounts. The account index is either based on
-link:config-gerrit.html#index.type[Lucene or Elasticsearch].
+index for accounts. The account index is based on
+link:config-gerrit.html#index.type[Lucene].
Via the link:rest-api-accounts.html#query-account[Query Account] REST
endpoint link:user-search-accounts.html[generic account queries] are
diff --git a/Documentation/config-gerrit.txt b/Documentation/config-gerrit.txt
index a56055a..248cb8b 100644
--- a/Documentation/config-gerrit.txt
+++ b/Documentation/config-gerrit.txt
@@ -666,8 +666,9 @@
the link:config-accounts.html#external-ids[External ID documentation].
+
Gerrit provides the
-link:pgm-ChangeExternalIdCaseSensitivity.html[ChangeExternalIdCaseSensitivity tool]
-to migrate existing accounts to match the new scheme.
+link:pgm-ChangeExternalIdCaseSensitivity.html[offline]
+and the online link:externalid-case-insensitivity.html#online-migration[online]
+tools to migrate existing accounts to match the new scheme.
+
Naturally, if there were two accounts only different in capitalization,
e.g. `johndoe` and `JohnDoe`, the account `JohnDoe` will not be able
@@ -676,6 +677,16 @@
note name would be identical and thus conflict. These duplicates thus
have to be deleted manually by deleting the respective external ID.
+
+For newly initialized sites this option defaults to true.
++
+Default is false.
+
+[[auth.userNameCaseInsensitiveMigrationMode]]auth.userNameCaseInsensitiveMigrationMode::
++
+Setting migration mode to true allows to fallback to case sensitive
+behaviour if the migrated external ID cannot be found. This allows to
+trigger the migration while Gerrit process is running.
++
Default is false.
[[auth.enableRunAs]]auth.enableRunAs::
@@ -832,18 +843,21 @@
entry is relatively the same, memoryLimit is currently defined to be
the number of entries held by the cache (each entry costs 1).
+
-For caches where the size of an entry can vary significantly between
-individual entries (notably `"diff"`, `"diff_intraline"`), memoryLimit
-is an approximation of the total number of bytes stored by the cache.
-Larger entries that represent bigger patch sets or longer source files
-will consume a bigger portion of the memoryLimit. For these caches the
-memoryLimit should be set to roughly the amount of RAM (in bytes) the
-administrator can dedicate to the cache.
+For caches where the size of an entry can vary significantly between individual
+entries (notably `"git_modified_files"`, `"modified_files"`, `"git_file_diff"`,
+`"gerrit_file_diff"`, `"diff_intraline"`), memoryLimit is an approximation of
+the total number of bytes stored by the cache. Larger entries that represent
+bigger patch sets or longer source files will consume a bigger portion of the
+memoryLimit. For these caches the memoryLimit should be set to roughly the
+amount of RAM (in bytes) the administrator can dedicate to the cache.
+
Default is 1024 for most caches, except:
+
* `"adv_bases"`: default is `4096`
-* `"diff"`: default is `10m` (10 MiB of memory)
+* `"git_modified_files"`: default is `10m` (10 MiB of memory)
+* `"modified_files"`: default is `10m` (10 MiB of memory)
+* `"git_file_diff"`: default is `10m` (10 MiB of memory)
+* `"gerrit_file_diff"`: default is `10m` (10 MiB of memory)
* `"diff_intraline"`: default is `10m` (10 MiB of memory)
* `"diff_summary"`: default is `10m` (10 MiB of memory)
* `"external_ids_map"`: default is `2` and should not be changed
@@ -968,16 +982,38 @@
The cache should be flushed whenever the database changes table is modified
outside of Gerrit.
-cache `"diff"`::
+cache `"git_modified_files"`::
+
-Each item caches the differences between two commits, at both the
-directory and file levels. Gerrit uses this cache to accelerate
-the display of affected file names, as well as file contents.
+Each item caches the list of git modified files between two git trees
+corresponding to two different commits. This cache does not read the actual
+file contents nor does it include the edits (modified regions) of the files.
+
+cache `"modified_files"`::
++
+Each item caches the list of modified files between two commits. This cache
+is similar to the `git_modified_files` cache but performs extra logic including
+filtering out files that are untouched by both commits because they were purely
+modified between the parent commits.
+
+cache `"git_file_diff"`::
++
+Each item caches the pure git diff between two git trees for a specific file
+path. The diff includes all the file attributes (old/new paths, change/patch
+types) as well as the list of edits corresponding to the modified regions in
+the file.
+
+cache `"gerrit_file_diff"`::
++
+Each item caches the diff between two git commits for a specific file path.
+This cache is similar to the `git_file_diff` cache but performs extra logic
+including identifying the edits that are due to rebase. The diff for the
+"commit message" and "merge list" can also be requested from this cache.
+
Entries in this cache are relatively large, so memoryLimit is an
estimate in bytes of memory used. Administrators should try to target
cache.diff.memoryLimit to fit all changes users will view in a 1 or 2
-day span.
+day span. The same applies for other diff caches: `"git_modified_files"`,
+`"modified_files"` and `"git_file_diff"`.
cache `"diff_intraline"`::
+
@@ -1200,9 +1236,9 @@
==== [[cache_options]]Cache Options
-[[cache.diff.timeout]]cache.diff.timeout::
+[[cache.git_file_diff.timeout]]cache.git_file_diff.timeout::
+
-Maximum number of milliseconds to wait for diff data before giving up and
+Maximum number of milliseconds to wait for git diff data before giving up and
falling back on a simpler diff algorithm that will not be able to break down
modified regions into smaller ones. This is a work around for an infinite loop
bug in the default difference algorithm implementation.
@@ -1324,15 +1360,16 @@
[[change.cacheAutomerge]]change.cacheAutomerge::
+
-When reviewing merge commits, the left-hand side shows the output of the
-result of JGit's automatic merge algorithm. This option controls whether
-this output is cached in the change repository, or if only the diff is
-cached in the persistent `diff` cache.
+When reviewing merge commits, the left-hand side shows the output of the result
+of JGit's automatic merge algorithm. This option controls whether this output is
+cached in the change repository, or if only the diff is cached in the persistent
+diff caches (`"git_modified_files"`, `modified_files`, `"git_file_diff"`,
+`"file_diff"`).
+
If true, automerge results are stored in the repository under
`refs/cache-automerge/*`; the results of diffing the change against its
-automerge base are stored in the diff cache. If false, no extra data is
-stored in the repository, only the diff cache. This can result in slight
+automerge base are stored in the diff caches. If false, no extra data is
+stored in the repository, only the diff caches. This can result in slight
performance improvements by reducing the number of refs in the repo.
+
Default is true.
@@ -2282,6 +2319,25 @@
+
By default unset.
+[[gerrit.installIndexModule]]gerrit.installIndexModule::
++
+Class name of the Guice modules to load as alternate implementation
+for the Gerrit indexes backend.
+Classes are resolved using the primary Gerrit class loader, hence the
+class needs to be either declared in Gerrit or an additional JAR
+located under the `/lib` directory.
++
+NOTE: The `gerrit.installIndexModule` has precedence over the
+`index.type`.
++
+By default unset.
++
+Example:
+----
+[gerrit]
+ installIndexModule = com.google.gerrit.elasticsearch.ElasticIndexModule
+----
++
[[gerrit.installModule]]gerrit.installModule::
+
Repeatable list of class name of additional Guice modules to load at
@@ -3101,23 +3157,13 @@
[[index.type]]index.type::
+
-Type of secondary indexing employed by Gerrit. The supported
-values are:
+*(DEPRECATED)* The only supported value is `LUCENE`, which is the default,
+that means a link:http://lucene.apache.org/[Lucene]
+index is used.
+
-* `LUCENE`
+For using other indexing backends (e.g. ElasticSearch), refer to
+`gerrit.installIndexModule` setting.
+
-A link:http://lucene.apache.org/[Lucene] index is used.
-+
-+
-* `ELASTICSEARCH` look into link:#elasticsearch[Elasticsearch section]
-+
-An link:https://www.elastic.co/products/elasticsearch[Elasticsearch,role=external,window=_blank] index is
-used. Refer to the link:#elasticsearch[Elasticsearch section] for further
-configuration details.
-
-+
-By default, `LUCENE`.
-
[[index.threads]]index.threads::
+
Number of threads to use for indexing in normal interactive operations. Setting
@@ -3130,7 +3176,7 @@
[[index.batchThreads]]index.batchThreads::
+
Number of threads to use for indexing in background operations, such as
-online schema upgrades, and also for offline reindexing.
+online schema upgrades, and also the default for offline reindexing.
+
If not set or set to a zero, defaults to the number of logical CPUs as returned
by the JVM. If set to a negative value, defaults to a direct executor.
@@ -3153,11 +3199,6 @@
limit will truncate the list (but will still set `_more_changes` on
result lists). Set to 0 for no limit.
+
-When `index.type` is set to `ELASTICSEARCH`, this value should not exceed
-the `index.max_result_window` value configured on the Elasticsearch
-server. If a value is not configured during site initialization, defaults to
-10000, which is the default value of `index.max_result_window` in Elasticsearch.
-+
When `index.type` is set to `LUCENE`, defaults to no limit.
[[index.maxPages]]index.maxPages::
@@ -3226,7 +3267,7 @@
+
Whether the scheduled indexer is enabled. If the scheduled indexer is
disabled you must implement other means to keep the group index for the
-replica up-to-date (e.g. by using ElasticSearch for the indexes).
+replica up-to-date.
+
Defaults to `true`.
@@ -3333,6 +3374,11 @@
+
Defaults to true (throttling enabled).
+During offline reindexing, setting ramBufferSize greater than the size
+of index (size of specific index folder under <site_dir>/index) and
+maxBufferedDocs as -1 avoids unnecessary flushes and triggers only a
+single flush at the end of the process.
+
Sample Lucene index configuration:
----
[index]
@@ -3354,95 +3400,6 @@
----
-[[elasticsearch]]
-=== Section elasticsearch
-
-WARNING: Support for Elasticsearch is still experimental and is not recommended
-for production use. For compatibility information, please refer to the
-link:https://www.gerritcodereview.com/elasticsearch.html[project homepage,role=external,window=_blank].
-
-Note that when Gerrit is configured to use Elasticsearch, the Elasticsearch
-server(s) must be reachable during the site initialization.
-
-[[elasticsearch.prefix]]elasticsearch.prefix::
-+
-This setting can be used to prefix index names to allow multiple Gerrit
-instances in a single Elasticsearch cluster. Prefix `gerrit1_` would result in a
-change index named `gerrit1_changes_0001`.
-+
-Not set by default.
-
-[[elasticsearch.server]]elasticsearch.server::
-+
-Elasticsearch server URI in the form `http[s]://hostname:port`. The `port` is
-optional and defaults to `9200` if not specified.
-+
-At least one server must be specified. May be specified multiple times to
-configure multiple Elasticsearch servers.
-+
-Note that the site initialization program only allows to configure a single
-server. To configure multiple servers the `gerrit.config` file must be edited
-manually.
-
-[[elasticsearch.numberOfShards]]elasticsearch.numberOfShards::
-+
-Sets the number of shards to use per index. Refer to the
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#_static_index_settings[
-Elasticsearch documentation,role=external,window=_blank] for details.
-+
-Defaults to 1.
-
-[[elasticsearch.numberOfReplicas]]elasticsearch.numberOfReplicas::
-+
-Sets the number of replicas to use per index. Refer to the
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#dynamic-index-settings[
-Elasticsearch documentation,role=external,window=_blank] for details.
-+
-Defaults to 1.
-
-[[elasticsearch.maxResultWindow]]elasticsearch.maxResultWindow::
-+
-Sets the maximum value of `from + size` for searches to use per index. Refer to the
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#dynamic-index-settings[
-Elasticsearch documentation,role=external,window=_blank] for details.
-+
-Defaults to 10000.
-
-[[elasticsearch.connectTimeout]]elasticsearch.connectTimeout::
-+
-Sets the timeout for connecting to elasticsearch.
-+
-Defaults to `1 second`.
-
-[[elasticsearch.socketTimeout]]elasticsearch.socketTimeout::
-+
-Sets the timeout for the underlying connection. For more information, refer to
-link:#httpd.idleTimeout[`httpd.idleTimeout`].
-+
-Defaults to `30 seconds`.
-
-==== Elasticsearch Security
-
-When security is enabled in Elasticsearch, the username and password must be provided.
-Note that the same username and password are used for all servers.
-
-For further information about Elasticsearch security, please refer to
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/security-getting-started.html[the documentation,role=external,window=_blank].
-This is the current documentation link. Select another Elasticsearch version
-from the dropdown menu available on that page if need be.
-
-[[elasticsearch.username]]elasticsearch.username::
-+
-Username used to connect to Elasticsearch.
-+
-If a password is set, defaults to `elastic`, otherwise not set by default.
-
-[[elasticsearch.password]]elasticsearch.password::
-+
-Password used to connect to Elasticsearch.
-+
-Not set by default.
-
[[event]]
=== Section event
diff --git a/Documentation/dev-bazel.txt b/Documentation/dev-bazel.txt
index 4069446..9792d8e 100644
--- a/Documentation/dev-bazel.txt
+++ b/Documentation/dev-bazel.txt
@@ -268,7 +268,15 @@
bazel-bin/Documentation/searchfree.zip
----
-To build the executable WAR with the documentation included:
+To generate HTML files skipping the zip archiving:
+
+----
+ bazel build Documentation
+----
+
+And open `bazel-bin/Documentation/index.html`.
+
+To build the Gerrit executable WAR with the documentation included:
----
bazel build withdocs
@@ -329,12 +337,6 @@
bazel test --test_tag_filters=-flaky //...
----
-To exclude tests that require a Docker host:
-
-----
- bazel test --test_tag_filters=-docker //...
-----
-
To exclude tests that require very recent git client version:
----
@@ -358,15 +360,11 @@
bazel test --test_env=GERRIT_INDEX_TYPE=LUCENE //...
----
-Elastic search is not currently supported in integration tests.
-
The following values are currently supported for the group name:
* annotation
* api
-* docker
* edit
-* elastic
* git
* git-protocol-v2
* git-upload-archive
@@ -399,23 +397,6 @@
Now attach with a debugger to the port `5005`. For example use "Remote Java Application" launch
configuration in Eclipe and specify the port `5005`.
-[[elasticsearch]]
-=== Elasticsearch
-
-Successfully running the Elasticsearch tests requires Docker, and
-may require setting the local virtual memory on
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html[linux,role=external,window=_blank] and
-link:https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_set_vm_max_map_count_to_at_least_262144[macOS,role=external,window=_blank].
-
-On macOS, if using link:https://docs.docker.com/docker-for-mac/[Docker Desktop,role=external,window=_blank],
-the effective memory value can be set in the Preferences, under the Advanced tab.
-The default value usually does not suffice and is causing premature container exits.
-That default is currently 2 GB and should be set to at least 5 (GB).
-
-If Docker is not available, the Elasticsearch tests will be skipped.
-Note that Bazel currently does not show
-link:https://github.com/bazelbuild/bazel/issues/3476[the skipped tests,role=external,window=_blank].
-
[[logging]]
=== Controlling logging level
diff --git a/Documentation/dev-plugins.txt b/Documentation/dev-plugins.txt
index e367b07..35909c7 100644
--- a/Documentation/dev-plugins.txt
+++ b/Documentation/dev-plugins.txt
@@ -838,6 +838,63 @@
}
----
+Plugins can receive a bean object for each of the gerrit ssh and the REST API
+commands by implementing BeanParseListener interface and registering it to a
+command class name in the plugin module's `configure()` method. The below
+example shows a plugin that always limits the number of projects returned
+by the ls-projects SSH command.
+
+[source, java]
+----
+protected static class PluginModule extends AbstractModule {
+ @Override
+ public void configure() {
+ bind(DynamicOptions.DynamicBean.class)
+ .annotatedWith(Exports.named(ListProjectsCommand.class))
+ .to(ListProjectsCommandBeanListener.class);
+ }
+
+ protected static class ListProjectsCommandBeanListener
+ implements DynamicOptions.BeanParseListener {
+ @Override
+ public void onBeanParseStart(String plugin, Object bean) {
+ ListProjectsCommand command = (ListProjectsCommand) bean;
+ command.impl.setLimit(1);
+ }
+
+ @Override
+ public void onBeanParseEnd(String plugin, Object bean) {}
+ }
+}
+----
+
+The below example shows a plugin that always limits the number of projects
+returned by the /projects/ REST API.
+
+[source, java]
+----
+protected static class PluginModule extends AbstractModule {
+ @Override
+ public void configure() {
+ bind(DynamicOptions.DynamicBean.class)
+ .annotatedWith(Exports.named(ListProjects.class))
+ .to(ListProjectsBeanListener.class);
+ }
+
+ protected static class ListProjectsBeanListener
+ implements DynamicOptions.BeanParseListener {
+ @Override
+ public void onBeanParseStart(String plugin, Object bean) {
+ ListProjects listProjects = (ListProjects) bean;
+ listProjects.setLimit(1);
+ }
+
+ @Override
+ public void onBeanParseEnd(String plugin, Object bean) {}
+ }
+}
+----
+
=== Calling Command Options ===
Within an OptionHandler, during the processing of an option, plugins can
@@ -899,7 +956,7 @@
[[query_attributes]]
== Change Attributes
-==== ChangePluginDefinedInfoFactory
+=== ChangePluginDefinedInfoFactory
Plugins can provide additional attributes to be returned from the Get Change and
Query Change APIs by implementing the `ChangePluginDefinedInfoFactory` interface
@@ -2209,8 +2266,7 @@
DropWizard Metrics,role=external,window=_blank].
Metric reporting plugin implementations are provided for
-link:https://gerrit.googlesource.com/plugins/metrics-reporter-jmx/[JMX,role=external,window=_blank],
-link:https://gerrit.googlesource.com/plugins/metrics-reporter-elasticsearch/[Elastic Search,role=external,window=_blank],
+link:https://gerrit.googlesource.com/plugins/metrics-reporter-jmx/[JMX,role=external,window=_blank]
and link:https://gerrit.googlesource.com/plugins/metrics-reporter-graphite/[Graphite,role=external,window=_blank].
There is also a working example of reporting metrics to the console in the
diff --git a/Documentation/externalid-case-insensitivity.txt b/Documentation/externalid-case-insensitivity.txt
new file mode 100644
index 0000000..b4e8140
--- /dev/null
+++ b/Documentation/externalid-case-insensitivity.txt
@@ -0,0 +1,128 @@
+:linkattrs:
+= Gerrit Code Review - ExternalId case insensitivity
+
+Gerrit usernames are case insensitive by default: e.g. johndoe and JohnDoe
+represents the same account. However, for installations older than v3.5.x,
+the usernames were case sensitive, e.g. johndoe and JohnDoe can both exist
+as separate accounts. This could lead to issues when migrating an account
+from LDAP to an internal account, if ldap.localUsernameToLowerCase was set.
+Such usernames can also be rather confusing for users, if they try to identify
+authors of comments or changes.
+
+When Gerrit handles case insensitive usernames (external IDs using the
+`gerrit:` or `username:` scheme, their external IDs SHA-1 is always computed
+using the lowercase external ID, hence there cannot be any account differing
+only in the capitalization of their usernames.
+
+Gerrit installations older than v3.5.x that are switching to the case-insensitive
+username need to migrating all their existing accounts SHA-1s.
+
+[[migration]]
+== Migration
+
+Migrating external ID notes can take several minutes for large sites(for example
+migration ~45000 accounts can take up to five minutes), so administrators choose
+whether to do the migration offline or online, depending on their available
+resources and tolerance for downtime.
+
+NOTE: Migration is required only on Gerrit primary instances.
+
+[[offline-migration]]
+=== Offline
+
+To run the offline migration execute following steps:
+* Stop all Gerrit primary instances
+* Set the `auth.userNameCaseInsensitive` to false
+----
+[auth]
+ userNameCaseInsensitive = false
+----
+
+* Run:
+[verse]
+--
+_java_ -jar gerrit.war _ChangeExternalIdCaseSensitivity_
+ -d <SITE_PATH>
+ [--batch]
+--
+
+See: link:pgm-ChangeExternalIdCaseSensitivity.html
+
+* During the migration `auth.userNameCaseInsensitive` will be set to true
+on a node which is executing the migration. When the migration is finished,
+on all other primary nodes set `auth.userNameCaseInsensitive` to true
+* Start all Gerrit primary instances
+
+[[online-migration]]
+=== Online
+
+To start the online migration, set the `auth.userNameCaseInsensitive` and
+`auth.userNameCaseInsensitiveMigrationMode` options in `gerrit.config` and
+restart Gerrit:
+----
+[auth]
+ userNameCaseInsensitive = true
+ userNameCaseInsensitiveMigrationMode = true
+----
+* Trigger online migration:
+----
+$ ssh -p <port> <host> gerrit migrate-externalids-to-insensitive
+----
+
+See: link:cmd-migrate-externalids-to-insensitive.html
+
+[online-ha-migration]
+== Online migration for high-availability setup
+
+To start the online migration with a setup containing multiple primary
+instances execute following steps:
+* On all Gerrit primary instances set `auth.userNameCaseInsensitive` and
+`auth.userNameCaseInsensitiveMigrationMode` and perform a rolling restart
+----
+[auth]
+ userNameCaseInsensitive = true
+ userNameCaseInsensitiveMigrationMode = true
+----
+* Trigger online migration:
+----
+$ ssh -p <port> <host> gerrit migrate-externalids-to-insensitive
+----
+
+See: link:cmd-migrate-externalids-to-insensitive.html
+
+* When the migration is finished, on all other primary nodes set
+`auth.userNameCaseInsensitiveMigrationMode` to false and perform a
+rolling restart
+----
+[auth]
+ userNameCaseInsensitive = true
+ userNameCaseInsensitiveMigrationMode = false
+----
+
+== External ID case insensitivity rollback
+
+The offline migration tool allows to calculate external ID notes named with the SHA-1
+from the case sensitive external ID.
+
+To rollback external ID notes migration execute following steps:
+* Stop all Gerrit primary instances
+* Set the `auth.userNameCaseInsensitive` to true
+----
+[auth]
+ userNameCaseInsensitive = true
+----
+
+* Run:
+[verse]
+--
+_java_ -jar gerrit.war _ChangeExternalIdCaseSensitivity_
+ -d <SITE_PATH>
+ [--batch]
+--
+
+See: link:pgm-ChangeExternalIdCaseSensitivity.html
+
+* During the migration `auth.userNameCaseInsensitive` will be set to false
+on a node which is executing the migration. When the migration is finished,
+on all other primary nodes set `auth.userNameCaseInsensitive` to false
+* Start all Gerrit primary instances
diff --git a/Documentation/intro-user.txt b/Documentation/intro-user.txt
index 11b6807..8fc3852 100644
--- a/Documentation/intro-user.txt
+++ b/Documentation/intro-user.txt
@@ -459,7 +459,7 @@
push permission] on the destination branch.
The move operation will not update the change's parent and users will have
-to link:#rebase[rebase] the change. Also, merge commits cannot be moved.
+to link:#rebase[rebase] the change.
[[abandon]]
[[restore]]
diff --git a/Documentation/licenses.txt b/Documentation/licenses.txt
index 1302329..490e141 100644
--- a/Documentation/licenses.txt
+++ b/Documentation/licenses.txt
@@ -62,11 +62,8 @@
* guice:guice-library
* guice:guice-servlet
* guice:javax_inject
-* httpcomponents:httpasyncclient
* httpcomponents:httpclient
* httpcomponents:httpcore
-* httpcomponents:httpcore-nio
-* jackson:jackson-core
* jetty:http
* jetty:io
* jetty:jmx
@@ -1114,22 +1111,6 @@
----
-[[elasticsearch]]
-elasticsearch
-
-* elasticsearch-rest-client:elasticsearch-rest-client
-
-[[elasticsearch_license]]
-----
-Elasticsearch
-Copyright 2009-2015 Elasticsearch
-
-This product includes software developed by The Apache Software
-Foundation (http://www.apache.org/).
-
-----
-
-
[[flexmark]]
flexmark
diff --git a/Documentation/logs.txt b/Documentation/logs.txt
index ba370b1..e120fdd 100644
--- a/Documentation/logs.txt
+++ b/Documentation/logs.txt
@@ -56,6 +56,11 @@
CPU time in kernel mode is `total_cpu - user_cpu`.
* `memory`: memory allocated in bytes to execute command. -1 if the JVM does
not support this metric.
+* `command status`: the overall result of the git command over HTTP. Currently
+ populated only for the transfer phase of `git-upload-pack` commands.
+ Possible values:
+** `-1`: The `git-upload-pack` transfer was ultimately not successful
+** `0`: The `git-upload-pack` transfer was ultimately successful
Example:
```
diff --git a/Documentation/pgm-reindex.txt b/Documentation/pgm-reindex.txt
index 5167277..0653d8d 100644
--- a/Documentation/pgm-reindex.txt
+++ b/Documentation/pgm-reindex.txt
@@ -20,7 +20,8 @@
== OPTIONS
--threads::
- Number of threads to use for indexing.
+ Number of threads to use for indexing. Default is
+ link:config-gerrit.html#index.batchThreads[index.batchThreads]
--changes-schema-version::
Schema version to reindex; default is most recent version.
@@ -35,6 +36,10 @@
Reindex only index with given name. This option can be supplied
more than once to reindex multiple indices.
+--disable-cache-stats::
+ Disables printing cache statistics at the end of program to reduce
+ noise. Defaulted when reindex is run from init on a new site.
+
== CONTEXT
The secondary index must be enabled. See
link:config-gerrit.html#index.type[index.type].
diff --git a/Documentation/rest-api-projects.txt b/Documentation/rest-api-projects.txt
index 5fa2b2e..e52a87d 100644
--- a/Documentation/rest-api-projects.txt
+++ b/Documentation/rest-api-projects.txt
@@ -421,11 +421,11 @@
----
The `/projects/` URL also accepts a start integer in the `start`
-parameter. The results will skip `start` groups from project list.
+parameter. The results will skip `start` projects from project list.
Query 25 projects starting from index 50.
----
- GET /groups/?query=<query>&limit=25&start=50 HTTP/1.0
+ GET /projects/?query=<query>&limit=25&start=50 HTTP/1.0
----
[[project-query-options]]
@@ -1541,6 +1541,47 @@
}
----
+[[commits-included-in]]
+=== Get Commits Included In Refs
+--
+'GET /projects/link:#project-name[\{project-name\}]/commits:in'
+--
+
+Gets refs in which the specified commits were merged into. Returns a map of commits
+to sets of full ref names.
+
+One or more `commit` query parameters are required and each specifies a
+commit-id (SHA-1 in 40 digit hex representation). Commits will not be contained
+in the map if they do not exist or are not reachable from visible, specified refs.
+
+One or more `ref` query parameters are required and each specifies a full ref name.
+Refs which are not visible to the calling user according to the project's read
+permissions and refs which do not exist will be filtered out from the result.
+
+.Request
+----
+ GET /projects/work%2Fmy-project/commits:in?commit=a8a477efffbbf3b44169bb9a1d3a334cbbd9aa96&commit=6d2a3adb10e844c33617fc948dbeb88e868d396e&ref=refs/heads/master&ref=refs/heads/branch1 HTTP/1.0
+----
+
+.Response
+----
+ HTTP/1.1 200 OK
+ Content-Disposition: attachment
+ Content-Type: application/json;charset=UTF-8
+
+ )]}'
+ {
+ "a8a477efffbbf3b44169bb9a1d3a334cbbd9aa96": [
+ "refs/heads/master"
+ ],
+ "6d2a3adb10e844c33617fc948dbeb88e868d396e": [
+ "refs/heads/branch1",
+ "refs/heads/master"
+ ]
+ }
+
+----
+
[[branch-endpoints]]
== Branch Endpoints
diff --git a/Documentation/user-dashboards.txt b/Documentation/user-dashboards.txt
index e64d625..364b0d9 100644
--- a/Documentation/user-dashboards.txt
+++ b/Documentation/user-dashboards.txt
@@ -12,7 +12,7 @@
Dashboards are available via URLs like:
----
- /#/dashboard/?title=Custom+View&To+Review=reviewer:john.doe@example.com&Pending+In+myproject=project:myproject+is:open
+ /dashboard/?title=Custom+View&To+Review=reviewer:john.doe@example.com&Pending+In+myproject=project:myproject+is:open
----
This opens a view showing the title "Custom View" with two sections,
"To Review" and "Pending in myproject":
@@ -51,7 +51,7 @@
to changes for the current user:
----
- /#/dashboard/?title=Mine&foreach=owner:self&My+Pending=is:open&My+Merged=is:merged
+ /dashboard/?title=Mine&foreach=owner:self&My+Pending=is:open&My+Merged=is:merged
----
diff --git a/WORKSPACE b/WORKSPACE
index 29a35e8..7ff4c0d 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -30,26 +30,17 @@
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
load("//tools/bzl:maven_jar.bzl", "GERRIT", "MAVEN_LOCAL", "maven_jar")
load("//plugins:external_plugin_deps.bzl", "external_plugin_deps")
-load("//tools:nongoogle.bzl", "TESTCONTAINERS_VERSION", "declare_nongoogle_deps")
+load("//tools:nongoogle.bzl", "declare_nongoogle_deps")
load("//tools:deps.bzl", "CAFFEINE_VERS", "java_dependencies")
http_archive(
- name = "bazel_toolchains",
- sha256 = "1adf7a8e9901287c644dcf9ca08dd8d67a69df94bedbd57a841490a84dc1e9ed",
- strip_prefix = "bazel-toolchains-5.0.0",
- urls = [
- "https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/v5.0.0.tar.gz",
- "https://github.com/bazelbuild/bazel-toolchains/archive/v5.0.0.tar.gz",
- ],
-)
-
-load("@bazel_toolchains//rules:rbe_repo.bzl", "rbe_autoconfig")
-
-# Creates a default toolchain config for RBE.
-rbe_autoconfig(
name = "rbe_jdk11",
- java_home = "/usr/lib/jvm/11.29.3-ca-jdk11.0.2/reduced",
- use_checked_in_confs = "Force",
+ sha256 = "766796de71916118e528b9f4334c29c9c9b4e926227bf3264dee555e6a4306c8",
+ strip_prefix = "rbe_autoconfig-2.0.0",
+ urls = [
+ "https://gerrit-bazel.storage.googleapis.com/rbe_autoconfig/v2.0.0.tar.gz",
+ "https://github.com/davido/rbe_autoconfig/archive/v2.0.0.tar.gz",
+ ],
)
http_archive(
diff --git a/java/com/google/gerrit/acceptance/AbstractDaemonTest.java b/java/com/google/gerrit/acceptance/AbstractDaemonTest.java
index 0e0a849..b5835c9 100644
--- a/java/com/google/gerrit/acceptance/AbstractDaemonTest.java
+++ b/java/com/google/gerrit/acceptance/AbstractDaemonTest.java
@@ -1167,7 +1167,7 @@
ProjectConfig config = projectConfigFactory.read(md);
config.updateProject(p -> p.setBooleanConfig(BooleanProjectConfig.USE_SIGNED_OFF_BY, value));
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
}
@@ -1176,7 +1176,7 @@
ProjectConfig config = projectConfigFactory.read(md);
config.updateProject(p -> p.setBooleanConfig(BooleanProjectConfig.REQUIRE_CHANGE_ID, value));
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
}
@@ -1718,7 +1718,7 @@
projectConfig.commit(metaDataUpdate);
metaDataUpdate.close();
metaDataUpdate = null;
- projectCache.evict(projectConfig.getProject());
+ projectCache.evictAndReindex(projectConfig.getProject());
}
@Override
diff --git a/java/com/google/gerrit/acceptance/BUILD b/java/com/google/gerrit/acceptance/BUILD
index cb5f667..37b4780 100644
--- a/java/com/google/gerrit/acceptance/BUILD
+++ b/java/com/google/gerrit/acceptance/BUILD
@@ -79,7 +79,6 @@
PGM_DEPLOY_ENV = [
"//lib:caffeine",
"//lib:caffeine-guava",
- "//lib/jackson:jackson-core",
"//lib/prolog:cafeteria",
]
diff --git a/java/com/google/gerrit/acceptance/DisabledAccountIndex.java b/java/com/google/gerrit/acceptance/DisabledAccountIndex.java
index 271d15c..7bd0c73 100644
--- a/java/com/google/gerrit/acceptance/DisabledAccountIndex.java
+++ b/java/com/google/gerrit/acceptance/DisabledAccountIndex.java
@@ -45,6 +45,11 @@
}
@Override
+ public void insert(AccountState obj) {
+ throw new UnsupportedOperationException("AccountIndex is disabled");
+ }
+
+ @Override
public void replace(AccountState obj) {
throw new UnsupportedOperationException("AccountIndex is disabled");
}
diff --git a/java/com/google/gerrit/acceptance/DisabledChangeIndex.java b/java/com/google/gerrit/acceptance/DisabledChangeIndex.java
index 34f72f5c..7671ad4 100644
--- a/java/com/google/gerrit/acceptance/DisabledChangeIndex.java
+++ b/java/com/google/gerrit/acceptance/DisabledChangeIndex.java
@@ -52,6 +52,11 @@
}
@Override
+ public void insert(ChangeData obj) {
+ throw new UnsupportedOperationException("ChangeIndex is disabled");
+ }
+
+ @Override
public void replace(ChangeData obj) {
throw new UnsupportedOperationException("ChangeIndex is disabled");
}
diff --git a/java/com/google/gerrit/acceptance/DisabledProjectIndex.java b/java/com/google/gerrit/acceptance/DisabledProjectIndex.java
index ed119ff..2e3dd90 100644
--- a/java/com/google/gerrit/acceptance/DisabledProjectIndex.java
+++ b/java/com/google/gerrit/acceptance/DisabledProjectIndex.java
@@ -50,6 +50,11 @@
}
@Override
+ public void insert(ProjectData obj) {
+ throw new UnsupportedOperationException("ProjectIndex is disabled");
+ }
+
+ @Override
public void replace(ProjectData obj) {
throw new UnsupportedOperationException("ProjectIndex is disabled");
}
diff --git a/java/com/google/gerrit/acceptance/GerritServer.java b/java/com/google/gerrit/acceptance/GerritServer.java
index 402d21d..a149f29 100644
--- a/java/com/google/gerrit/acceptance/GerritServer.java
+++ b/java/com/google/gerrit/acceptance/GerritServer.java
@@ -61,6 +61,7 @@
import com.google.gerrit.server.experiments.ConfigExperimentFeatures.ConfigExperimentFeaturesModule;
import com.google.gerrit.server.git.receive.AsyncReceiveCommits.AsyncReceiveCommitsModule;
import com.google.gerrit.server.git.validators.CommitValidationListener;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.schema.JdbcAccountPatchReviewStore;
import com.google.gerrit.server.ssh.NoSshModule;
import com.google.gerrit.server.util.ReplicaUtil;
@@ -492,11 +493,11 @@
}
if (indexType.isLucene()) {
daemon.setIndexModule(
- LuceneIndexModule.singleVersionAllLatest(0, ReplicaUtil.isReplica(baseConfig)));
+ LuceneIndexModule.singleVersionAllLatest(
+ 0, ReplicaUtil.isReplica(baseConfig), AutoFlush.ENABLED));
} else {
daemon.setIndexModule(FakeIndexModule.latestVersion(false));
}
- // Elastic search is not supported in integration tests yet.
daemon.setEnableHttpd(desc.httpd());
daemon.setInMemory(true);
diff --git a/java/com/google/gerrit/acceptance/ProjectResetter.java b/java/com/google/gerrit/acceptance/ProjectResetter.java
index d885303..46f7496 100644
--- a/java/com/google/gerrit/acceptance/ProjectResetter.java
+++ b/java/com/google/gerrit/acceptance/ProjectResetter.java
@@ -307,7 +307,7 @@
Sets.union(
projectsWithConfigChanges(restoredRefsByProject),
projectsWithConfigChanges(deletedRefsByProject))) {
- projectCache.evict(project);
+ projectCache.evictAndReindex(project);
}
}
diff --git a/java/com/google/gerrit/acceptance/ReadOnlyChangeIndex.java b/java/com/google/gerrit/acceptance/ReadOnlyChangeIndex.java
index e943519..f7a0669 100644
--- a/java/com/google/gerrit/acceptance/ReadOnlyChangeIndex.java
+++ b/java/com/google/gerrit/acceptance/ReadOnlyChangeIndex.java
@@ -45,6 +45,11 @@
}
@Override
+ public void insert(ChangeData obj) {
+ // do nothing
+ }
+
+ @Override
public void replace(ChangeData obj) {
// do nothing
}
diff --git a/java/com/google/gerrit/acceptance/testsuite/project/ProjectOperationsImpl.java b/java/com/google/gerrit/acceptance/testsuite/project/ProjectOperationsImpl.java
index 16dca66..18da4b3 100644
--- a/java/com/google/gerrit/acceptance/testsuite/project/ProjectOperationsImpl.java
+++ b/java/com/google/gerrit/acceptance/testsuite/project/ProjectOperationsImpl.java
@@ -147,7 +147,7 @@
setExclusiveGroupPermissions(projectConfig, projectUpdate.exclusiveGroupPermissions());
projectConfig.commit(metaDataUpdate);
}
- projectCache.evict(nameKey);
+ projectCache.evictAndReindex(nameKey);
}
private void removePermissions(
@@ -292,7 +292,7 @@
setConfig(projectConfig);
try {
- projectCache.evict(nameKey);
+ projectCache.evictAndReindex(nameKey);
} catch (Exception e) {
// Evicting the project from the cache, also triggers a reindex of the project.
// The reindex step fails if the project config is invalid. That's fine, since it was our
@@ -310,7 +310,7 @@
testProjectInvalidation.projectConfigUpdater().forEach(c -> c.accept(projectConfig));
setConfig(projectConfig);
try {
- projectCache.evict(nameKey);
+ projectCache.evictAndReindex(nameKey);
} catch (Exception e) {
// Evicting the project from the cache, also triggers a reindex of the project.
// The reindex step fails if the project config is invalid. That's fine, since it was our
diff --git a/java/com/google/gerrit/elasticsearch/AbstractElasticIndex.java b/java/com/google/gerrit/elasticsearch/AbstractElasticIndex.java
deleted file mode 100644
index 562464d..0000000
--- a/java/com/google/gerrit/elasticsearch/AbstractElasticIndex.java
+++ /dev/null
@@ -1,408 +0,0 @@
-// Copyright (C) 2014 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static com.google.common.base.Preconditions.checkArgument;
-import static com.google.common.collect.ImmutableList.toImmutableList;
-import static com.google.gson.FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES;
-import static java.nio.charset.StandardCharsets.UTF_8;
-
-import com.google.common.collect.ArrayListMultimap;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ListMultimap;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Streams;
-import com.google.common.flogger.FluentLogger;
-import com.google.common.io.BaseEncoding;
-import com.google.common.io.CharStreams;
-import com.google.gerrit.common.Nullable;
-import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
-import com.google.gerrit.elasticsearch.builders.QueryBuilder;
-import com.google.gerrit.elasticsearch.builders.SearchSourceBuilder;
-import com.google.gerrit.elasticsearch.bulk.DeleteRequest;
-import com.google.gerrit.entities.converter.ProtoConverter;
-import com.google.gerrit.exceptions.StorageException;
-import com.google.gerrit.index.FieldDef;
-import com.google.gerrit.index.FieldType;
-import com.google.gerrit.index.Index;
-import com.google.gerrit.index.QueryOptions;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.query.DataSource;
-import com.google.gerrit.index.query.FieldBundle;
-import com.google.gerrit.index.query.ListResultSet;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.index.query.ResultSet;
-import com.google.gerrit.proto.Protos;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.IndexUtils;
-import com.google.gson.Gson;
-import com.google.gson.GsonBuilder;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.gson.JsonParser;
-import com.google.protobuf.MessageLite;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.InputStreamReader;
-import java.io.Reader;
-import java.io.UnsupportedEncodingException;
-import java.net.URLEncoder;
-import java.sql.Timestamp;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.function.Function;
-import org.apache.http.HttpEntity;
-import org.apache.http.HttpStatus;
-import org.apache.http.StatusLine;
-import org.apache.http.client.methods.HttpPost;
-import org.apache.http.entity.ContentType;
-import org.apache.http.nio.entity.NStringEntity;
-import org.elasticsearch.client.Request;
-import org.elasticsearch.client.Response;
-
-abstract class AbstractElasticIndex<K, V> implements Index<K, V> {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- protected static final String BULK = "_bulk";
- protected static final String MAPPINGS = "mappings";
- protected static final String ORDER = "order";
- protected static final String DESC_SORT_ORDER = "desc";
- protected static final String ASC_SORT_ORDER = "asc";
- protected static final String UNMAPPED_TYPE = "unmapped_type";
- protected static final String SEARCH = "_search";
- protected static final String SETTINGS = "settings";
-
- static byte[] decodeBase64(String base64String) {
- return BaseEncoding.base64().decode(base64String);
- }
-
- protected static <T> List<T> decodeProtos(
- JsonObject doc, String fieldName, ProtoConverter<?, T> converter) {
- JsonArray field = doc.getAsJsonArray(fieldName);
- if (field == null) {
- return null;
- }
- return Streams.stream(field)
- .map(JsonElement::getAsString)
- .map(AbstractElasticIndex::decodeBase64)
- .map(bytes -> parseProtoFrom(bytes, converter))
- .collect(toImmutableList());
- }
-
- protected static <P extends MessageLite, T> T parseProtoFrom(
- byte[] bytes, ProtoConverter<P, T> converter) {
- P message = Protos.parseUnchecked(converter.getParser(), bytes);
- return converter.fromProto(message);
- }
-
- static String getContent(Response response) throws IOException {
- HttpEntity responseEntity = response.getEntity();
- String content = "";
- if (responseEntity != null) {
- InputStream contentStream = responseEntity.getContent();
- try (Reader reader = new InputStreamReader(contentStream, UTF_8)) {
- content = CharStreams.toString(reader);
- }
- }
- return content;
- }
-
- private final ElasticConfiguration config;
- private final Schema<V> schema;
- private final SitePaths sitePaths;
- private final String indexNameRaw;
-
- protected final ElasticRestClientProvider client;
- protected final String indexName;
- protected final Gson gson;
- protected final ElasticQueryBuilder queryBuilder;
-
- AbstractElasticIndex(
- ElasticConfiguration config,
- SitePaths sitePaths,
- Schema<V> schema,
- ElasticRestClientProvider client,
- String indexName) {
- this.config = config;
- this.sitePaths = sitePaths;
- this.schema = schema;
- this.gson = new GsonBuilder().setFieldNamingPolicy(LOWER_CASE_WITH_UNDERSCORES).create();
- this.queryBuilder = new ElasticQueryBuilder();
- this.indexName = config.getIndexName(indexName, schema.getVersion());
- this.indexNameRaw = indexName;
- this.client = client;
- }
-
- @Override
- public Schema<V> getSchema() {
- return schema;
- }
-
- @Override
- public void close() {
- // Do nothing. Client is closed by the provider.
- }
-
- @Override
- public void markReady(boolean ready) {
- IndexUtils.setReady(sitePaths, indexNameRaw, schema.getVersion(), ready);
- }
-
- @Override
- public void delete(K id) {
- String uri = getURI(BULK);
- Response response = postRequest(uri, getDeleteActions(id), getRefreshParam());
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format("Failed to delete %s from index %s: %s", id, indexName, statusCode));
- }
- }
-
- @Override
- public void deleteAll() {
- // Delete the index, if it exists.
- String endpoint = indexName + client.adapter().indicesExistParams();
- Response response = performRequest("HEAD", endpoint);
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode == HttpStatus.SC_OK) {
- response = performRequest("DELETE", indexName);
- statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format("Failed to delete index %s: %s", indexName, statusCode));
- }
- }
-
- // Recreate the index.
- String indexCreationFields = concatJsonString(getSettings(), getMappings());
- response = performRequest("PUT", indexName, indexCreationFields);
- statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- String error = String.format("Failed to create index %s: %s", indexName, statusCode);
- throw new StorageException(error);
- }
- }
-
- protected abstract String getDeleteActions(K id);
-
- protected abstract String getMappings();
-
- private String getSettings() {
- return gson.toJson(ImmutableMap.of(SETTINGS, ElasticSetting.createSetting(config)));
- }
-
- protected abstract String getId(V v);
-
- protected String getMappingsForSingleType(MappingProperties properties) {
- return getMappingsFor(properties);
- }
-
- protected String getMappingsFor(MappingProperties properties) {
- JsonObject mappings = new JsonObject();
-
- mappings.add(MAPPINGS, gson.toJsonTree(properties));
- return gson.toJson(mappings);
- }
-
- protected String getDeleteRequest(K id) {
- return new DeleteRequest(id.toString(), indexName).toString();
- }
-
- protected abstract V fromDocument(JsonObject doc, Set<String> fields);
-
- protected FieldBundle toFieldBundle(JsonObject doc) {
- Map<String, FieldDef<V, ?>> allFields = getSchema().getFields();
- ListMultimap<String, Object> rawFields = ArrayListMultimap.create();
- for (Map.Entry<String, JsonElement> element :
- doc.get(client.adapter().rawFieldsKey()).getAsJsonObject().entrySet()) {
- checkArgument(
- allFields.containsKey(element.getKey()), "Unrecognized field " + element.getKey());
- FieldType<?> type = allFields.get(element.getKey()).getType();
- Iterable<JsonElement> innerItems =
- element.getValue().isJsonArray()
- ? element.getValue().getAsJsonArray()
- : Collections.singleton(element.getValue());
- for (JsonElement inner : innerItems) {
- if (type == FieldType.EXACT || type == FieldType.FULL_TEXT || type == FieldType.PREFIX) {
- rawFields.put(element.getKey(), inner.getAsString());
- } else if (type == FieldType.INTEGER || type == FieldType.INTEGER_RANGE) {
- rawFields.put(element.getKey(), inner.getAsInt());
- } else if (type == FieldType.LONG) {
- rawFields.put(element.getKey(), inner.getAsLong());
- } else if (type == FieldType.TIMESTAMP) {
- rawFields.put(element.getKey(), new Timestamp(inner.getAsLong()));
- } else if (type == FieldType.STORED_ONLY) {
- rawFields.put(element.getKey(), decodeBase64(inner.getAsString()));
- } else {
- throw FieldType.badFieldType(type);
- }
- }
- }
- return new FieldBundle(rawFields);
- }
-
- protected String toAction(String type, String id, String action) {
- JsonObject properties = new JsonObject();
- properties.addProperty("_id", id);
- properties.addProperty("_index", indexName);
- properties.addProperty("_type", type);
-
- JsonObject jsonAction = new JsonObject();
- jsonAction.add(action, properties);
- return jsonAction.toString() + System.lineSeparator();
- }
-
- protected void addNamedElement(String name, JsonObject element, JsonArray array) {
- JsonObject arrayElement = new JsonObject();
- arrayElement.add(name, element);
- array.add(arrayElement);
- }
-
- protected Map<String, String> getRefreshParam() {
- Map<String, String> params = new HashMap<>();
- params.put("refresh", "true");
- return params;
- }
-
- protected String getSearch(SearchSourceBuilder searchSource, JsonArray sortArray) {
- JsonObject search = new JsonParser().parse(searchSource.toString()).getAsJsonObject();
- search.add("sort", sortArray);
- return gson.toJson(search);
- }
-
- protected JsonArray getSortArray(String idFieldName) {
- JsonObject properties = new JsonObject();
- properties.addProperty(ORDER, ASC_SORT_ORDER);
-
- JsonArray sortArray = new JsonArray();
- addNamedElement(idFieldName, properties, sortArray);
- return sortArray;
- }
-
- protected String getURI(String request) {
- try {
- return URLEncoder.encode(indexName, UTF_8.toString()) + "/" + request;
- } catch (UnsupportedEncodingException e) {
- throw new StorageException(e);
- }
- }
-
- protected Response postRequest(String uri, Object payload) {
- return performRequest("POST", uri, payload);
- }
-
- protected Response postRequest(String uri, Object payload, Map<String, String> params) {
- return performRequest("POST", uri, payload, params);
- }
-
- private String concatJsonString(String target, String addition) {
- return target.substring(0, target.length() - 1) + "," + addition.substring(1);
- }
-
- private Response performRequest(String method, String uri) {
- return performRequest(method, uri, null);
- }
-
- private Response performRequest(String method, String uri, @Nullable Object payload) {
- return performRequest(method, uri, payload, Collections.emptyMap());
- }
-
- private Response performRequest(
- String method, String uri, @Nullable Object payload, Map<String, String> params) {
- Request request = new Request(method, uri.startsWith("/") ? uri : "/" + uri);
- if (payload != null) {
- String payloadStr = payload instanceof String ? (String) payload : payload.toString();
- request.setEntity(new NStringEntity(payloadStr, ContentType.APPLICATION_JSON));
- }
- for (Map.Entry<String, String> entry : params.entrySet()) {
- request.addParameter(entry.getKey(), entry.getValue());
- }
- try {
- return client.get().performRequest(request);
- } catch (IOException e) {
- throw new StorageException(e);
- }
- }
-
- protected class ElasticQuerySource implements DataSource<V> {
- private final QueryOptions opts;
- private final String search;
-
- ElasticQuerySource(Predicate<V> p, QueryOptions opts, JsonArray sortArray)
- throws QueryParseException {
- this.opts = opts;
- QueryBuilder qb = queryBuilder.toQueryBuilder(p);
- SearchSourceBuilder searchSource =
- new SearchSourceBuilder(client.adapter())
- .query(qb)
- .from(opts.start())
- .size(opts.limit())
- .fields(Lists.newArrayList(opts.fields()));
- search = getSearch(searchSource, sortArray);
- }
-
- @Override
- public int getCardinality() {
- return 10;
- }
-
- @Override
- public ResultSet<V> read() {
- return readImpl(doc -> AbstractElasticIndex.this.fromDocument(doc, opts.fields()));
- }
-
- @Override
- public ResultSet<FieldBundle> readRaw() {
- return readImpl(AbstractElasticIndex.this::toFieldBundle);
- }
-
- private <T> ResultSet<T> readImpl(Function<JsonObject, T> mapper) {
- try {
- String uri = getURI(SEARCH);
- Response response =
- performRequest(HttpPost.METHOD_NAME, uri, search, Collections.emptyMap());
- StatusLine statusLine = response.getStatusLine();
- if (statusLine.getStatusCode() == HttpStatus.SC_OK) {
- String content = getContent(response);
- JsonObject obj =
- new JsonParser().parse(content).getAsJsonObject().getAsJsonObject("hits");
- if (obj.get("hits") != null) {
- JsonArray json = obj.getAsJsonArray("hits");
- ImmutableList.Builder<T> results = ImmutableList.builderWithExpectedSize(json.size());
- for (int i = 0; i < json.size(); i++) {
- T mapperResult = mapper.apply(json.get(i).getAsJsonObject());
- if (mapperResult != null) {
- results.add(mapperResult);
- }
- }
- return new ListResultSet<>(results.build());
- }
- } else {
- logger.atSevere().log(statusLine.getReasonPhrase());
- }
- return new ListResultSet<>(ImmutableList.of());
- } catch (IOException e) {
- throw new StorageException(e);
- }
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/BUILD b/java/com/google/gerrit/elasticsearch/BUILD
deleted file mode 100644
index 8bab80b..0000000
--- a/java/com/google/gerrit/elasticsearch/BUILD
+++ /dev/null
@@ -1,33 +0,0 @@
-load("@rules_java//java:defs.bzl", "java_library")
-
-java_library(
- name = "elasticsearch",
- srcs = glob(["**/*.java"]),
- visibility = ["//visibility:public"],
- deps = [
- "//java/com/google/gerrit/common:annotations",
- "//java/com/google/gerrit/entities",
- "//java/com/google/gerrit/exceptions",
- "//java/com/google/gerrit/extensions:api",
- "//java/com/google/gerrit/index",
- "//java/com/google/gerrit/index:query_exception",
- "//java/com/google/gerrit/index/project",
- "//java/com/google/gerrit/lifecycle",
- "//java/com/google/gerrit/proto",
- "//java/com/google/gerrit/server",
- "//lib:gson",
- "//lib:guava",
- "//lib:jgit",
- "//lib:protobuf",
- "//lib/commons:lang",
- "//lib/elasticsearch-rest-client",
- "//lib/flogger:api",
- "//lib/guice",
- "//lib/guice:guice-assistedinject",
- "//lib/httpcomponents:httpasyncclient",
- "//lib/httpcomponents:httpclient",
- "//lib/httpcomponents:httpcore",
- "//lib/httpcomponents:httpcore-nio",
- "//lib/jackson:jackson-core",
- ],
-)
diff --git a/java/com/google/gerrit/elasticsearch/ElasticAccountIndex.java b/java/com/google/gerrit/elasticsearch/ElasticAccountIndex.java
deleted file mode 100644
index 8967789..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticAccountIndex.java
+++ /dev/null
@@ -1,143 +0,0 @@
-// Copyright (C) 2016 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.collect.ImmutableSet;
-import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
-import com.google.gerrit.elasticsearch.bulk.BulkRequest;
-import com.google.gerrit.elasticsearch.bulk.IndexRequest;
-import com.google.gerrit.elasticsearch.bulk.UpdateRequest;
-import com.google.gerrit.entities.Account;
-import com.google.gerrit.exceptions.StorageException;
-import com.google.gerrit.index.QueryOptions;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.query.DataSource;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.server.account.AccountCache;
-import com.google.gerrit.server.account.AccountState;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.IndexUtils;
-import com.google.gerrit.server.index.account.AccountField;
-import com.google.gerrit.server.index.account.AccountIndex;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.inject.Inject;
-import com.google.inject.Provider;
-import com.google.inject.assistedinject.Assisted;
-import java.util.Set;
-import org.apache.http.HttpStatus;
-import org.elasticsearch.client.Response;
-
-public class ElasticAccountIndex extends AbstractElasticIndex<Account.Id, AccountState>
- implements AccountIndex {
- static class AccountMapping {
- final MappingProperties accounts;
-
- AccountMapping(Schema<AccountState> schema, ElasticQueryAdapter adapter) {
- this.accounts = ElasticMapping.createMapping(schema, adapter);
- }
- }
-
- private static final String ACCOUNTS = "accounts";
-
- private final AccountMapping mapping;
- private final Provider<AccountCache> accountCache;
- private final Schema<AccountState> schema;
-
- @Inject
- ElasticAccountIndex(
- ElasticConfiguration cfg,
- SitePaths sitePaths,
- Provider<AccountCache> accountCache,
- ElasticRestClientProvider client,
- @Assisted Schema<AccountState> schema) {
- super(cfg, sitePaths, schema, client, ACCOUNTS);
- this.accountCache = accountCache;
- this.mapping = new AccountMapping(schema, client.adapter());
- this.schema = schema;
- }
-
- @Override
- public void replace(AccountState as) {
- BulkRequest bulk =
- new IndexRequest(getId(as), indexName)
- .add(new UpdateRequest<>(schema, as, ImmutableSet.of()));
-
- String uri = getURI(BULK);
- Response response = postRequest(uri, bulk, getRefreshParam());
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format(
- "Failed to replace account %s in index %s: %s",
- as.account().id(), indexName, statusCode));
- }
- }
-
- @Override
- public DataSource<AccountState> getSource(Predicate<AccountState> p, QueryOptions opts)
- throws QueryParseException {
- JsonArray sortArray =
- getSortArray(
- schema.useLegacyNumericFields()
- ? AccountField.ID.getName()
- : AccountField.ID_STR.getName());
- return new ElasticQuerySource(
- p,
- opts.filterFields(o -> IndexUtils.accountFields(o, schema.useLegacyNumericFields())),
- sortArray);
- }
-
- @Override
- protected String getDeleteActions(Account.Id a) {
- return getDeleteRequest(a);
- }
-
- @Override
- protected String getMappings() {
- return getMappingsForSingleType(mapping.accounts);
- }
-
- @Override
- protected String getId(AccountState as) {
- return as.account().id().toString();
- }
-
- @Override
- protected AccountState fromDocument(JsonObject json, Set<String> fields) {
- JsonElement source = json.get("_source");
- if (source == null) {
- source = json.getAsJsonObject().get("fields");
- }
-
- Account.Id id =
- Account.id(
- source
- .getAsJsonObject()
- .get(
- schema.useLegacyNumericFields()
- ? AccountField.ID.getName()
- : AccountField.ID_STR.getName())
- .getAsInt());
- // Use the AccountCache rather than depending on any stored fields in the document (of which
- // there shouldn't be any). The most expensive part to compute anyway is the effective group
- // IDs, and we don't have a good way to reindex when those change.
- // If the account doesn't exist return an empty AccountState to represent the missing account
- // to account the fact that the account exists in the index.
- return accountCache.get().getEvenIfMissing(id);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticChangeIndex.java b/java/com/google/gerrit/elasticsearch/ElasticChangeIndex.java
deleted file mode 100644
index 7d4e0c7..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticChangeIndex.java
+++ /dev/null
@@ -1,181 +0,0 @@
-// Copyright (C) 2014 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static java.util.Objects.requireNonNull;
-
-import com.google.common.collect.ImmutableSet;
-import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
-import com.google.gerrit.elasticsearch.bulk.BulkRequest;
-import com.google.gerrit.elasticsearch.bulk.IndexRequest;
-import com.google.gerrit.elasticsearch.bulk.UpdateRequest;
-import com.google.gerrit.entities.Change;
-import com.google.gerrit.entities.Project;
-import com.google.gerrit.entities.converter.ChangeProtoConverter;
-import com.google.gerrit.exceptions.StorageException;
-import com.google.gerrit.index.FieldDef;
-import com.google.gerrit.index.QueryOptions;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.query.DataSource;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.server.change.MergeabilityComputationBehavior;
-import com.google.gerrit.server.config.GerritServerConfig;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.IndexUtils;
-import com.google.gerrit.server.index.change.ChangeField;
-import com.google.gerrit.server.index.change.ChangeIndex;
-import com.google.gerrit.server.query.change.ChangeData;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.inject.Inject;
-import com.google.inject.assistedinject.Assisted;
-import java.util.Set;
-import org.apache.http.HttpStatus;
-import org.eclipse.jgit.lib.Config;
-import org.elasticsearch.client.Response;
-
-/** Secondary index implementation using Elasticsearch. */
-class ElasticChangeIndex extends AbstractElasticIndex<Change.Id, ChangeData>
- implements ChangeIndex {
- static class ChangeMapping {
- final MappingProperties changes;
- final MappingProperties openChanges;
- final MappingProperties closedChanges;
-
- ChangeMapping(Schema<ChangeData> schema, ElasticQueryAdapter adapter) {
- MappingProperties mapping = ElasticMapping.createMapping(schema, adapter);
- this.changes = mapping;
- this.openChanges = mapping;
- this.closedChanges = mapping;
- }
- }
-
- private static final String CHANGES = "changes";
-
- private final ChangeMapping mapping;
- private final ChangeData.Factory changeDataFactory;
- private final Schema<ChangeData> schema;
- private final FieldDef<ChangeData, ?> idField;
- private final ImmutableSet<String> skipFields;
-
- @Inject
- ElasticChangeIndex(
- ElasticConfiguration cfg,
- ChangeData.Factory changeDataFactory,
- SitePaths sitePaths,
- ElasticRestClientProvider clientBuilder,
- @GerritServerConfig Config gerritConfig,
- @Assisted Schema<ChangeData> schema) {
- super(cfg, sitePaths, schema, clientBuilder, CHANGES);
- this.changeDataFactory = changeDataFactory;
- this.schema = schema;
- this.mapping = new ChangeMapping(schema, client.adapter());
- this.idField =
- this.schema.useLegacyNumericFields() ? ChangeField.LEGACY_ID : ChangeField.LEGACY_ID_STR;
- this.skipFields =
- MergeabilityComputationBehavior.fromConfig(gerritConfig).includeInIndex()
- ? ImmutableSet.of()
- : ImmutableSet.of(ChangeField.MERGEABLE.getName());
- }
-
- @Override
- public void replace(ChangeData cd) {
- BulkRequest bulk =
- new IndexRequest(getId(cd), indexName).add(new UpdateRequest<>(schema, cd, skipFields));
-
- String uri = getURI(BULK);
- Response response = postRequest(uri, bulk, getRefreshParam());
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format(
- "Failed to replace change %s in index %s: %s", cd.getId(), indexName, statusCode));
- }
- }
-
- @Override
- public DataSource<ChangeData> getSource(Predicate<ChangeData> p, QueryOptions opts)
- throws QueryParseException {
- QueryOptions filteredOpts =
- opts.filterFields(o -> IndexUtils.changeFields(o, schema.useLegacyNumericFields()));
- return new ElasticQuerySource(p, filteredOpts, getSortArray());
- }
-
- private JsonArray getSortArray() {
- JsonObject properties = new JsonObject();
- properties.addProperty(ORDER, DESC_SORT_ORDER);
-
- JsonArray sortArray = new JsonArray();
- addNamedElement(ChangeField.UPDATED.getName(), properties, sortArray);
- addNamedElement(ChangeField.MERGED_ON.getName(), getMergedOnSortOptions(), sortArray);
- addNamedElement(idField.getName(), properties, sortArray);
- return sortArray;
- }
-
- private JsonObject getMergedOnSortOptions() {
- JsonObject sortOptions = new JsonObject();
- sortOptions.addProperty(ORDER, DESC_SORT_ORDER);
- // Ignore the sort field if it does not exist in index. Otherwise the search would fail on open
- // changes, because the corresponding documents do not have mergedOn field.
- sortOptions.addProperty(UNMAPPED_TYPE, ElasticMapping.TIMESTAMP_FIELD_TYPE);
- return sortOptions;
- }
-
- @Override
- protected String getDeleteActions(Change.Id c) {
- return getDeleteRequest(c);
- }
-
- @Override
- protected String getMappings() {
- return getMappingsFor(mapping.changes);
- }
-
- @Override
- protected String getId(ChangeData cd) {
- return cd.getId().toString();
- }
-
- @Override
- protected ChangeData fromDocument(JsonObject json, Set<String> fields) {
- JsonElement sourceElement = json.get("_source");
- if (sourceElement == null) {
- sourceElement = json.getAsJsonObject().get("fields");
- }
- JsonObject source = sourceElement.getAsJsonObject();
- JsonElement c = source.get(ChangeField.CHANGE.getName());
-
- if (c == null) {
- int id = source.get(idField.getName()).getAsInt();
- // IndexUtils#changeFields ensures either CHANGE or PROJECT is always present.
- String projectName = requireNonNull(source.get(ChangeField.PROJECT.getName()).getAsString());
- return changeDataFactory.create(Project.nameKey(projectName), Change.id(id));
- }
-
- ChangeData cd =
- changeDataFactory.create(
- parseProtoFrom(decodeBase64(c.getAsString()), ChangeProtoConverter.INSTANCE));
-
- for (FieldDef<ChangeData, ?> field : getSchema().getFields().values()) {
- if (fields.contains(field.getName()) && source.get(field.getName()) != null) {
- field.setIfPossible(cd, new ElasticStoredValue(source.get(field.getName())));
- }
- }
-
- return cd;
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticConfiguration.java b/java/com/google/gerrit/elasticsearch/ElasticConfiguration.java
deleted file mode 100644
index c4435297..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticConfiguration.java
+++ /dev/null
@@ -1,137 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static com.google.common.base.MoreObjects.firstNonNull;
-
-import com.google.common.base.Strings;
-import com.google.common.flogger.FluentLogger;
-import com.google.gerrit.server.config.GerritServerConfig;
-import com.google.inject.Inject;
-import com.google.inject.ProvisionException;
-import com.google.inject.Singleton;
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.concurrent.TimeUnit;
-import org.apache.http.HttpHost;
-import org.eclipse.jgit.lib.Config;
-import org.elasticsearch.client.RestClientBuilder;
-
-@Singleton
-class ElasticConfiguration {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- static final String SECTION_ELASTICSEARCH = "elasticsearch";
- static final String KEY_PASSWORD = "password";
- static final String KEY_USERNAME = "username";
- static final String KEY_PREFIX = "prefix";
- static final String KEY_SERVER = "server";
- static final String KEY_NUMBER_OF_SHARDS = "numberOfShards";
- static final String KEY_NUMBER_OF_REPLICAS = "numberOfReplicas";
- static final String KEY_MAX_RESULT_WINDOW = "maxResultWindow";
- static final String KEY_CONNECT_TIMEOUT = "connectTimeout";
- static final String KEY_SOCKET_TIMEOUT = "socketTimeout";
-
- static final String DEFAULT_PORT = "9200";
- static final String DEFAULT_USERNAME = "elastic";
- static final int DEFAULT_NUMBER_OF_SHARDS = 1;
- static final int DEFAULT_NUMBER_OF_REPLICAS = 1;
- static final int DEFAULT_MAX_RESULT_WINDOW = 10000;
- static final int DEFAULT_CONNECT_TIMEOUT = RestClientBuilder.DEFAULT_CONNECT_TIMEOUT_MILLIS;
- static final int DEFAULT_SOCKET_TIMEOUT = RestClientBuilder.DEFAULT_SOCKET_TIMEOUT_MILLIS;
-
- private final Config cfg;
- private final List<HttpHost> hosts;
-
- final String username;
- final String password;
- final int numberOfShards;
- final int numberOfReplicas;
- final int maxResultWindow;
- final int connectTimeout;
- final int socketTimeout;
- final String prefix;
-
- @Inject
- ElasticConfiguration(@GerritServerConfig Config cfg) {
- this.cfg = cfg;
- this.password = cfg.getString(SECTION_ELASTICSEARCH, null, KEY_PASSWORD);
- this.username =
- password == null
- ? null
- : firstNonNull(
- cfg.getString(SECTION_ELASTICSEARCH, null, KEY_USERNAME), DEFAULT_USERNAME);
- this.prefix = Strings.nullToEmpty(cfg.getString(SECTION_ELASTICSEARCH, null, KEY_PREFIX));
- this.numberOfShards =
- cfg.getInt(SECTION_ELASTICSEARCH, null, KEY_NUMBER_OF_SHARDS, DEFAULT_NUMBER_OF_SHARDS);
- this.numberOfReplicas =
- cfg.getInt(SECTION_ELASTICSEARCH, null, KEY_NUMBER_OF_REPLICAS, DEFAULT_NUMBER_OF_REPLICAS);
- this.maxResultWindow =
- cfg.getInt(SECTION_ELASTICSEARCH, null, KEY_MAX_RESULT_WINDOW, DEFAULT_MAX_RESULT_WINDOW);
- this.connectTimeout =
- (int)
- cfg.getTimeUnit(
- SECTION_ELASTICSEARCH,
- null,
- KEY_CONNECT_TIMEOUT,
- DEFAULT_CONNECT_TIMEOUT,
- TimeUnit.MILLISECONDS);
- this.socketTimeout =
- (int)
- cfg.getTimeUnit(
- SECTION_ELASTICSEARCH,
- null,
- KEY_SOCKET_TIMEOUT,
- DEFAULT_SOCKET_TIMEOUT,
- TimeUnit.MILLISECONDS);
- this.hosts = new ArrayList<>();
- for (String server : cfg.getStringList(SECTION_ELASTICSEARCH, null, KEY_SERVER)) {
- try {
- URI uri = new URI(server);
- int port = uri.getPort();
- HttpHost httpHost =
- new HttpHost(
- uri.getHost(), port == -1 ? Integer.valueOf(DEFAULT_PORT) : port, uri.getScheme());
- this.hosts.add(httpHost);
- } catch (URISyntaxException | IllegalArgumentException e) {
- logger.atSevere().log("Invalid server URI %s: %s", server, e.getMessage());
- }
- }
-
- if (hosts.isEmpty()) {
- throw new ProvisionException("No valid Elasticsearch servers configured");
- }
-
- logger.atInfo().log("Elasticsearch servers: %s", hosts);
- }
-
- Config getConfig() {
- return cfg;
- }
-
- HttpHost[] getHosts() {
- return hosts.toArray(new HttpHost[hosts.size()]);
- }
-
- String getIndexName(String name, int schemaVersion) {
- return String.format("%s%s_%04d", prefix, name, schemaVersion);
- }
-
- int getNumberOfShards() {
- return numberOfShards;
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticException.java b/java/com/google/gerrit/elasticsearch/ElasticException.java
deleted file mode 100644
index d4baf75..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticException.java
+++ /dev/null
@@ -1,27 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-class ElasticException extends RuntimeException {
- private static final long serialVersionUID = 1L;
-
- ElasticException(String message) {
- super(message);
- }
-
- ElasticException(String message, Throwable cause) {
- super(message, cause);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticGroupIndex.java b/java/com/google/gerrit/elasticsearch/ElasticGroupIndex.java
deleted file mode 100644
index 781ed43..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticGroupIndex.java
+++ /dev/null
@@ -1,126 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.collect.ImmutableSet;
-import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
-import com.google.gerrit.elasticsearch.bulk.BulkRequest;
-import com.google.gerrit.elasticsearch.bulk.IndexRequest;
-import com.google.gerrit.elasticsearch.bulk.UpdateRequest;
-import com.google.gerrit.entities.AccountGroup;
-import com.google.gerrit.entities.InternalGroup;
-import com.google.gerrit.exceptions.StorageException;
-import com.google.gerrit.index.QueryOptions;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.query.DataSource;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.server.account.GroupCache;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.IndexUtils;
-import com.google.gerrit.server.index.group.GroupField;
-import com.google.gerrit.server.index.group.GroupIndex;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.inject.Inject;
-import com.google.inject.Provider;
-import com.google.inject.assistedinject.Assisted;
-import java.util.Set;
-import org.apache.http.HttpStatus;
-import org.elasticsearch.client.Response;
-
-public class ElasticGroupIndex extends AbstractElasticIndex<AccountGroup.UUID, InternalGroup>
- implements GroupIndex {
- static class GroupMapping {
- final MappingProperties groups;
-
- GroupMapping(Schema<InternalGroup> schema, ElasticQueryAdapter adapter) {
- this.groups = ElasticMapping.createMapping(schema, adapter);
- }
- }
-
- private static final String GROUPS = "groups";
-
- private final GroupMapping mapping;
- private final Provider<GroupCache> groupCache;
- private final Schema<InternalGroup> schema;
-
- @Inject
- ElasticGroupIndex(
- ElasticConfiguration cfg,
- SitePaths sitePaths,
- Provider<GroupCache> groupCache,
- ElasticRestClientProvider client,
- @Assisted Schema<InternalGroup> schema) {
- super(cfg, sitePaths, schema, client, GROUPS);
- this.groupCache = groupCache;
- this.mapping = new GroupMapping(schema, client.adapter());
- this.schema = schema;
- }
-
- @Override
- public void replace(InternalGroup group) {
- BulkRequest bulk =
- new IndexRequest(getId(group), indexName)
- .add(new UpdateRequest<>(schema, group, ImmutableSet.of()));
-
- String uri = getURI(BULK);
- Response response = postRequest(uri, bulk, getRefreshParam());
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format(
- "Failed to replace group %s in index %s: %s",
- group.getGroupUUID().get(), indexName, statusCode));
- }
- }
-
- @Override
- public DataSource<InternalGroup> getSource(Predicate<InternalGroup> p, QueryOptions opts)
- throws QueryParseException {
- JsonArray sortArray = getSortArray(GroupField.UUID.getName());
- return new ElasticQuerySource(p, opts.filterFields(IndexUtils::groupFields), sortArray);
- }
-
- @Override
- protected String getDeleteActions(AccountGroup.UUID g) {
- return getDeleteRequest(g);
- }
-
- @Override
- protected String getMappings() {
- return getMappingsForSingleType(mapping.groups);
- }
-
- @Override
- protected String getId(InternalGroup group) {
- return group.getGroupUUID().get();
- }
-
- @Override
- protected InternalGroup fromDocument(JsonObject json, Set<String> fields) {
- JsonElement source = json.get("_source");
- if (source == null) {
- source = json.getAsJsonObject().get("fields");
- }
-
- AccountGroup.UUID uuid =
- AccountGroup.uuid(source.getAsJsonObject().get(GroupField.UUID.getName()).getAsString());
- // Use the GroupCache rather than depending on any stored fields in the
- // document (of which there shouldn't be any).
- return groupCache.get().get(uuid).orElse(null);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticIndexModule.java b/java/com/google/gerrit/elasticsearch/ElasticIndexModule.java
deleted file mode 100644
index 15d6126..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticIndexModule.java
+++ /dev/null
@@ -1,69 +0,0 @@
-// Copyright (C) 2014 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.index.project.ProjectIndex;
-import com.google.gerrit.server.index.AbstractIndexModule;
-import com.google.gerrit.server.index.VersionManager;
-import com.google.gerrit.server.index.account.AccountIndex;
-import com.google.gerrit.server.index.change.ChangeIndex;
-import com.google.gerrit.server.index.group.GroupIndex;
-import java.util.Map;
-
-public class ElasticIndexModule extends AbstractIndexModule {
- public static ElasticIndexModule singleVersionWithExplicitVersions(
- Map<String, Integer> versions, int threads, boolean slave) {
- return new ElasticIndexModule(versions, threads, slave);
- }
-
- public static ElasticIndexModule latestVersion(boolean slave) {
- return new ElasticIndexModule(null, 0, slave);
- }
-
- private ElasticIndexModule(Map<String, Integer> singleVersions, int threads, boolean slave) {
- super(singleVersions, threads, slave);
- }
-
- @Override
- public void configure() {
- super.configure();
- install(ElasticRestClientProvider.module());
- }
-
- @Override
- protected Class<? extends AccountIndex> getAccountIndex() {
- return ElasticAccountIndex.class;
- }
-
- @Override
- protected Class<? extends ChangeIndex> getChangeIndex() {
- return ElasticChangeIndex.class;
- }
-
- @Override
- protected Class<? extends GroupIndex> getGroupIndex() {
- return ElasticGroupIndex.class;
- }
-
- @Override
- protected Class<? extends ProjectIndex> getProjectIndex() {
- return ElasticProjectIndex.class;
- }
-
- @Override
- protected Class<? extends VersionManager> getVersionManager() {
- return ElasticIndexVersionManager.class;
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticIndexVersionDiscovery.java b/java/com/google/gerrit/elasticsearch/ElasticIndexVersionDiscovery.java
deleted file mode 100644
index 100022a..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticIndexVersionDiscovery.java
+++ /dev/null
@@ -1,61 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static java.util.stream.Collectors.toList;
-
-import com.google.common.flogger.FluentLogger;
-import com.google.gson.JsonParser;
-import com.google.inject.Inject;
-import com.google.inject.Singleton;
-import java.io.IOException;
-import java.util.List;
-import org.apache.http.HttpStatus;
-import org.apache.http.StatusLine;
-import org.elasticsearch.client.Request;
-import org.elasticsearch.client.Response;
-
-@Singleton
-class ElasticIndexVersionDiscovery {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- private final ElasticRestClientProvider client;
-
- @Inject
- ElasticIndexVersionDiscovery(ElasticRestClientProvider client) {
- this.client = client;
- }
-
- List<String> discover(String prefix, String indexName) throws IOException {
- String name = prefix + indexName + "_";
- Request request = new Request("GET", client.adapter().getVersionDiscoveryUrl(name));
- Response response = client.get().performRequest(request);
-
- StatusLine statusLine = response.getStatusLine();
- if (statusLine.getStatusCode() != HttpStatus.SC_OK) {
- String message =
- String.format(
- "Failed to discover index versions for %s: %d: %s",
- name, statusLine.getStatusCode(), statusLine.getReasonPhrase());
- logger.atSevere().log(message);
- throw new IOException(message);
- }
-
- return new JsonParser()
- .parse(AbstractElasticIndex.getContent(response)).getAsJsonObject().entrySet().stream()
- .map(e -> e.getKey().replace(name, ""))
- .collect(toList());
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticIndexVersionManager.java b/java/com/google/gerrit/elasticsearch/ElasticIndexVersionManager.java
deleted file mode 100644
index b9d86d5..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticIndexVersionManager.java
+++ /dev/null
@@ -1,82 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.base.Strings;
-import com.google.common.flogger.FluentLogger;
-import com.google.common.primitives.Ints;
-import com.google.gerrit.index.Index;
-import com.google.gerrit.index.IndexDefinition;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.server.config.GerritServerConfig;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.GerritIndexStatus;
-import com.google.gerrit.server.index.OnlineUpgradeListener;
-import com.google.gerrit.server.index.VersionManager;
-import com.google.gerrit.server.plugincontext.PluginSetContext;
-import com.google.inject.Inject;
-import com.google.inject.Singleton;
-import java.io.IOException;
-import java.util.Collection;
-import java.util.List;
-import java.util.TreeMap;
-import org.eclipse.jgit.lib.Config;
-
-@Singleton
-public class ElasticIndexVersionManager extends VersionManager {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- private final String prefix;
- private final ElasticIndexVersionDiscovery versionDiscovery;
-
- @Inject
- ElasticIndexVersionManager(
- @GerritServerConfig Config cfg,
- SitePaths sitePaths,
- PluginSetContext<OnlineUpgradeListener> listeners,
- Collection<IndexDefinition<?, ?, ?>> defs,
- ElasticIndexVersionDiscovery versionDiscovery) {
- super(sitePaths, listeners, defs, VersionManager.getOnlineUpgrade(cfg));
- this.versionDiscovery = versionDiscovery;
- prefix = Strings.nullToEmpty(cfg.getString("elasticsearch", null, "prefix"));
- }
-
- @Override
- protected <K, V, I extends Index<K, V>> TreeMap<Integer, Version<V>> scanVersions(
- IndexDefinition<K, V, I> def, GerritIndexStatus cfg) {
- TreeMap<Integer, Version<V>> versions = new TreeMap<>();
- try {
- List<String> discovered = versionDiscovery.discover(prefix, def.getName());
- logger.atFine().log("Discovered versions for %s: %s", def.getName(), discovered);
- for (String version : discovered) {
- Integer v = Ints.tryParse(version);
- if (v == null || version.length() != 4) {
- logger.atWarning().log("Unrecognized version in index %s: %s", def.getName(), version);
- continue;
- }
- versions.put(v, new Version<>(null, v, true, cfg.getReady(def.getName(), v)));
- }
- } catch (IOException e) {
- logger.atSevere().withCause(e).log("Error scanning index: %s", def.getName());
- }
-
- for (Schema<V> schema : def.getSchemas().values()) {
- int v = schema.getVersion();
- boolean exists = versions.containsKey(v);
- versions.put(v, new Version<>(schema, v, exists, cfg.getReady(def.getName(), v)));
- }
- return versions;
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticMapping.java b/java/com/google/gerrit/elasticsearch/ElasticMapping.java
deleted file mode 100644
index edd05c9..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticMapping.java
+++ /dev/null
@@ -1,123 +0,0 @@
-// Copyright (C) 2016 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.collect.ImmutableMap;
-import com.google.gerrit.index.FieldDef;
-import com.google.gerrit.index.FieldType;
-import com.google.gerrit.index.Schema;
-import java.util.Map;
-
-class ElasticMapping {
-
- protected static final String TIMESTAMP_FIELD_TYPE = "date";
- protected static final String TIMESTAMP_FIELD_FORMAT = "dateOptionalTime";
-
- static MappingProperties createMapping(Schema<?> schema, ElasticQueryAdapter adapter) {
- ElasticMapping.Builder mapping = new ElasticMapping.Builder(adapter);
- for (FieldDef<?, ?> field : schema.getFields().values()) {
- String name = field.getName();
- FieldType<?> fieldType = field.getType();
- if (fieldType == FieldType.EXACT) {
- mapping.addExactField(name);
- } else if (fieldType == FieldType.TIMESTAMP) {
- mapping.addTimestamp(name);
- } else if (fieldType == FieldType.INTEGER
- || fieldType == FieldType.INTEGER_RANGE
- || fieldType == FieldType.LONG) {
- mapping.addNumber(name);
- } else if (fieldType == FieldType.FULL_TEXT) {
- mapping.addStringWithAnalyzer(name);
- } else if (fieldType == FieldType.PREFIX || fieldType == FieldType.STORED_ONLY) {
- mapping.addString(name);
- } else {
- throw new IllegalStateException("Unsupported field type: " + fieldType.getName());
- }
- }
- return mapping.build();
- }
-
- static class Builder {
- private final ElasticQueryAdapter adapter;
- private final ImmutableMap.Builder<String, FieldProperties> fields =
- new ImmutableMap.Builder<>();
-
- Builder(ElasticQueryAdapter adapter) {
- this.adapter = adapter;
- }
-
- MappingProperties build() {
- MappingProperties properties = new MappingProperties();
- properties.properties = fields.build();
- return properties;
- }
-
- Builder addExactField(String name) {
- FieldProperties key = new FieldProperties(adapter.exactFieldType());
- key.index = adapter.indexProperty();
- FieldProperties properties;
- properties = new FieldProperties(adapter.exactFieldType());
- properties.fields = ImmutableMap.of("key", key);
- fields.put(name, properties);
- return this;
- }
-
- Builder addTimestamp(String name) {
- FieldProperties properties = new FieldProperties(TIMESTAMP_FIELD_TYPE);
- properties.type = TIMESTAMP_FIELD_TYPE;
- properties.format = TIMESTAMP_FIELD_FORMAT;
- fields.put(name, properties);
- return this;
- }
-
- Builder addNumber(String name) {
- fields.put(name, new FieldProperties("long"));
- return this;
- }
-
- Builder addString(String name) {
- fields.put(name, new FieldProperties(adapter.stringFieldType()));
- return this;
- }
-
- Builder addStringWithAnalyzer(String name) {
- FieldProperties key = new FieldProperties(adapter.stringFieldType());
- key.analyzer = "custom_with_char_filter";
- fields.put(name, key);
- return this;
- }
-
- Builder add(String name, String type) {
- fields.put(name, new FieldProperties(type));
- return this;
- }
- }
-
- static class MappingProperties {
- Map<String, FieldProperties> properties;
- }
-
- static class FieldProperties {
- String type;
- String index;
- String format;
- String analyzer;
- Map<String, FieldProperties> fields;
-
- FieldProperties(String type) {
- this.type = type;
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticProjectIndex.java b/java/com/google/gerrit/elasticsearch/ElasticProjectIndex.java
deleted file mode 100644
index b8bfc38..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticProjectIndex.java
+++ /dev/null
@@ -1,130 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.collect.ImmutableSet;
-import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
-import com.google.gerrit.elasticsearch.bulk.BulkRequest;
-import com.google.gerrit.elasticsearch.bulk.IndexRequest;
-import com.google.gerrit.elasticsearch.bulk.UpdateRequest;
-import com.google.gerrit.entities.Project;
-import com.google.gerrit.exceptions.StorageException;
-import com.google.gerrit.index.QueryOptions;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.project.ProjectData;
-import com.google.gerrit.index.project.ProjectField;
-import com.google.gerrit.index.project.ProjectIndex;
-import com.google.gerrit.index.query.DataSource;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.server.config.SitePaths;
-import com.google.gerrit.server.index.IndexUtils;
-import com.google.gerrit.server.project.ProjectCache;
-import com.google.gerrit.server.project.ProjectState;
-import com.google.gson.JsonArray;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.inject.Inject;
-import com.google.inject.Provider;
-import com.google.inject.assistedinject.Assisted;
-import java.util.Optional;
-import java.util.Set;
-import org.apache.http.HttpStatus;
-import org.elasticsearch.client.Response;
-
-public class ElasticProjectIndex extends AbstractElasticIndex<Project.NameKey, ProjectData>
- implements ProjectIndex {
- static class ProjectMapping {
- MappingProperties projects;
-
- ProjectMapping(Schema<ProjectData> schema, ElasticQueryAdapter adapter) {
- this.projects = ElasticMapping.createMapping(schema, adapter);
- }
- }
-
- static final String PROJECTS = "projects";
-
- private final ProjectMapping mapping;
- private final Provider<ProjectCache> projectCache;
- private final Schema<ProjectData> schema;
-
- @Inject
- ElasticProjectIndex(
- ElasticConfiguration cfg,
- SitePaths sitePaths,
- Provider<ProjectCache> projectCache,
- ElasticRestClientProvider client,
- @Assisted Schema<ProjectData> schema) {
- super(cfg, sitePaths, schema, client, PROJECTS);
- this.projectCache = projectCache;
- this.schema = schema;
- this.mapping = new ProjectMapping(schema, client.adapter());
- }
-
- @Override
- public void replace(ProjectData projectState) {
- BulkRequest bulk =
- new IndexRequest(projectState.getProject().getName(), indexName)
- .add(new UpdateRequest<>(schema, projectState, ImmutableSet.of()));
-
- String uri = getURI(BULK);
- Response response = postRequest(uri, bulk, getRefreshParam());
- int statusCode = response.getStatusLine().getStatusCode();
- if (statusCode != HttpStatus.SC_OK) {
- throw new StorageException(
- String.format(
- "Failed to replace project %s in index %s: %s",
- projectState.getProject().getName(), indexName, statusCode));
- }
- }
-
- @Override
- public DataSource<ProjectData> getSource(Predicate<ProjectData> p, QueryOptions opts)
- throws QueryParseException {
- JsonArray sortArray = getSortArray(ProjectField.NAME.getName());
- return new ElasticQuerySource(p, opts.filterFields(IndexUtils::projectFields), sortArray);
- }
-
- @Override
- protected String getDeleteActions(Project.NameKey nameKey) {
- return getDeleteRequest(nameKey);
- }
-
- @Override
- protected String getMappings() {
- return getMappingsForSingleType(mapping.projects);
- }
-
- @Override
- protected String getId(ProjectData projectState) {
- return projectState.getProject().getName();
- }
-
- @Override
- protected ProjectData fromDocument(JsonObject json, Set<String> fields) {
- JsonElement source = json.get("_source");
- if (source == null) {
- source = json.getAsJsonObject().get("fields");
- }
-
- Project.NameKey nameKey =
- Project.nameKey(source.getAsJsonObject().get(ProjectField.NAME.getName()).getAsString());
- Optional<ProjectState> state = projectCache.get().get(nameKey);
- if (!state.isPresent()) {
- return null;
- }
- return state.get().toProjectData();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticQueryAdapter.java b/java/com/google/gerrit/elasticsearch/ElasticQueryAdapter.java
deleted file mode 100644
index 19d9901..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticQueryAdapter.java
+++ /dev/null
@@ -1,63 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-public class ElasticQueryAdapter {
- private static final String INDICES = "?allow_no_indices=false";
-
- private final String searchFilteringName;
- private final String exactFieldType;
- private final String stringFieldType;
- private final String indexProperty;
- private final String rawFieldsKey;
- private final String versionDiscoveryUrl;
-
- ElasticQueryAdapter() {
- this.versionDiscoveryUrl = "/%s*";
- this.searchFilteringName = "_source";
- this.exactFieldType = "keyword";
- this.stringFieldType = "text";
- this.indexProperty = "true";
- this.rawFieldsKey = "_source";
- }
-
- public String searchFilteringName() {
- return searchFilteringName;
- }
-
- String indicesExistParams() {
- return INDICES;
- }
-
- String exactFieldType() {
- return exactFieldType;
- }
-
- String stringFieldType() {
- return stringFieldType;
- }
-
- String indexProperty() {
- return indexProperty;
- }
-
- String rawFieldsKey() {
- return rawFieldsKey;
- }
-
- String getVersionDiscoveryUrl(String name) {
- return String.format(versionDiscoveryUrl, name);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticQueryBuilder.java b/java/com/google/gerrit/elasticsearch/ElasticQueryBuilder.java
deleted file mode 100644
index 3eaf708..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticQueryBuilder.java
+++ /dev/null
@@ -1,163 +0,0 @@
-// Copyright (C) 2014 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.elasticsearch.builders.BoolQueryBuilder;
-import com.google.gerrit.elasticsearch.builders.QueryBuilder;
-import com.google.gerrit.elasticsearch.builders.QueryBuilders;
-import com.google.gerrit.index.FieldDef;
-import com.google.gerrit.index.FieldType;
-import com.google.gerrit.index.query.AndPredicate;
-import com.google.gerrit.index.query.IndexPredicate;
-import com.google.gerrit.index.query.IntegerRangePredicate;
-import com.google.gerrit.index.query.NotPredicate;
-import com.google.gerrit.index.query.OrPredicate;
-import com.google.gerrit.index.query.PostFilterPredicate;
-import com.google.gerrit.index.query.Predicate;
-import com.google.gerrit.index.query.QueryParseException;
-import com.google.gerrit.index.query.RegexPredicate;
-import com.google.gerrit.index.query.TimestampRangePredicate;
-import java.time.Instant;
-
-public class ElasticQueryBuilder {
-
- <T> QueryBuilder toQueryBuilder(Predicate<T> p) throws QueryParseException {
- if (p instanceof AndPredicate) {
- return and(p);
- } else if (p instanceof OrPredicate) {
- return or(p);
- } else if (p instanceof NotPredicate) {
- return not(p);
- } else if (p instanceof IndexPredicate) {
- return fieldQuery((IndexPredicate<T>) p);
- } else if (p instanceof PostFilterPredicate) {
- return QueryBuilders.matchAllQuery();
- } else {
- throw new QueryParseException("cannot create query for index: " + p);
- }
- }
-
- private <T> BoolQueryBuilder and(Predicate<T> p) throws QueryParseException {
- BoolQueryBuilder b = QueryBuilders.boolQuery();
- for (Predicate<T> c : p.getChildren()) {
- b.must(toQueryBuilder(c));
- }
- return b;
- }
-
- private <T> BoolQueryBuilder or(Predicate<T> p) throws QueryParseException {
- BoolQueryBuilder q = QueryBuilders.boolQuery();
- for (Predicate<T> c : p.getChildren()) {
- q.should(toQueryBuilder(c));
- }
- return q;
- }
-
- private <T> QueryBuilder not(Predicate<T> p) throws QueryParseException {
- Predicate<T> n = p.getChild(0);
- if (n instanceof TimestampRangePredicate) {
- return notTimestamp((TimestampRangePredicate<T>) n);
- }
-
- // Lucene does not support negation, start with all and subtract.
- BoolQueryBuilder q = QueryBuilders.boolQuery();
- q.must(QueryBuilders.matchAllQuery());
- q.mustNot(toQueryBuilder(n));
- return q;
- }
-
- private <T> QueryBuilder fieldQuery(IndexPredicate<T> p) throws QueryParseException {
- FieldType<?> type = p.getType();
- FieldDef<?, ?> field = p.getField();
- String name = field.getName();
- String value = p.getValue();
-
- if (type == FieldType.INTEGER) {
- // QueryBuilder encodes integer fields as prefix coded bits,
- // which elasticsearch's queryString can't handle.
- // Create integer terms with string representations instead.
- return QueryBuilders.termQuery(name, value);
- } else if (type == FieldType.INTEGER_RANGE) {
- return intRangeQuery(p);
- } else if (type == FieldType.TIMESTAMP) {
- return timestampQuery(p);
- } else if (type == FieldType.EXACT) {
- return exactQuery(p);
- } else if (type == FieldType.PREFIX) {
- return QueryBuilders.matchPhrasePrefixQuery(name, value);
- } else if (type == FieldType.FULL_TEXT) {
- return QueryBuilders.matchPhraseQuery(name, value);
- } else {
- throw FieldType.badFieldType(p.getType());
- }
- }
-
- private <T> QueryBuilder intRangeQuery(IndexPredicate<T> p) throws QueryParseException {
- if (p instanceof IntegerRangePredicate) {
- IntegerRangePredicate<T> r = (IntegerRangePredicate<T>) p;
- int minimum = r.getMinimumValue();
- int maximum = r.getMaximumValue();
- if (minimum == maximum) {
- // Just fall back to a standard integer query.
- return QueryBuilders.termQuery(p.getField().getName(), minimum);
- }
- return QueryBuilders.rangeQuery(p.getField().getName()).gte(minimum).lte(maximum);
- }
- throw new QueryParseException("not an integer range: " + p);
- }
-
- private <T> QueryBuilder notTimestamp(TimestampRangePredicate<T> r) throws QueryParseException {
- if (r.getMinTimestamp().toEpochMilli() == 0) {
- return QueryBuilders.rangeQuery(r.getField().getName())
- .gt(Instant.ofEpochMilli(r.getMaxTimestamp().toEpochMilli()));
- }
- throw new QueryParseException("cannot negate: " + r);
- }
-
- private <T> QueryBuilder timestampQuery(IndexPredicate<T> p) throws QueryParseException {
- if (p instanceof TimestampRangePredicate) {
- TimestampRangePredicate<T> r = (TimestampRangePredicate<T>) p;
- if (r.getMaxTimestamp().toEpochMilli() == Long.MAX_VALUE) {
- // The time range only has the start value, search from the start to the max supported value
- // Long.MAX_VALUE
- return QueryBuilders.rangeQuery(r.getField().getName())
- .gte(Instant.ofEpochMilli(r.getMinTimestamp().toEpochMilli()));
- }
- return QueryBuilders.rangeQuery(r.getField().getName())
- .gte(Instant.ofEpochMilli(r.getMinTimestamp().toEpochMilli()))
- .lte(Instant.ofEpochMilli(r.getMaxTimestamp().toEpochMilli()));
- }
- throw new QueryParseException("not a timestamp: " + p);
- }
-
- private <T> QueryBuilder exactQuery(IndexPredicate<T> p) {
- String name = p.getField().getName();
- String value = p.getValue();
-
- if (!p.getField().isRepeatable() && value.isEmpty()) {
- return new BoolQueryBuilder().mustNot(QueryBuilders.existsQuery(name));
- } else if (p instanceof RegexPredicate) {
- if (value.startsWith("^")) {
- value = value.substring(1);
- }
- if (value.endsWith("$") && !value.endsWith("\\$") && !value.endsWith("\\\\$")) {
- value = value.substring(0, value.length() - 1);
- }
- return QueryBuilders.regexpQuery(name + ".key", value);
- } else {
- return QueryBuilders.termQuery(name + ".key", value);
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticRestClientProvider.java b/java/com/google/gerrit/elasticsearch/ElasticRestClientProvider.java
deleted file mode 100644
index b41f365..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticRestClientProvider.java
+++ /dev/null
@@ -1,157 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.flogger.FluentLogger;
-import com.google.gerrit.extensions.events.LifecycleListener;
-import com.google.gerrit.lifecycle.LifecycleModule;
-import com.google.gson.JsonParser;
-import com.google.inject.Inject;
-import com.google.inject.Provider;
-import com.google.inject.Singleton;
-import java.io.IOException;
-import org.apache.http.HttpStatus;
-import org.apache.http.StatusLine;
-import org.apache.http.auth.AuthScope;
-import org.apache.http.auth.UsernamePasswordCredentials;
-import org.apache.http.client.CredentialsProvider;
-import org.apache.http.client.config.RequestConfig;
-import org.apache.http.impl.client.BasicCredentialsProvider;
-import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
-import org.elasticsearch.client.Request;
-import org.elasticsearch.client.Response;
-import org.elasticsearch.client.RestClient;
-import org.elasticsearch.client.RestClientBuilder;
-
-@Singleton
-class ElasticRestClientProvider implements Provider<RestClient>, LifecycleListener {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- private final ElasticConfiguration cfg;
-
- private volatile RestClient client;
- private ElasticQueryAdapter adapter;
-
- @Inject
- ElasticRestClientProvider(ElasticConfiguration cfg) {
- this.cfg = cfg;
- }
-
- public static LifecycleModule module() {
- return new LifecycleModule() {
- @Override
- protected void configure() {
- listener().to(ElasticRestClientProvider.class);
- }
- };
- }
-
- @Override
- public RestClient get() {
- if (client == null) {
- synchronized (this) {
- if (client == null) {
- client = build();
- ElasticVersion version = getVersion();
- logger.atInfo().log("Elasticsearch integration version %s", version);
- adapter = new ElasticQueryAdapter();
- }
- }
- }
- return client;
- }
-
- @Override
- public void start() {}
-
- @Override
- public void stop() {
- if (client != null) {
- try {
- client.close();
- } catch (IOException e) {
- // Ignore. We can't do anything about it.
- }
- }
- }
-
- ElasticQueryAdapter adapter() {
- get(); // Make sure we're connected
- return adapter;
- }
-
- public static class FailedToGetVersion extends ElasticException {
- private static final long serialVersionUID = 1L;
- private static final String MESSAGE = "Failed to get Elasticsearch version";
-
- FailedToGetVersion(StatusLine status) {
- super(String.format("%s: %d %s", MESSAGE, status.getStatusCode(), status.getReasonPhrase()));
- }
-
- FailedToGetVersion(Throwable cause) {
- super(MESSAGE, cause);
- }
- }
-
- private ElasticVersion getVersion() throws ElasticException {
- try {
- Response response = client.performRequest(new Request("GET", "/"));
- StatusLine statusLine = response.getStatusLine();
- if (statusLine.getStatusCode() != HttpStatus.SC_OK) {
- throw new FailedToGetVersion(statusLine);
- }
- String version =
- new JsonParser()
- .parse(AbstractElasticIndex.getContent(response))
- .getAsJsonObject()
- .get("version")
- .getAsJsonObject()
- .get("number")
- .getAsString();
- logger.atInfo().log("Connected to Elasticsearch version %s", version);
- return ElasticVersion.forVersion(version);
- } catch (IOException e) {
- throw new FailedToGetVersion(e);
- }
- }
-
- private RestClient build() {
- RestClientBuilder builder = RestClient.builder(cfg.getHosts());
- setConfiguredTimeouts(builder);
- setConfiguredCredentialsIfAny(builder);
- return builder.build();
- }
-
- private void setConfiguredTimeouts(RestClientBuilder builder) {
- builder.setRequestConfigCallback(
- (RequestConfig.Builder requestConfigBuilder) ->
- requestConfigBuilder
- .setConnectTimeout(cfg.connectTimeout)
- .setSocketTimeout(cfg.socketTimeout));
- }
-
- private void setConfiguredCredentialsIfAny(RestClientBuilder builder) {
- String username = cfg.username;
- String password = cfg.password;
- if (username != null && password != null) {
- CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
- credentialsProvider.setCredentials(
- AuthScope.ANY, new UsernamePasswordCredentials(username, password));
- builder.setHttpClientConfigCallback(
- (HttpAsyncClientBuilder httpClientBuilder) ->
- httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticSetting.java b/java/com/google/gerrit/elasticsearch/ElasticSetting.java
deleted file mode 100644
index 7ec0566..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticSetting.java
+++ /dev/null
@@ -1,97 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.collect.ImmutableMap;
-import java.util.Map;
-
-class ElasticSetting {
- /** The custom char mappings of "." to " " and "_" to " " in the form of UTF-8 */
- private static final ImmutableMap<String, String> CUSTOM_CHAR_MAPPING =
- ImmutableMap.of("\\u002E", "\\u0020", "\\u005F", "\\u0020");
-
- static SettingProperties createSetting(ElasticConfiguration config) {
- return new ElasticSetting.Builder().addCharFilter().addAnalyzer().build(config);
- }
-
- static class Builder {
- private final ImmutableMap.Builder<String, FieldProperties> fields =
- new ImmutableMap.Builder<>();
-
- SettingProperties build(ElasticConfiguration config) {
- SettingProperties properties = new SettingProperties();
- properties.analysis = fields.build();
- properties.numberOfShards = config.getNumberOfShards();
- properties.numberOfReplicas = config.numberOfReplicas;
- properties.maxResultWindow = config.maxResultWindow;
- return properties;
- }
-
- Builder addCharFilter() {
- FieldProperties charMapping = new FieldProperties("mapping");
- charMapping.mappings = getCustomCharMappings(CUSTOM_CHAR_MAPPING);
-
- FieldProperties charFilter = new FieldProperties();
- charFilter.customMapping = charMapping;
- fields.put("char_filter", charFilter);
- return this;
- }
-
- Builder addAnalyzer() {
- FieldProperties customAnalyzer = new FieldProperties("custom");
- customAnalyzer.tokenizer = "standard";
- customAnalyzer.charFilter = new String[] {"custom_mapping"};
- customAnalyzer.filter = new String[] {"lowercase"};
-
- FieldProperties analyzer = new FieldProperties();
- analyzer.customWithCharFilter = customAnalyzer;
- fields.put("analyzer", analyzer);
- return this;
- }
-
- private static String[] getCustomCharMappings(ImmutableMap<String, String> map) {
- int mappingIndex = 0;
- int numOfMappings = map.size();
- String[] mapping = new String[numOfMappings];
- for (Map.Entry<String, String> e : map.entrySet()) {
- mapping[mappingIndex++] = e.getKey() + "=>" + e.getValue();
- }
- return mapping;
- }
- }
-
- static class SettingProperties {
- Map<String, FieldProperties> analysis;
- Integer numberOfShards;
- Integer numberOfReplicas;
- Integer maxResultWindow;
- }
-
- static class FieldProperties {
- String tokenizer;
- String type;
- String[] charFilter;
- String[] filter;
- String[] mappings;
- FieldProperties customMapping;
- FieldProperties customWithCharFilter;
-
- FieldProperties() {}
-
- FieldProperties(String type) {
- this.type = type;
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticStoredValue.java b/java/com/google/gerrit/elasticsearch/ElasticStoredValue.java
deleted file mode 100644
index a02a715..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticStoredValue.java
+++ /dev/null
@@ -1,86 +0,0 @@
-// Copyright (C) 2021 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static com.google.common.collect.ImmutableList.toImmutableList;
-
-import com.google.gerrit.index.StoredValue;
-import com.google.gson.JsonElement;
-import java.sql.Timestamp;
-import java.time.Instant;
-import java.time.format.DateTimeFormatter;
-import java.util.stream.StreamSupport;
-
-/** Bridge to recover fields from the elastic index. */
-public class ElasticStoredValue implements StoredValue {
- private final JsonElement field;
-
- ElasticStoredValue(JsonElement field) {
- this.field = field;
- }
-
- @Override
- public String asString() {
- return field.getAsString();
- }
-
- @Override
- public Iterable<String> asStrings() {
- return StreamSupport.stream(field.getAsJsonArray().spliterator(), false)
- .map(f -> f.getAsString())
- .collect(toImmutableList());
- }
-
- @Override
- public Integer asInteger() {
- return field.getAsInt();
- }
-
- @Override
- public Iterable<Integer> asIntegers() {
- return StreamSupport.stream(field.getAsJsonArray().spliterator(), false)
- .map(f -> f.getAsInt())
- .collect(toImmutableList());
- }
-
- @Override
- public Long asLong() {
- return field.getAsLong();
- }
-
- @Override
- public Iterable<Long> asLongs() {
- return StreamSupport.stream(field.getAsJsonArray().spliterator(), false)
- .map(f -> f.getAsLong())
- .collect(toImmutableList());
- }
-
- @Override
- public Timestamp asTimestamp() {
- return Timestamp.from(Instant.from(DateTimeFormatter.ISO_INSTANT.parse(field.getAsString())));
- }
-
- @Override
- public byte[] asByteArray() {
- return AbstractElasticIndex.decodeBase64(field.getAsString());
- }
-
- @Override
- public Iterable<byte[]> asByteArrays() {
- return StreamSupport.stream(field.getAsJsonArray().spliterator(), false)
- .map(f -> AbstractElasticIndex.decodeBase64(f.getAsString()))
- .collect(toImmutableList());
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/ElasticVersion.java b/java/com/google/gerrit/elasticsearch/ElasticVersion.java
deleted file mode 100644
index b5bf44b..0000000
--- a/java/com/google/gerrit/elasticsearch/ElasticVersion.java
+++ /dev/null
@@ -1,66 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.common.base.Joiner;
-import java.util.regex.Pattern;
-
-public enum ElasticVersion {
- V7_6("7.6.*"),
- V7_7("7.7.*"),
- V7_8("7.8.*");
-
- private final String version;
- private final Pattern pattern;
-
- ElasticVersion(String version) {
- this.version = version;
- this.pattern = Pattern.compile(version);
- }
-
- public static class UnsupportedVersion extends ElasticException {
- private static final long serialVersionUID = 1L;
-
- UnsupportedVersion(String version) {
- super(
- String.format(
- "Unsupported version: [%s]. Supported versions: %s", version, supportedVersions()));
- }
- }
-
- /**
- * Convert a version String to an ElasticVersion if supported.
- *
- * @param version for which to return an ElasticVersion
- * @return the corresponding ElasticVersion if supported
- */
- public static ElasticVersion forVersion(String version) {
- for (ElasticVersion value : ElasticVersion.values()) {
- if (value.pattern.matcher(version).matches()) {
- return value;
- }
- }
- throw new UnsupportedVersion(version);
- }
-
- public static String supportedVersions() {
- return Joiner.on(", ").join(ElasticVersion.values());
- }
-
- @Override
- public String toString() {
- return version;
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/BoolQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/BoolQueryBuilder.java
deleted file mode 100644
index a204919..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/BoolQueryBuilder.java
+++ /dev/null
@@ -1,89 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
-
-/**
- * A Query that matches documents matching boolean combinations of other queries.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.BoolQueryBuilder.
- */
-public class BoolQueryBuilder extends QueryBuilder {
-
- private final List<QueryBuilder> mustClauses = new ArrayList<>();
-
- private final List<QueryBuilder> mustNotClauses = new ArrayList<>();
-
- private final List<QueryBuilder> filterClauses = new ArrayList<>();
-
- private final List<QueryBuilder> shouldClauses = new ArrayList<>();
-
- /**
- * Adds a query that <b>must</b> appear in the matching documents and will contribute to scoring.
- */
- public BoolQueryBuilder must(QueryBuilder queryBuilder) {
- mustClauses.add(queryBuilder);
- return this;
- }
-
- /**
- * Adds a query that <b>must not</b> appear in the matching documents and will not contribute to
- * scoring.
- */
- public BoolQueryBuilder mustNot(QueryBuilder queryBuilder) {
- mustNotClauses.add(queryBuilder);
- return this;
- }
-
- /**
- * Adds a query that <i>should</i> appear in the matching documents. For a boolean query with no
- * <tt>MUST</tt> clauses one or more <code>SHOULD</code> clauses must match a document for the
- * BooleanQuery to match.
- */
- public BoolQueryBuilder should(QueryBuilder queryBuilder) {
- shouldClauses.add(queryBuilder);
- return this;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("bool");
- doXArrayContent("must", mustClauses, builder);
- doXArrayContent("filter", filterClauses, builder);
- doXArrayContent("must_not", mustNotClauses, builder);
- doXArrayContent("should", shouldClauses, builder);
- builder.endObject();
- }
-
- private void doXArrayContent(String field, List<QueryBuilder> clauses, XContentBuilder builder)
- throws IOException {
- if (clauses.isEmpty()) {
- return;
- }
- if (clauses.size() == 1) {
- builder.field(field);
- clauses.get(0).toXContent(builder);
- } else {
- builder.startArray(field);
- for (QueryBuilder clause : clauses) {
- clause.toXContent(builder);
- }
- builder.endArray();
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/ExistsQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/ExistsQueryBuilder.java
deleted file mode 100644
index 1b058d7..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/ExistsQueryBuilder.java
+++ /dev/null
@@ -1,38 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/**
- * Constructs a query that only match on documents that the field has a value in them.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.ExistsQueryBuilder.
- */
-class ExistsQueryBuilder extends QueryBuilder {
-
- private final String name;
-
- ExistsQueryBuilder(String name) {
- this.name = name;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("exists");
- builder.field("field", name);
- builder.endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/MatchAllQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/MatchAllQueryBuilder.java
deleted file mode 100644
index a3b303c..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/MatchAllQueryBuilder.java
+++ /dev/null
@@ -1,31 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/**
- * A query that matches on all documents.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.MatchAllQueryBuilder.
- */
-class MatchAllQueryBuilder extends QueryBuilder {
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("match_all");
- builder.endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/MatchQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/MatchQueryBuilder.java
deleted file mode 100644
index c0becd1..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/MatchQueryBuilder.java
+++ /dev/null
@@ -1,62 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-import java.util.Locale;
-
-/**
- * Match query is a query that analyzes the text and constructs a query as the result of the
- * analysis. It can construct different queries based on the type provided.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.MatchQueryBuilder.
- */
-class MatchQueryBuilder extends QueryBuilder {
-
- enum Type {
- /** The text is analyzed and used as a phrase query. */
- MATCH_PHRASE,
- /** The text is analyzed and used in a phrase query, with the last term acting as a prefix. */
- MATCH_PHRASE_PREFIX;
-
- @Override
- public String toString() {
- return name().toLowerCase(Locale.US);
- }
- }
-
- private final String name;
-
- private final Object text;
-
- private Type type;
-
- /** Constructs a new text query. */
- MatchQueryBuilder(String name, Object text) {
- this.name = name;
- this.text = text;
- }
-
- /** Sets the type of the text query. */
- MatchQueryBuilder type(Type type) {
- this.type = type;
- return this;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject(type.toString()).field(name, text).endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/QueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/QueryBuilder.java
deleted file mode 100644
index d6f154e..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/QueryBuilder.java
+++ /dev/null
@@ -1,31 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/** A trimmed down version of org.elasticsearch.index.query.QueryBuilder. */
-public abstract class QueryBuilder {
-
- protected QueryBuilder() {}
-
- protected void toXContent(XContentBuilder builder) throws IOException {
- builder.startObject();
- doXContent(builder);
- builder.endObject();
- }
-
- protected abstract void doXContent(XContentBuilder builder) throws IOException;
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/QueryBuilders.java b/java/com/google/gerrit/elasticsearch/builders/QueryBuilders.java
deleted file mode 100644
index 940146f..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/QueryBuilders.java
+++ /dev/null
@@ -1,103 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-/**
- * A static factory for simple "import static" usage.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.QueryBuilders.
- */
-public abstract class QueryBuilders {
-
- /** A query that match on all documents. */
- public static MatchAllQueryBuilder matchAllQuery() {
- return new MatchAllQueryBuilder();
- }
-
- /**
- * Creates a text query with type "PHRASE" for the provided field name and text.
- *
- * @param name The field name.
- * @param text The query text (to be analyzed).
- */
- public static MatchQueryBuilder matchPhraseQuery(String name, Object text) {
- return new MatchQueryBuilder(name, text).type(MatchQueryBuilder.Type.MATCH_PHRASE);
- }
-
- /**
- * Creates a match query with type "PHRASE_PREFIX" for the provided field name and text.
- *
- * @param name The field name.
- * @param text The query text (to be analyzed).
- */
- public static MatchQueryBuilder matchPhrasePrefixQuery(String name, Object text) {
- return new MatchQueryBuilder(name, text).type(MatchQueryBuilder.Type.MATCH_PHRASE_PREFIX);
- }
-
- /**
- * A Query that matches documents containing a term.
- *
- * @param name The name of the field
- * @param value The value of the term
- */
- public static TermQueryBuilder termQuery(String name, String value) {
- return new TermQueryBuilder(name, value);
- }
-
- /**
- * A Query that matches documents containing a term.
- *
- * @param name The name of the field
- * @param value The value of the term
- */
- public static TermQueryBuilder termQuery(String name, int value) {
- return new TermQueryBuilder(name, value);
- }
-
- /**
- * A Query that matches documents within an range of terms.
- *
- * @param name The field name
- */
- public static RangeQueryBuilder rangeQuery(String name) {
- return new RangeQueryBuilder(name);
- }
-
- /**
- * A Query that matches documents containing terms with a specified regular expression.
- *
- * @param name The name of the field
- * @param regexp The regular expression
- */
- public static RegexpQueryBuilder regexpQuery(String name, String regexp) {
- return new RegexpQueryBuilder(name, regexp);
- }
-
- /** A Query that matches documents matching boolean combinations of other queries. */
- public static BoolQueryBuilder boolQuery() {
- return new BoolQueryBuilder();
- }
-
- /**
- * A filter to filter only documents where a field exists in them.
- *
- * @param name The name of the field
- */
- public static ExistsQueryBuilder existsQuery(String name) {
- return new ExistsQueryBuilder(name);
- }
-
- private QueryBuilders() {}
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/QuerySourceBuilder.java b/java/com/google/gerrit/elasticsearch/builders/QuerySourceBuilder.java
deleted file mode 100644
index 1cb5c82..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/QuerySourceBuilder.java
+++ /dev/null
@@ -1,32 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/** A trimmed down and modified version of org.elasticsearch.action.support.QuerySourceBuilder. */
-class QuerySourceBuilder {
-
- private final QueryBuilder queryBuilder;
-
- QuerySourceBuilder(QueryBuilder queryBuilder) {
- this.queryBuilder = queryBuilder;
- }
-
- void innerToXContent(XContentBuilder builder) throws IOException {
- builder.field("query");
- queryBuilder.toXContent(builder);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/RangeQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/RangeQueryBuilder.java
deleted file mode 100644
index 32dbc0e..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/RangeQueryBuilder.java
+++ /dev/null
@@ -1,89 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/**
- * A Query that matches documents within an range of terms.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.RangeQueryBuilder.
- */
-public class RangeQueryBuilder extends QueryBuilder {
-
- private final String name;
- private Object from;
- private Object to;
- private boolean includeLower = true;
- private boolean includeUpper = true;
-
- /**
- * A Query that matches documents within an range of terms.
- *
- * @param name The field name
- */
- RangeQueryBuilder(String name) {
- this.name = name;
- }
-
- /** The from part of the range query. Null indicates unbounded. */
- public RangeQueryBuilder gt(Object from) {
- this.from = from;
- this.includeLower = false;
- return this;
- }
-
- /** The from part of the range query. Null indicates unbounded. */
- public RangeQueryBuilder gte(Object from) {
- this.from = from;
- this.includeLower = true;
- return this;
- }
-
- /** The from part of the range query. Null indicates unbounded. */
- public RangeQueryBuilder gte(int from) {
- this.from = from;
- this.includeLower = true;
- return this;
- }
-
- /** The to part of the range query. Null indicates unbounded. */
- public RangeQueryBuilder lte(Object to) {
- this.to = to;
- this.includeUpper = true;
- return this;
- }
-
- /** The to part of the range query. Null indicates unbounded. */
- public RangeQueryBuilder lte(int to) {
- this.to = to;
- this.includeUpper = true;
- return this;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("range");
- builder.startObject(name);
-
- builder.field("from", from);
- builder.field("to", to);
- builder.field("include_lower", includeLower);
- builder.field("include_upper", includeUpper);
-
- builder.endObject();
- builder.endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/RegexpQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/RegexpQueryBuilder.java
deleted file mode 100644
index b81ec20..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/RegexpQueryBuilder.java
+++ /dev/null
@@ -1,51 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/**
- * A Query that does fuzzy matching for a specific value.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.RegexpQueryBuilder.
- */
-class RegexpQueryBuilder extends QueryBuilder {
-
- private final String name;
- private final String regexp;
-
- /**
- * Constructs a new term query.
- *
- * @param name The name of the field
- * @param regexp The regular expression
- */
- RegexpQueryBuilder(String name, String regexp) {
- this.name = name;
- this.regexp = regexp;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("regexp");
- builder.startObject(name);
-
- builder.field("value", regexp);
- builder.field("flags_value", 65535);
-
- builder.endObject();
- builder.endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/SearchSourceBuilder.java b/java/com/google/gerrit/elasticsearch/builders/SearchSourceBuilder.java
deleted file mode 100644
index 35cbea9..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/SearchSourceBuilder.java
+++ /dev/null
@@ -1,112 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import com.google.gerrit.elasticsearch.ElasticQueryAdapter;
-import java.io.IOException;
-import java.util.List;
-
-/**
- * A search source builder allowing to easily build search source.
- *
- * <p>A trimmed down and modified version of org.elasticsearch.search.builder.SearchSourceBuilder.
- */
-public class SearchSourceBuilder {
- private final ElasticQueryAdapter adapter;
-
- private QuerySourceBuilder querySourceBuilder;
-
- private int from = -1;
-
- private int size = -1;
-
- private List<String> fieldNames;
-
- /** Constructs a new search source builder. */
- public SearchSourceBuilder(ElasticQueryAdapter adapter) {
- this.adapter = adapter;
- }
-
- /** Constructs a new search source builder with a search query. */
- public SearchSourceBuilder query(QueryBuilder query) {
- if (this.querySourceBuilder == null) {
- this.querySourceBuilder = new QuerySourceBuilder(query);
- }
- return this;
- }
-
- /** From index to start the search from. Defaults to <tt>0</tt>. */
- public SearchSourceBuilder from(int from) {
- this.from = from;
- return this;
- }
-
- /** The number of search hits to return. Defaults to <tt>10</tt>. */
- public SearchSourceBuilder size(int size) {
- this.size = size;
- return this;
- }
-
- /**
- * Sets the fields to load and return as part of the search request. If none are specified, the
- * source of the document will be returned.
- */
- public SearchSourceBuilder fields(List<String> fields) {
- this.fieldNames = fields;
- return this;
- }
-
- @Override
- public final String toString() {
- try {
- XContentBuilder builder = new XContentBuilder();
- toXContent(builder);
- return builder.string();
- } catch (IOException ioe) {
- return "";
- }
- }
-
- private void toXContent(XContentBuilder builder) throws IOException {
- builder.startObject();
- innerToXContent(builder);
- builder.endObject();
- }
-
- private void innerToXContent(XContentBuilder builder) throws IOException {
- if (from != -1) {
- builder.field("from", from);
- }
- if (size != -1) {
- builder.field("size", size);
- }
-
- if (querySourceBuilder != null) {
- querySourceBuilder.innerToXContent(builder);
- }
-
- if (fieldNames != null) {
- if (fieldNames.size() == 1) {
- builder.field(adapter.searchFilteringName(), fieldNames.get(0));
- } else {
- builder.startArray(adapter.searchFilteringName());
- for (String fieldName : fieldNames) {
- builder.value(fieldName);
- }
- builder.endArray();
- }
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/TermQueryBuilder.java b/java/com/google/gerrit/elasticsearch/builders/TermQueryBuilder.java
deleted file mode 100644
index 2b407c6..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/TermQueryBuilder.java
+++ /dev/null
@@ -1,67 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import java.io.IOException;
-
-/**
- * A Query that matches documents containing a term.
- *
- * <p>A trimmed down version of org.elasticsearch.index.query.TermQueryBuilder.
- */
-class TermQueryBuilder extends QueryBuilder {
-
- private final String name;
-
- private final Object value;
-
- /**
- * Constructs a new term query.
- *
- * @param name The name of the field
- * @param value The value of the term
- */
- TermQueryBuilder(String name, String value) {
- this(name, (Object) value);
- }
-
- /**
- * Constructs a new term query.
- *
- * @param name The name of the field
- * @param value The value of the term
- */
- TermQueryBuilder(String name, int value) {
- this(name, (Object) value);
- }
-
- /**
- * Constructs a new term query.
- *
- * @param name The name of the field
- * @param value The value of the term
- */
- private TermQueryBuilder(String name, Object value) {
- this.name = name;
- this.value = value;
- }
-
- @Override
- protected void doXContent(XContentBuilder builder) throws IOException {
- builder.startObject("term");
- builder.field(name, value);
- builder.endObject();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/builders/XContentBuilder.java b/java/com/google/gerrit/elasticsearch/builders/XContentBuilder.java
deleted file mode 100644
index 9c44583..0000000
--- a/java/com/google/gerrit/elasticsearch/builders/XContentBuilder.java
+++ /dev/null
@@ -1,168 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project, 2009-2015 Elasticsearch
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.builders;
-
-import static java.nio.charset.StandardCharsets.UTF_8;
-import static java.time.format.DateTimeFormatter.ISO_INSTANT;
-
-import com.fasterxml.jackson.core.JsonEncoding;
-import com.fasterxml.jackson.core.JsonFactory;
-import com.fasterxml.jackson.core.JsonGenerator;
-import com.fasterxml.jackson.core.json.JsonReadFeature;
-import com.fasterxml.jackson.core.json.JsonWriteFeature;
-import java.io.ByteArrayOutputStream;
-import java.io.Closeable;
-import java.io.IOException;
-import java.util.Date;
-
-/** A trimmed down and modified version of org.elasticsearch.common.xcontent.XContentBuilder. */
-public final class XContentBuilder implements Closeable {
-
- private final JsonGenerator generator;
-
- private final ByteArrayOutputStream bos = new ByteArrayOutputStream();
-
- /**
- * Constructs a new builder. Make sure to call {@link #close()} when the builder is done with.
- * Inspired from org.elasticsearch.common.xcontent.json.JsonXContent static block.
- */
- public XContentBuilder() throws IOException {
- this.generator =
- JsonFactory.builder()
- .configure(JsonReadFeature.ALLOW_UNQUOTED_FIELD_NAMES, true)
- .configure(JsonWriteFeature.QUOTE_FIELD_NAMES, true)
- .configure(JsonReadFeature.ALLOW_JAVA_COMMENTS, true)
- .configure(JsonFactory.Feature.FAIL_ON_SYMBOL_HASH_OVERFLOW, false)
- .build()
- .createGenerator(bos, JsonEncoding.UTF8);
- }
-
- public XContentBuilder startObject(String name) throws IOException {
- field(name);
- startObject();
- return this;
- }
-
- public XContentBuilder startObject() throws IOException {
- generator.writeStartObject();
- return this;
- }
-
- public XContentBuilder endObject() throws IOException {
- generator.writeEndObject();
- return this;
- }
-
- public void startArray(String name) throws IOException {
- field(name);
- startArray();
- }
-
- private void startArray() throws IOException {
- generator.writeStartArray();
- }
-
- public void endArray() throws IOException {
- generator.writeEndArray();
- }
-
- public XContentBuilder field(String name) throws IOException {
- generator.writeFieldName(name);
- return this;
- }
-
- public XContentBuilder field(String name, String value) throws IOException {
- field(name);
- generator.writeString(value);
- return this;
- }
-
- public XContentBuilder field(String name, int value) throws IOException {
- field(name);
- generator.writeNumber(value);
- return this;
- }
-
- public XContentBuilder field(String name, Iterable<?> value) throws IOException {
- startArray(name);
- for (Object o : value) {
- value(o);
- }
- endArray();
- return this;
- }
-
- public XContentBuilder field(String name, Object value) throws IOException {
- field(name);
- writeValue(value);
- return this;
- }
-
- public XContentBuilder value(Object value) throws IOException {
- writeValue(value);
- return this;
- }
-
- public XContentBuilder field(String name, boolean value) throws IOException {
- field(name);
- generator.writeBoolean(value);
- return this;
- }
-
- public XContentBuilder value(String value) throws IOException {
- generator.writeString(value);
- return this;
- }
-
- @Override
- public void close() {
- try {
- generator.close();
- } catch (IOException e) {
- // ignore
- }
- }
-
- /** Returns a string representation of the builder (only applicable for text based xcontent). */
- public String string() {
- close();
- byte[] bytesArray = bos.toByteArray();
- return new String(bytesArray, UTF_8);
- }
-
- private void writeValue(Object value) throws IOException {
- if (value == null) {
- generator.writeNull();
- return;
- }
- Class<?> type = value.getClass();
- if (type == String.class) {
- generator.writeString((String) value);
- } else if (type == Integer.class) {
- generator.writeNumber(((Integer) value));
- } else if (type == byte[].class) {
- generator.writeBinary((byte[]) value);
- } else if (value instanceof Date) {
- generator.writeString(ISO_INSTANT.format(((Date) value).toInstant()));
- } else {
- // if this is a "value" object, like enum, DistanceUnit, ..., just toString it
- // yea, it can be misleading when toString a Java class, but really, jackson should be used in
- // that case
- generator.writeString(value.toString());
- // throw new ElasticsearchIllegalArgumentException("type not supported for generic value
- // conversion: " + type);
- }
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/bulk/ActionRequest.java b/java/com/google/gerrit/elasticsearch/bulk/ActionRequest.java
deleted file mode 100644
index 16b821c..0000000
--- a/java/com/google/gerrit/elasticsearch/bulk/ActionRequest.java
+++ /dev/null
@@ -1,41 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.bulk;
-
-import com.google.gson.JsonObject;
-
-abstract class ActionRequest extends BulkRequest {
-
- private final String action;
- private final String id;
- private final String index;
-
- protected ActionRequest(String action, String id, String index) {
- this.action = action;
- this.id = id;
- this.index = index;
- }
-
- @Override
- protected String getRequest() {
- JsonObject properties = new JsonObject();
- properties.addProperty("_id", id);
- properties.addProperty("_index", index);
-
- JsonObject jsonAction = new JsonObject();
- jsonAction.add(action, properties);
- return jsonAction.toString() + System.lineSeparator();
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/bulk/BulkRequest.java b/java/com/google/gerrit/elasticsearch/bulk/BulkRequest.java
deleted file mode 100644
index be5ad8d..0000000
--- a/java/com/google/gerrit/elasticsearch/bulk/BulkRequest.java
+++ /dev/null
@@ -1,43 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.bulk;
-
-import java.util.ArrayList;
-import java.util.List;
-
-public abstract class BulkRequest {
-
- private final List<BulkRequest> requests = new ArrayList<>();
-
- protected BulkRequest() {
- add(this);
- }
-
- public BulkRequest add(BulkRequest request) {
- requests.add(request);
- return this;
- }
-
- @Override
- public String toString() {
- StringBuilder builder = new StringBuilder();
- for (BulkRequest request : requests) {
- builder.append(request.getRequest());
- }
- return builder.toString();
- }
-
- protected abstract String getRequest();
-}
diff --git a/java/com/google/gerrit/elasticsearch/bulk/IndexRequest.java b/java/com/google/gerrit/elasticsearch/bulk/IndexRequest.java
deleted file mode 100644
index d90b80f..0000000
--- a/java/com/google/gerrit/elasticsearch/bulk/IndexRequest.java
+++ /dev/null
@@ -1,22 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.bulk;
-
-public class IndexRequest extends ActionRequest {
-
- public IndexRequest(String id, String index) {
- super("index", id, index);
- }
-}
diff --git a/java/com/google/gerrit/elasticsearch/bulk/UpdateRequest.java b/java/com/google/gerrit/elasticsearch/bulk/UpdateRequest.java
deleted file mode 100644
index 196b8d6..0000000
--- a/java/com/google/gerrit/elasticsearch/bulk/UpdateRequest.java
+++ /dev/null
@@ -1,63 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch.bulk;
-
-import static java.util.stream.Collectors.toList;
-
-import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.Iterables;
-import com.google.common.collect.Streams;
-import com.google.gerrit.elasticsearch.builders.XContentBuilder;
-import com.google.gerrit.index.Schema;
-import com.google.gerrit.index.Schema.Values;
-import java.io.IOException;
-
-public class UpdateRequest<V> extends BulkRequest {
-
- private final Schema<V> schema;
- private final V v;
- private final ImmutableSet<String> skipFields;
-
- public UpdateRequest(Schema<V> schema, V v, ImmutableSet<String> skipFields) {
- this.schema = schema;
- this.v = v;
- this.skipFields = skipFields;
- }
-
- @Override
- protected String getRequest() {
- try (XContentBuilder closeable = new XContentBuilder()) {
- XContentBuilder builder = closeable.startObject();
- for (Values<V> values : schema.buildFields(v, skipFields)) {
- String name = values.getField().getName();
- if (values.getField().isRepeatable()) {
- builder.field(name, Streams.stream(values.getValues()).collect(toList()));
- } else {
- Object element = Iterables.getOnlyElement(values.getValues(), "");
- if (shouldAddElement(element)) {
- builder.field(name, element);
- }
- }
- }
- return builder.endObject().string() + System.lineSeparator();
- } catch (IOException e) {
- return e.toString();
- }
- }
-
- private boolean shouldAddElement(Object element) {
- return !(element instanceof String) || !((String) element).isEmpty();
- }
-}
diff --git a/java/com/google/gerrit/extensions/api/projects/ProjectApi.java b/java/com/google/gerrit/extensions/api/projects/ProjectApi.java
index 9873995..59475a4 100644
--- a/java/com/google/gerrit/extensions/api/projects/ProjectApi.java
+++ b/java/com/google/gerrit/extensions/api/projects/ProjectApi.java
@@ -24,7 +24,10 @@
import com.google.gerrit.extensions.common.ProjectInfo;
import com.google.gerrit.extensions.restapi.NotImplementedException;
import com.google.gerrit.extensions.restapi.RestApiException;
+import java.util.Collection;
import java.util.List;
+import java.util.Map;
+import java.util.Set;
public interface ProjectApi {
ProjectApi create() throws RestApiException;
@@ -51,6 +54,9 @@
ConfigInfo config(ConfigInput in) throws RestApiException;
+ Map<String, Set<String>> commitsIn(Collection<String> commits, Collection<String> refs)
+ throws RestApiException;
+
ListRefsRequest<BranchInfo> branches();
ListRefsRequest<TagInfo> tags();
@@ -289,6 +295,12 @@
}
@Override
+ public Map<String, Set<String>> commitsIn(Collection<String> commits, Collection<String> refs)
+ throws RestApiException {
+ throw new NotImplementedException();
+ }
+
+ @Override
public void description(DescriptionInput in) throws RestApiException {
throw new NotImplementedException();
}
diff --git a/java/com/google/gerrit/extensions/restapi/PreconditionFailedException.java b/java/com/google/gerrit/extensions/restapi/PreconditionFailedException.java
index 86b821b..1b88b1a 100644
--- a/java/com/google/gerrit/extensions/restapi/PreconditionFailedException.java
+++ b/java/com/google/gerrit/extensions/restapi/PreconditionFailedException.java
@@ -25,7 +25,7 @@
/**
* @param msg message to return to the client describing the error.
- * @param cause cause of this exception.
+ * @param cause original cause of the failed precondition.
*/
public PreconditionFailedException(String msg, Throwable cause) {
super(msg, cause);
diff --git a/java/com/google/gerrit/httpd/GitOverHttpServlet.java b/java/com/google/gerrit/httpd/GitOverHttpServlet.java
index 26c9a30..7ed79c4 100644
--- a/java/com/google/gerrit/httpd/GitOverHttpServlet.java
+++ b/java/com/google/gerrit/httpd/GitOverHttpServlet.java
@@ -14,11 +14,15 @@
package com.google.gerrit.httpd;
+import static javax.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
+import static org.eclipse.jgit.http.server.GitSmartHttpTools.sendError;
+
import com.google.common.annotations.VisibleForTesting;
import com.google.common.cache.Cache;
import com.google.common.collect.ImmutableListMultimap;
import com.google.common.collect.ListMultimap;
import com.google.common.collect.Lists;
+import com.google.common.flogger.FluentLogger;
import com.google.gerrit.common.data.Capable;
import com.google.gerrit.entities.Project;
import com.google.gerrit.extensions.registration.DynamicSet;
@@ -54,6 +58,7 @@
import com.google.inject.TypeLiteral;
import com.google.inject.name.Named;
import java.io.IOException;
+import java.text.MessageFormat;
import java.time.Duration;
import java.util.Arrays;
import java.util.Collections;
@@ -70,10 +75,13 @@
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletResponseWrapper;
+import org.eclipse.jgit.errors.PackProtocolException;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.http.server.GitServlet;
import org.eclipse.jgit.http.server.GitSmartHttpTools;
+import org.eclipse.jgit.http.server.HttpServerText;
import org.eclipse.jgit.http.server.ServletUtils;
+import org.eclipse.jgit.http.server.UploadPackErrorHandler;
import org.eclipse.jgit.http.server.resolver.AsIsFileService;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Repository;
@@ -100,6 +108,23 @@
private static final String ID_CACHE = "adv_bases";
public static final String URL_REGEX;
+ public static final String GIT_COMMAND_STATUS_HEADER = "X-git-command-status";
+
+ private enum GIT_COMMAND_STATUS {
+ OK(0),
+ FAIL(-1);
+
+ private final int exitStatus;
+
+ GIT_COMMAND_STATUS(int exitStatus) {
+ this.exitStatus = exitStatus;
+ }
+
+ @Override
+ public String toString() {
+ return Integer.toString(exitStatus);
+ }
+ }
static {
StringBuilder url = new StringBuilder();
@@ -212,12 +237,14 @@
Resolver resolver,
UploadFactory upload,
UploadFilter uploadFilter,
+ GerritUploadPackErrorHandler uploadPackErrorHandler,
ReceivePackFactory<HttpServletRequest> receive,
ReceiveFilter receiveFilter) {
setRepositoryResolver(resolver);
setAsIsFileService(AsIsFileService.DISABLED);
setUploadPackFactory(upload);
+ setUploadPackErrorHandler(uploadPackErrorHandler);
addUploadPackFilter(uploadFilter);
setReceivePackFactory(receive);
@@ -456,6 +483,35 @@
public void destroy() {}
}
+ static class GerritUploadPackErrorHandler implements UploadPackErrorHandler {
+ private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+
+ @Override
+ public void upload(HttpServletRequest req, HttpServletResponse rsp, UploadPackRunnable r)
+ throws IOException {
+ rsp.setHeader(GIT_COMMAND_STATUS_HEADER, GIT_COMMAND_STATUS.FAIL.toString());
+ try {
+ r.upload();
+ rsp.setHeader(GIT_COMMAND_STATUS_HEADER, GIT_COMMAND_STATUS.OK.toString());
+ } catch (ServiceMayNotContinueException e) {
+ if (!e.isOutput() && !rsp.isCommitted()) {
+ rsp.reset();
+ sendError(req, rsp, e.getStatusCode(), e.getMessage());
+ }
+ } catch (Throwable e) {
+ logger.atSevere().withCause(e).log(
+ MessageFormat.format(
+ HttpServerText.get().internalErrorDuringUploadPack,
+ ServletUtils.getRepository(req)));
+ if (!rsp.isCommitted()) {
+ rsp.reset();
+ String msg = e instanceof PackProtocolException ? e.getMessage() : null;
+ sendError(req, rsp, SC_INTERNAL_SERVER_ERROR, msg);
+ }
+ }
+ }
+ }
+
static class ReceiveFactory implements ReceivePackFactory<HttpServletRequest> {
private final AsyncReceiveCommits.Factory factory;
private final Provider<CurrentUser> userProvider;
diff --git a/java/com/google/gerrit/httpd/init/BUILD b/java/com/google/gerrit/httpd/init/BUILD
index adfbdcc..990b5d7 100644
--- a/java/com/google/gerrit/httpd/init/BUILD
+++ b/java/com/google/gerrit/httpd/init/BUILD
@@ -6,7 +6,6 @@
visibility = ["//visibility:public"],
deps = [
"//java/com/google/gerrit/auth",
- "//java/com/google/gerrit/elasticsearch",
"//java/com/google/gerrit/extensions:api",
"//java/com/google/gerrit/gpg",
"//java/com/google/gerrit/httpd",
diff --git a/java/com/google/gerrit/httpd/init/WebAppInitializer.java b/java/com/google/gerrit/httpd/init/WebAppInitializer.java
index fbdbd90..1535c87 100644
--- a/java/com/google/gerrit/httpd/init/WebAppInitializer.java
+++ b/java/com/google/gerrit/httpd/init/WebAppInitializer.java
@@ -19,7 +19,6 @@
import com.google.common.base.Splitter;
import com.google.common.flogger.FluentLogger;
import com.google.gerrit.auth.AuthModule;
-import com.google.gerrit.elasticsearch.ElasticIndexModule;
import com.google.gerrit.extensions.client.AuthType;
import com.google.gerrit.gpg.GpgModule;
import com.google.gerrit.httpd.AllRequestFilter;
@@ -53,6 +52,7 @@
import com.google.gerrit.server.StartupChecks.StartupChecksModule;
import com.google.gerrit.server.account.AccountDeactivator.AccountDeactivatorModule;
import com.google.gerrit.server.account.InternalAccountDirectory.InternalAccountDirectoryModule;
+import com.google.gerrit.server.account.externalids.ExternalIdCaseSensitivityMigrator;
import com.google.gerrit.server.api.GerritApiModule;
import com.google.gerrit.server.api.PluginApiModule;
import com.google.gerrit.server.audit.AuditModule;
@@ -82,6 +82,7 @@
import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.index.OnlineUpgrader.OnlineUpgraderModule;
import com.google.gerrit.server.index.VersionManager;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.mail.SignedTokenEmailTokenVerifier.SignedTokenEmailTokenVerifierModule;
import com.google.gerrit.server.mail.receive.MailReceiver.MailReceiverModule;
import com.google.gerrit.server.mail.send.SmtpEmailSender.SmtpEmailSenderModule;
@@ -106,6 +107,7 @@
import com.google.gerrit.sshd.SshModule;
import com.google.gerrit.sshd.SshSessionFactoryInitializer;
import com.google.gerrit.sshd.commands.DefaultCommandModule;
+import com.google.gerrit.sshd.commands.ExternalIdCommandsModule;
import com.google.gerrit.sshd.commands.IndexCommandsModule;
import com.google.gerrit.sshd.commands.SequenceCommandsModule;
import com.google.gerrit.sshd.plugin.LfsPluginAuthCommand.LfsPluginAuthCommandModule;
@@ -290,7 +292,7 @@
modules.add(new AuthConfigModule());
return cfgInjector.createChildInjector(
ModuleOverloader.override(
- modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.DB_MODULE)));
+ modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.DB_MODULE_TYPE)));
}
private Injector createSysInjector() {
@@ -357,16 +359,15 @@
modules.add(new ChangeCleanupRunnerModule());
modules.add(new AccountDeactivatorModule());
modules.add(new DefaultProjectNameLockManagerModule());
+ modules.add(new ExternalIdCaseSensitivityMigrator.ExternalIdCaseSensitivityMigratorModule());
return dbInjector.createChildInjector(
ModuleOverloader.override(
- modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.SYS_MODULE)));
+ modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.SYS_MODULE_TYPE)));
}
private Module createIndexModule() {
if (indexType.isLucene()) {
- return LuceneIndexModule.latestVersion(false);
- } else if (indexType.isElasticsearch()) {
- return ElasticIndexModule.latestVersion(false);
+ return LuceneIndexModule.latestVersion(false, AutoFlush.ENABLED);
} else if (indexType.isFake()) {
// Use Reflection so that we can omit the fake index binary in production code. Test code does
// compile the component in.
@@ -400,6 +401,7 @@
sysInjector.getInstance(LfsPluginAuthCommandModule.class)));
modules.add(new IndexCommandsModule(sysInjector));
modules.add(new SequenceCommandsModule());
+ modules.add(new ExternalIdCommandsModule());
return sysInjector.createChildInjector(modules);
}
diff --git a/java/com/google/gerrit/httpd/plugins/PluginServletContext.java b/java/com/google/gerrit/httpd/plugins/PluginServletContext.java
index 52a9a9b..5e875d7 100644
--- a/java/com/google/gerrit/httpd/plugins/PluginServletContext.java
+++ b/java/com/google/gerrit/httpd/plugins/PluginServletContext.java
@@ -199,6 +199,11 @@
String v = Version.getVersion();
return "Gerrit Code Review/" + (v != null ? v : "dev");
}
+
+ @Override
+ public String getVirtualServerName() {
+ return null;
+ }
}
interface API {
@@ -255,5 +260,7 @@
int getMinorVersion();
String getServerInfo();
+
+ String getVirtualServerName();
}
}
diff --git a/java/com/google/gerrit/index/Index.java b/java/com/google/gerrit/index/Index.java
index ead302d..cc3117d 100644
--- a/java/com/google/gerrit/index/Index.java
+++ b/java/com/google/gerrit/index/Index.java
@@ -40,6 +40,16 @@
void close();
/**
+ * Insert a document into the index.
+ *
+ * <p>Results may not be immediately visible to searchers, but should be visible within a
+ * reasonable amount of time.
+ *
+ * @param obj document object
+ */
+ void insert(V obj);
+
+ /**
* Update a document in the index.
*
* <p>Semantically equivalent to deleting the document and reinserting it with new field values. A
diff --git a/java/com/google/gerrit/index/IndexType.java b/java/com/google/gerrit/index/IndexType.java
index 0c3a76a..75f8351 100644
--- a/java/com/google/gerrit/index/IndexType.java
+++ b/java/com/google/gerrit/index/IndexType.java
@@ -24,8 +24,7 @@
/**
* Index types supported by the secondary index.
*
- * <p>The explicitly known index types are Lucene (the default), Elasticsearch and a fake index used
- * in tests.
+ * <p>The explicitly known index types are Lucene (the default) and a fake index used in tests.
*
* <p>The third supported index type is any other type String value, deemed as custom. This is for
* configuring index types that are internal or not to be disclosed. Supporting custom index types
@@ -36,7 +35,6 @@
private static final String ENV_VAR = "GERRIT_INDEX_TYPE";
private static final String LUCENE = "lucene";
- private static final String ELASTICSEARCH = "elasticsearch";
private static final String FAKE = "fake";
private final String type;
@@ -77,17 +75,13 @@
}
public static ImmutableSet<String> getKnownTypes() {
- return ImmutableSet.of(LUCENE, ELASTICSEARCH, FAKE);
+ return ImmutableSet.of(LUCENE, FAKE);
}
public boolean isLucene() {
return type.equals(LUCENE);
}
- public boolean isElasticsearch() {
- return type.equals(ELASTICSEARCH);
- }
-
public boolean isFake() {
return type.equals(FAKE);
}
diff --git a/java/com/google/gerrit/index/testing/AbstractFakeIndex.java b/java/com/google/gerrit/index/testing/AbstractFakeIndex.java
index b727e96..6ece45a 100644
--- a/java/com/google/gerrit/index/testing/AbstractFakeIndex.java
+++ b/java/com/google/gerrit/index/testing/AbstractFakeIndex.java
@@ -236,6 +236,9 @@
}
return cd;
}
+
+ @Override
+ public void insert(ChangeData obj) {}
}
/** Fake implementation of {@link AccountIndex} where all filtering happens in-memory. */
@@ -265,6 +268,9 @@
protected Comparator<AccountState> sortingComparator() {
return Comparator.comparing(a -> a.account().id().get());
}
+
+ @Override
+ public void insert(AccountState obj) {}
}
/** Fake implementation of {@link GroupIndex} where all filtering happens in-memory. */
@@ -295,6 +301,9 @@
protected Comparator<InternalGroup> sortingComparator() {
return Comparator.comparing(g -> g.getId().get());
}
+
+ @Override
+ public void insert(InternalGroup obj) {}
}
/** Fake implementation of {@link ProjectIndex} where all filtering happens in-memory. */
@@ -324,5 +333,8 @@
protected Comparator<ProjectData> sortingComparator() {
return Comparator.comparing(p -> p.getProject().getName());
}
+
+ @Override
+ public void insert(ProjectData obj) {}
}
}
diff --git a/java/com/google/gerrit/index/testing/FakeStoredValue.java b/java/com/google/gerrit/index/testing/FakeStoredValue.java
index ca46ed1..5091068 100644
--- a/java/com/google/gerrit/index/testing/FakeStoredValue.java
+++ b/java/com/google/gerrit/index/testing/FakeStoredValue.java
@@ -21,7 +21,7 @@
public class FakeStoredValue implements StoredValue {
private final Object field;
- FakeStoredValue(Object field) {
+ public FakeStoredValue(Object field) {
this.field = field;
}
diff --git a/java/com/google/gerrit/lucene/AbstractLuceneIndex.java b/java/com/google/gerrit/lucene/AbstractLuceneIndex.java
index f6013a1..988d6fb 100644
--- a/java/com/google/gerrit/lucene/AbstractLuceneIndex.java
+++ b/java/com/google/gerrit/lucene/AbstractLuceneIndex.java
@@ -45,6 +45,7 @@
import com.google.gerrit.index.query.ResultSet;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.IndexUtils;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.logging.LoggingContextAwareExecutorService;
import com.google.gerrit.server.logging.LoggingContextAwareScheduledExecutorService;
import java.io.IOException;
@@ -105,6 +106,7 @@
private final ReferenceManager<IndexSearcher> searcherManager;
private final ControlledRealTimeReopenThread<IndexSearcher> reopenThread;
private final Set<NrtFuture> notDoneNrtFutures;
+ private final AutoFlush autoFlush;
private ScheduledExecutorService autoCommitExecutor;
@SuppressWarnings("ThreadPriorityCheck")
@@ -116,13 +118,15 @@
ImmutableSet<String> skipFields,
String subIndex,
GerritIndexWriterConfig writerConfig,
- SearcherFactory searcherFactory)
+ SearcherFactory searcherFactory,
+ AutoFlush autoFlush)
throws IOException {
this.schema = schema;
this.sitePaths = sitePaths;
this.dir = dir;
this.name = name;
this.skipFields = skipFields;
+ this.autoFlush = autoFlush;
String index = Joiner.on('_').skipNulls().join(name, subIndex);
long commitPeriod = writerConfig.getCommitWithinMs();
@@ -216,7 +220,9 @@
}
});
- reopenThread.start();
+ if (autoFlush.equals(AutoFlush.ENABLED)) {
+ reopenThread.start();
+ }
}
@Override
@@ -485,6 +491,9 @@
}
private boolean isGenAvailableNowForCurrentSearcher() {
+ if (autoFlush.equals(AutoFlush.DISABLED)) {
+ return true;
+ }
try {
return reopenThread.waitForGeneration(gen, 0);
} catch (InterruptedException e) {
diff --git a/java/com/google/gerrit/lucene/ChangeSubIndex.java b/java/com/google/gerrit/lucene/ChangeSubIndex.java
index 43daf25..475dac4 100644
--- a/java/com/google/gerrit/lucene/ChangeSubIndex.java
+++ b/java/com/google/gerrit/lucene/ChangeSubIndex.java
@@ -34,6 +34,7 @@
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.change.ChangeField;
import com.google.gerrit.server.index.change.ChangeIndex;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.query.change.ChangeData;
import java.io.IOException;
import java.nio.file.Path;
@@ -52,7 +53,8 @@
Path path,
ImmutableSet<String> skipFields,
GerritIndexWriterConfig writerConfig,
- SearcherFactory searcherFactory)
+ SearcherFactory searcherFactory,
+ AutoFlush autoFlush)
throws IOException {
this(
schema,
@@ -61,7 +63,8 @@
path.getFileName().toString(),
skipFields,
writerConfig,
- searcherFactory);
+ searcherFactory,
+ autoFlush);
}
ChangeSubIndex(
@@ -71,9 +74,24 @@
String subIndex,
ImmutableSet<String> skipFields,
GerritIndexWriterConfig writerConfig,
- SearcherFactory searcherFactory)
+ SearcherFactory searcherFactory,
+ AutoFlush autoFlush)
throws IOException {
- super(schema, sitePaths, dir, NAME, skipFields, subIndex, writerConfig, searcherFactory);
+ super(
+ schema,
+ sitePaths,
+ dir,
+ NAME,
+ skipFields,
+ subIndex,
+ writerConfig,
+ searcherFactory,
+ autoFlush);
+ }
+
+ @Override
+ public void insert(ChangeData obj) {
+ throw new UnsupportedOperationException("don't use ChangeSubIndex directly");
}
@Override
diff --git a/java/com/google/gerrit/lucene/LuceneAccountIndex.java b/java/com/google/gerrit/lucene/LuceneAccountIndex.java
index 242cffd..c4a5240 100644
--- a/java/com/google/gerrit/lucene/LuceneAccountIndex.java
+++ b/java/com/google/gerrit/lucene/LuceneAccountIndex.java
@@ -36,6 +36,7 @@
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.IndexUtils;
import com.google.gerrit.server.index.account.AccountIndex;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.assistedinject.Assisted;
@@ -94,7 +95,8 @@
@GerritServerConfig Config cfg,
SitePaths sitePaths,
Provider<AccountCache> accountCache,
- @Assisted Schema<AccountState> schema)
+ @Assisted Schema<AccountState> schema,
+ AutoFlush autoFlush)
throws IOException {
super(
schema,
@@ -104,7 +106,8 @@
ImmutableSet.of(),
null,
new GerritIndexWriterConfig(cfg, ACCOUNTS),
- new SearcherFactory());
+ new SearcherFactory(),
+ autoFlush);
this.accountCache = accountCache;
indexWriterConfig = new GerritIndexWriterConfig(cfg, ACCOUNTS);
@@ -141,6 +144,15 @@
}
@Override
+ public void insert(AccountState as) {
+ try {
+ insert(toDocument(as)).get();
+ } catch (ExecutionException | InterruptedException e) {
+ throw new StorageException(e);
+ }
+ }
+
+ @Override
public void delete(Account.Id key) {
try {
delete(idTerm(getSchema().useLegacyNumericFields(), key)).get();
diff --git a/java/com/google/gerrit/lucene/LuceneChangeIndex.java b/java/com/google/gerrit/lucene/LuceneChangeIndex.java
index ac616ca..e7d47d0 100644
--- a/java/com/google/gerrit/lucene/LuceneChangeIndex.java
+++ b/java/com/google/gerrit/lucene/LuceneChangeIndex.java
@@ -55,6 +55,7 @@
import com.google.gerrit.server.index.change.ChangeField;
import com.google.gerrit.server.index.change.ChangeIndex;
import com.google.gerrit.server.index.change.ChangeIndexRewriter;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.gerrit.server.query.change.ChangeDataSource;
import com.google.inject.Inject;
@@ -145,7 +146,8 @@
SitePaths sitePaths,
@IndexExecutor(INTERACTIVE) ListeningExecutorService executor,
ChangeData.Factory changeDataFactory,
- @Assisted Schema<ChangeData> schema)
+ @Assisted Schema<ChangeData> schema,
+ AutoFlush autoFlush)
throws IOException {
this.executor = executor;
this.changeDataFactory = changeDataFactory;
@@ -170,7 +172,8 @@
"ramOpen",
skipFields,
openConfig,
- searcherFactory);
+ searcherFactory,
+ autoFlush);
closedIndex =
new ChangeSubIndex(
schema,
@@ -179,7 +182,8 @@
"ramClosed",
skipFields,
closedConfig,
- searcherFactory);
+ searcherFactory,
+ autoFlush);
} else {
Path dir = LuceneVersionManager.getDir(sitePaths, CHANGES, schema);
openIndex =
@@ -189,7 +193,8 @@
dir.resolve(CHANGES_OPEN),
skipFields,
openConfig,
- searcherFactory);
+ searcherFactory,
+ autoFlush);
closedIndex =
new ChangeSubIndex(
schema,
@@ -197,7 +202,8 @@
dir.resolve(CHANGES_CLOSED),
skipFields,
closedConfig,
- searcherFactory);
+ searcherFactory,
+ autoFlush);
}
idField = this.schema.useLegacyNumericFields() ? LEGACY_ID : LEGACY_ID_STR;
@@ -247,6 +253,22 @@
}
@Override
+ public void insert(ChangeData cd) {
+ // toDocument is essentially static and doesn't depend on the specific
+ // sub-index, so just pick one.
+ Document doc = openIndex.toDocument(cd);
+ try {
+ if (cd.change().isNew()) {
+ openIndex.insert(doc).get();
+ } else {
+ closedIndex.insert(doc).get();
+ }
+ } catch (ExecutionException | InterruptedException e) {
+ throw new StorageException(e);
+ }
+ }
+
+ @Override
public void delete(Change.Id changeId) {
Term id = LuceneChangeIndex.idTerm(idTerm, idField, changeId);
try {
diff --git a/java/com/google/gerrit/lucene/LuceneGroupIndex.java b/java/com/google/gerrit/lucene/LuceneGroupIndex.java
index 5cad588..f7a2248 100644
--- a/java/com/google/gerrit/lucene/LuceneGroupIndex.java
+++ b/java/com/google/gerrit/lucene/LuceneGroupIndex.java
@@ -33,6 +33,7 @@
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.IndexUtils;
import com.google.gerrit.server.index.group.GroupIndex;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.assistedinject.Assisted;
@@ -84,7 +85,8 @@
@GerritServerConfig Config cfg,
SitePaths sitePaths,
Provider<GroupCache> groupCache,
- @Assisted Schema<InternalGroup> schema)
+ @Assisted Schema<InternalGroup> schema,
+ AutoFlush autoFlush)
throws IOException {
super(
schema,
@@ -94,7 +96,8 @@
ImmutableSet.of(),
null,
new GerritIndexWriterConfig(cfg, GROUPS),
- new SearcherFactory());
+ new SearcherFactory(),
+ autoFlush);
this.groupCache = groupCache;
indexWriterConfig = new GerritIndexWriterConfig(cfg, GROUPS);
@@ -122,6 +125,15 @@
}
@Override
+ public void insert(InternalGroup group) {
+ try {
+ insert(toDocument(group)).get();
+ } catch (ExecutionException | InterruptedException e) {
+ throw new StorageException(e);
+ }
+ }
+
+ @Override
public void delete(AccountGroup.UUID key) {
try {
delete(idTerm(key)).get();
diff --git a/java/com/google/gerrit/lucene/LuceneIndexModule.java b/java/com/google/gerrit/lucene/LuceneIndexModule.java
index 302a2da..3aa9c6e 100644
--- a/java/com/google/gerrit/lucene/LuceneIndexModule.java
+++ b/java/com/google/gerrit/lucene/LuceneIndexModule.java
@@ -14,39 +14,60 @@
package com.google.gerrit.lucene;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMap;
import com.google.gerrit.index.IndexConfig;
import com.google.gerrit.index.project.ProjectIndex;
+import com.google.gerrit.server.ModuleImpl;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.index.AbstractIndexModule;
import com.google.gerrit.server.index.VersionManager;
import com.google.gerrit.server.index.account.AccountIndex;
import com.google.gerrit.server.index.change.ChangeIndex;
import com.google.gerrit.server.index.group.GroupIndex;
+import com.google.gerrit.server.index.options.AutoFlush;
import java.util.Map;
import org.apache.lucene.search.BooleanQuery;
import org.eclipse.jgit.lib.Config;
+@ModuleImpl(name = AbstractIndexModule.INDEX_MODULE)
public class LuceneIndexModule extends AbstractIndexModule {
- public static LuceneIndexModule singleVersionAllLatest(int threads, boolean slave) {
- return new LuceneIndexModule(ImmutableMap.of(), threads, slave);
+ private final AutoFlush autoFlush;
+
+ public static LuceneIndexModule singleVersionAllLatest(
+ int threads, boolean slave, AutoFlush autoFlush) {
+ return new LuceneIndexModule(ImmutableMap.of(), threads, slave, autoFlush);
+ }
+
+ @VisibleForTesting
+ public static LuceneIndexModule singleVersionWithExplicitVersions(
+ Map<String, Integer> versions, int threads, boolean slave) {
+ return new LuceneIndexModule(versions, threads, slave, AutoFlush.ENABLED);
}
public static LuceneIndexModule singleVersionWithExplicitVersions(
- Map<String, Integer> versions, int threads, boolean slave) {
- return new LuceneIndexModule(versions, threads, slave);
+ Map<String, Integer> versions, int threads, boolean slave, AutoFlush autoFlush) {
+ return new LuceneIndexModule(versions, threads, slave, autoFlush);
}
- public static LuceneIndexModule latestVersion(boolean slave) {
- return new LuceneIndexModule(null, 0, slave);
+ public static LuceneIndexModule latestVersion(boolean slave, AutoFlush autoFlush) {
+ return new LuceneIndexModule(null, 0, slave, autoFlush);
}
static boolean isInMemoryTest(Config cfg) {
return cfg.getBoolean("index", "lucene", "testInmemory", false);
}
- private LuceneIndexModule(Map<String, Integer> singleVersions, int threads, boolean slave) {
+ private LuceneIndexModule(
+ Map<String, Integer> singleVersions, int threads, boolean slave, AutoFlush autoFlush) {
super(singleVersions, threads, slave);
+ this.autoFlush = autoFlush;
+ }
+
+ @Override
+ protected void configure() {
+ super.configure();
+ bind(AutoFlush.class).toInstance(autoFlush);
}
@Override
diff --git a/java/com/google/gerrit/lucene/LuceneProjectIndex.java b/java/com/google/gerrit/lucene/LuceneProjectIndex.java
index 2a418ca..11707be 100644
--- a/java/com/google/gerrit/lucene/LuceneProjectIndex.java
+++ b/java/com/google/gerrit/lucene/LuceneProjectIndex.java
@@ -32,6 +32,7 @@
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.IndexUtils;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.project.ProjectCache;
import com.google.gerrit.server.project.ProjectState;
import com.google.inject.Inject;
@@ -84,7 +85,8 @@
@GerritServerConfig Config cfg,
SitePaths sitePaths,
Provider<ProjectCache> projectCache,
- @Assisted Schema<ProjectData> schema)
+ @Assisted Schema<ProjectData> schema,
+ AutoFlush autoFlush)
throws IOException {
super(
schema,
@@ -94,7 +96,8 @@
ImmutableSet.of(),
null,
new GerritIndexWriterConfig(cfg, PROJECTS),
- new SearcherFactory());
+ new SearcherFactory(),
+ autoFlush);
this.projectCache = projectCache;
indexWriterConfig = new GerritIndexWriterConfig(cfg, PROJECTS);
@@ -122,6 +125,15 @@
}
@Override
+ public void insert(ProjectData projectState) {
+ try {
+ insert(toDocument(projectState)).get();
+ } catch (ExecutionException | InterruptedException e) {
+ throw new StorageException(e);
+ }
+ }
+
+ @Override
public void delete(Project.NameKey nameKey) {
try {
delete(idTerm(nameKey)).get();
diff --git a/java/com/google/gerrit/pgm/BUILD b/java/com/google/gerrit/pgm/BUILD
index b16c470c..d4c5a87 100644
--- a/java/com/google/gerrit/pgm/BUILD
+++ b/java/com/google/gerrit/pgm/BUILD
@@ -10,7 +10,6 @@
"//java/com/google/gerrit/auth",
"//java/com/google/gerrit/common:annotations",
"//java/com/google/gerrit/common:server",
- "//java/com/google/gerrit/elasticsearch",
"//java/com/google/gerrit/entities",
"//java/com/google/gerrit/exceptions",
"//java/com/google/gerrit/extensions:api",
diff --git a/java/com/google/gerrit/pgm/ChangeExternalIdCaseSensitivity.java b/java/com/google/gerrit/pgm/ChangeExternalIdCaseSensitivity.java
index 01c76c1..2b7f23e 100644
--- a/java/com/google/gerrit/pgm/ChangeExternalIdCaseSensitivity.java
+++ b/java/com/google/gerrit/pgm/ChangeExternalIdCaseSensitivity.java
@@ -14,11 +14,7 @@
package com.google.gerrit.pgm;
-import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_GERRIT;
-import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_USERNAME;
-
import com.google.common.flogger.FluentLogger;
-import com.google.gerrit.exceptions.DuplicateKeyException;
import com.google.gerrit.extensions.config.FactoryModule;
import com.google.gerrit.extensions.registration.DynamicMap;
import com.google.gerrit.lifecycle.LifecycleManager;
@@ -26,28 +22,23 @@
import com.google.gerrit.pgm.util.SiteProgram;
import com.google.gerrit.server.account.externalids.DisabledExternalIdCache;
import com.google.gerrit.server.account.externalids.ExternalId;
-import com.google.gerrit.server.account.externalids.ExternalIdFactory;
-import com.google.gerrit.server.account.externalids.ExternalIdKeyFactory;
-import com.google.gerrit.server.account.externalids.ExternalIdNotes;
+import com.google.gerrit.server.account.externalids.ExternalIdCaseSensitivityMigrator;
import com.google.gerrit.server.account.externalids.ExternalIdUpsertPreprocessor;
import com.google.gerrit.server.account.externalids.ExternalIds;
-import com.google.gerrit.server.config.AllUsersName;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
-import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.meta.MetaDataUpdate;
import com.google.gerrit.server.index.account.AccountSchemaDefinitions;
import com.google.gerrit.server.schema.NoteDbSchemaVersionCheck;
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Key;
-import com.google.inject.Provider;
+import com.google.inject.assistedinject.FactoryModuleBuilder;
import java.io.IOException;
import java.util.Collection;
import org.eclipse.jgit.errors.ConfigInvalidException;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.ProgressMonitor;
-import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.lib.TextProgressMonitor;
import org.eclipse.jgit.storage.file.FileBasedConfig;
import org.eclipse.jgit.util.FS;
@@ -73,12 +64,8 @@
private boolean isUserNameCaseInsensitive;
private ConsoleUI ui;
- @Inject private GitRepositoryManager repoManager;
- @Inject private AllUsersName allUsersName;
- @Inject private Provider<MetaDataUpdate.Server> metaDataUpdateServerFactory;
- @Inject private ExternalIdNotes.FactoryNoReindex externalIdNotesFactory;
@Inject private ExternalIds externalIds;
- @Inject private ExternalIdFactory externalIdFactory;
+ @Inject private ExternalIdCaseSensitivityMigrator.Factory migratorFactory;
@Override
public int run() throws Exception {
@@ -93,6 +80,9 @@
@Override
protected void configure() {
bind(GitReferenceUpdated.class).toInstance(GitReferenceUpdated.DISABLED);
+ install(
+ new FactoryModuleBuilder()
+ .build(ExternalIdCaseSensitivityMigrator.Factory.class));
factory(MetaDataUpdate.InternalFactory.class);
DynamicMap.mapOf(binder(), ExternalIdUpsertPreprocessor.class);
@@ -122,23 +112,9 @@
manager.start();
try {
- try (Repository repo = repoManager.openRepository(allUsersName)) {
- ExternalIdNotes extIdNotes = externalIdNotesFactory.load(repo);
- for (ExternalId extId : todo) {
- recomputeExternalIdNoteId(extIdNotes, extId);
- monitor.update(1);
- }
- if (!dryrun) {
- try (MetaDataUpdate metaDataUpdate =
- metaDataUpdateServerFactory.get().create(allUsersName)) {
- metaDataUpdate.setMessage(
- String.format(
- "Migration to case %ssensitive usernames",
- isUserNameCaseInsensitive ? "" : "in"));
- extIdNotes.commit(metaDataUpdate);
- }
- }
- }
+ migratorFactory
+ .create(!isUserNameCaseInsensitive, dryrun)
+ .migrate(todo, () -> monitor.update(1));
} finally {
manager.stop();
monitor.endTask();
@@ -155,28 +131,6 @@
return exitCode;
}
- private void recomputeExternalIdNoteId(ExternalIdNotes extIdNotes, ExternalId extId)
- throws DuplicateKeyException, IOException {
- if (extId.isScheme(SCHEME_GERRIT) || extId.isScheme(SCHEME_USERNAME)) {
- ExternalIdKeyFactory keyFactory =
- new ExternalIdKeyFactory(
- new ExternalIdKeyFactory.Config() {
- @Override
- public boolean isUserNameCaseInsensitive() {
- return !isUserNameCaseInsensitive;
- }
- });
- ExternalId.Key updatedKey = keyFactory.create(extId.key().scheme(), extId.key().id());
- if (!extId.key().sha1().getName().equals(updatedKey.sha1().getName())) {
- logger.atInfo().log("Converting note name of external ID: %s", extId.key());
- ExternalId updatedExtId =
- externalIdFactory.create(
- updatedKey, extId.accountId(), extId.email(), extId.password(), extId.blobId());
- extIdNotes.replace(extId, updatedExtId);
- }
- }
- }
-
private void updateGerritConfig() throws IOException, ConfigInvalidException {
logger.atInfo().log("Setting auth.userNameCaseInsensitive to true in gerrit.config.");
FileBasedConfig config =
diff --git a/java/com/google/gerrit/pgm/Daemon.java b/java/com/google/gerrit/pgm/Daemon.java
index 402d839..75891fe 100644
--- a/java/com/google/gerrit/pgm/Daemon.java
+++ b/java/com/google/gerrit/pgm/Daemon.java
@@ -23,7 +23,6 @@
import com.google.common.flogger.FluentLogger;
import com.google.gerrit.auth.AuthModule;
import com.google.gerrit.common.Nullable;
-import com.google.gerrit.elasticsearch.ElasticIndexModule;
import com.google.gerrit.extensions.client.AuthType;
import com.google.gerrit.gpg.GpgModule;
import com.google.gerrit.httpd.AllRequestFilter;
@@ -62,6 +61,7 @@
import com.google.gerrit.server.StartupChecks.StartupChecksModule;
import com.google.gerrit.server.account.AccountDeactivator.AccountDeactivatorModule;
import com.google.gerrit.server.account.InternalAccountDirectory.InternalAccountDirectoryModule;
+import com.google.gerrit.server.account.externalids.ExternalIdCaseSensitivityMigrator;
import com.google.gerrit.server.api.GerritApiModule;
import com.google.gerrit.server.api.PluginApiModule;
import com.google.gerrit.server.audit.AuditModule;
@@ -91,6 +91,7 @@
import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.index.OnlineUpgrader.OnlineUpgraderModule;
import com.google.gerrit.server.index.VersionManager;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.gerrit.server.mail.SignedTokenEmailTokenVerifier.SignedTokenEmailTokenVerifierModule;
import com.google.gerrit.server.mail.receive.MailReceiver.MailReceiverModule;
import com.google.gerrit.server.mail.send.SmtpEmailSender.SmtpEmailSenderModule;
@@ -118,6 +119,7 @@
import com.google.gerrit.sshd.SshModule;
import com.google.gerrit.sshd.SshSessionFactoryInitializer;
import com.google.gerrit.sshd.commands.DefaultCommandModule;
+import com.google.gerrit.sshd.commands.ExternalIdCommandsModule;
import com.google.gerrit.sshd.commands.IndexCommandsModule;
import com.google.gerrit.sshd.commands.SequenceCommandsModule;
import com.google.gerrit.sshd.plugin.LfsPluginAuthCommand.LfsPluginAuthCommandModule;
@@ -520,12 +522,16 @@
modules.add(new LocalMergeSuperSetComputationModule());
modules.add(new DefaultProjectNameLockManagerModule());
- List<Module> libModules = LibModuleLoader.loadModules(cfgInjector, LibModuleType.SYS_MODULE);
+ List<Module> libModules =
+ LibModuleLoader.loadModules(cfgInjector, LibModuleType.SYS_MODULE_TYPE);
+ libModules.addAll(LibModuleLoader.loadModules(cfgInjector, LibModuleType.INDEX_MODULE_TYPE));
libModules.addAll(testSysModules);
AuthConfig authConfig = cfgInjector.getInstance(AuthConfig.class);
modules.add(new AuthModule(authConfig));
+ modules.add(new ExternalIdCaseSensitivityMigrator.ExternalIdCaseSensitivityMigratorModule());
+
return cfgInjector.createChildInjector(ModuleOverloader.override(modules, libModules));
}
@@ -534,10 +540,7 @@
return indexModule;
}
if (indexType.isLucene()) {
- return LuceneIndexModule.latestVersion(replica);
- }
- if (indexType.isElasticsearch()) {
- return ElasticIndexModule.latestVersion(replica);
+ return LuceneIndexModule.latestVersion(replica, AutoFlush.ENABLED);
}
if (indexType.isFake()) {
// Use Reflection so that we can omit the fake index binary in production code. Test code does
@@ -578,6 +581,7 @@
if (!replica) {
modules.add(new IndexCommandsModule(sysInjector));
modules.add(new SequenceCommandsModule());
+ modules.add(new ExternalIdCommandsModule());
}
return sysInjector.createChildInjector(modules);
}
diff --git a/java/com/google/gerrit/pgm/Init.java b/java/com/google/gerrit/pgm/Init.java
index 4e62a0f..4c7b47b 100644
--- a/java/com/google/gerrit/pgm/Init.java
+++ b/java/com/google/gerrit/pgm/Init.java
@@ -18,6 +18,7 @@
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import com.google.gerrit.common.IoUtil;
import com.google.gerrit.common.PageLinks;
@@ -91,6 +92,9 @@
@Option(name = "--skip-download", usage = "Don't download given library")
private List<String> skippedDownloads;
+ @Option(name = "--reindex-threads", usage = "Number of threads to use for reindex after init")
+ private int reindexThreads = 1;
+
@Inject Browser browser;
private GerritIndexStatus indexStatus;
@@ -157,11 +161,13 @@
modules.add(new GerritServerConfigModule());
Guice.createInjector(modules).injectMembers(this);
if (!ReplicaUtil.isReplica(run.flags.cfg)) {
+ List<String> indicesToReindex = new ArrayList<>();
for (SchemaDefinitions<?> schemaDef : schemaDefs) {
if (!indexStatus.exists(schemaDef.getName())) {
- reindex(schemaDef);
+ indicesToReindex.add(schemaDef.getName());
}
}
+ reindex(indicesToReindex, run.flags.isNew);
}
start(run);
}
@@ -274,17 +280,23 @@
}
}
- private void reindex(SchemaDefinitions<?> schemaDef) throws Exception {
+ private void reindex(List<String> indices, boolean isNewSite) throws Exception {
+ if (indices.isEmpty()) {
+ return;
+ }
List<String> reindexArgs =
- ImmutableList.of(
- "--site-path",
- getSitePath().toString(),
- "--threads",
- Integer.toString(1),
- "--index",
- schemaDef.getName());
+ Lists.newArrayList(
+ "--site-path", getSitePath().toString(), "--threads", Integer.toString(reindexThreads));
+ for (String index : indices) {
+ reindexArgs.add("--index");
+ reindexArgs.add(index);
+ }
+ if (isNewSite) {
+ reindexArgs.add("--disable-cache-stats");
+ }
+
getConsoleUI()
- .message(String.format("Init complete, reindexing %s with:", schemaDef.getName()));
+ .message(String.format("Init complete, reindexing %s with:", String.join(",", indices)));
getConsoleUI().message(" reindex " + reindexArgs.stream().collect(joining(" ")));
Reindex reindexPgm = new Reindex();
reindexPgm.main(reindexArgs.stream().toArray(String[]::new));
diff --git a/java/com/google/gerrit/pgm/Reindex.java b/java/com/google/gerrit/pgm/Reindex.java
index 6e99007..c4e185d 100644
--- a/java/com/google/gerrit/pgm/Reindex.java
+++ b/java/com/google/gerrit/pgm/Reindex.java
@@ -17,10 +17,11 @@
import static java.util.Objects.requireNonNull;
import static java.util.stream.Collectors.toSet;
+import com.google.common.cache.Cache;
import com.google.common.collect.Sets;
import com.google.gerrit.common.Die;
-import com.google.gerrit.elasticsearch.ElasticIndexModule;
import com.google.gerrit.extensions.config.FactoryModule;
+import com.google.gerrit.extensions.registration.DynamicMap;
import com.google.gerrit.index.Index;
import com.google.gerrit.index.IndexDefinition;
import com.google.gerrit.index.IndexType;
@@ -29,16 +30,26 @@
import com.google.gerrit.lucene.LuceneIndexModule;
import com.google.gerrit.pgm.util.BatchProgramModule;
import com.google.gerrit.pgm.util.SiteProgram;
+import com.google.gerrit.server.LibModuleLoader;
+import com.google.gerrit.server.ModuleOverloader;
+import com.google.gerrit.server.cache.CacheDisplay;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gerrit.server.change.ChangeResource;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.index.change.ChangeSchemaDefinitions;
+import com.google.gerrit.server.index.options.AutoFlush;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.gerrit.server.plugins.PluginGuiceEnvironment;
import com.google.gerrit.server.util.ReplicaUtil;
+import com.google.inject.AbstractModule;
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Key;
import com.google.inject.Module;
+import com.google.inject.multibindings.OptionalBinder;
+import java.io.StringWriter;
+import java.io.Writer;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
@@ -49,13 +60,17 @@
import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+import java.util.stream.StreamSupport;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.util.io.NullOutputStream;
import org.kohsuke.args4j.Option;
public class Reindex extends SiteProgram {
- @Option(name = "--threads", usage = "Number of threads to use for indexing")
- private int threads = Runtime.getRuntime().availableProcessors();
+ @Option(
+ name = "--threads",
+ usage = "Number of threads to use for indexing. Default is index.batchThreads from config.")
+ private int threads = 0;
@Option(
name = "--changes-schema-version",
@@ -71,12 +86,20 @@
@Option(name = "--index", usage = "Only reindex specified indices")
private List<String> indices = new ArrayList<>();
+ @Option(
+ name = "--disable-cache-stats",
+ usage =
+ "Disables printing the cache statistics."
+ + "Defaults to true when reindex is run from init on a new site, false otherwise")
+ private boolean disableCacheStats;
+
private Injector dbInjector;
private Injector sysInjector;
private Injector cfgInjector;
private Config globalConfig;
@Inject private Collection<IndexDefinition<?, ?, ?>> indexDefs;
+ @Inject private DynamicMap<Cache<?, ?>> cacheMap;
@Override
public int run() throws Exception {
@@ -100,6 +123,9 @@
try {
boolean ok = list ? list() : reindex();
+ if (!disableCacheStats) {
+ printCacheStats();
+ }
return ok ? 0 : 1;
} catch (Exception e) {
throw die(e.getMessage(), e);
@@ -152,10 +178,9 @@
Module indexModule;
IndexType indexType = IndexModule.getIndexType(dbInjector);
if (indexType.isLucene()) {
- indexModule = LuceneIndexModule.singleVersionWithExplicitVersions(versions, threads, replica);
- } else if (indexType.isElasticsearch()) {
indexModule =
- ElasticIndexModule.singleVersionWithExplicitVersions(versions, threads, replica);
+ LuceneIndexModule.singleVersionWithExplicitVersions(
+ versions, threads, replica, AutoFlush.DISABLED);
} else if (indexType.isFake()) {
// Use Reflection so that we can omit the fake index binary in production code. Test code does
// compile the component in.
@@ -175,6 +200,16 @@
throw new IllegalStateException("unsupported index.type = " + indexType);
}
modules.add(indexModule);
+ modules.add(
+ new AbstractModule() {
+ @Override
+ protected void configure() {
+ super.configure();
+ OptionalBinder.newOptionalBinder(binder(), IsFirstInsertForEntry.class)
+ .setBinding()
+ .toInstance(IsFirstInsertForEntry.YES);
+ }
+ });
modules.add(new BatchProgramModule(dbInjector));
modules.add(
new FactoryModule() {
@@ -184,7 +219,9 @@
}
});
- return dbInjector.createChildInjector(modules);
+ return dbInjector.createChildInjector(
+ ModuleOverloader.override(
+ modules, LibModuleLoader.loadReindexModules(cfgInjector, versions, threads, replica)));
}
private void overrideConfig() {
@@ -222,6 +259,22 @@
System.out.format(
"Index %s in version %d is %sready\n",
def.getName(), index.getSchema().getVersion(), result.success() ? "" : "NOT ");
+
return result.success();
}
+
+ private void printCacheStats() {
+ try (Writer sw = new StringWriter()) {
+ sw.write("Cache Statistics at the end of reindexing\n");
+ new CacheDisplay(
+ sw,
+ StreamSupport.stream(cacheMap.spliterator(), false)
+ .map(e -> new CacheInfo(e.getExportName(), e.get()))
+ .collect(Collectors.toList()))
+ .displayCaches();
+ System.out.print(sw.toString());
+ } catch (Exception e) {
+ System.out.format("Error displaying the cache statistics\n" + e.getMessage());
+ }
+ }
}
diff --git a/java/com/google/gerrit/pgm/http/jetty/HttpLog.java b/java/com/google/gerrit/pgm/http/jetty/HttpLog.java
index 56523dd..e88bb88 100644
--- a/java/com/google/gerrit/pgm/http/jetty/HttpLog.java
+++ b/java/com/google/gerrit/pgm/http/jetty/HttpLog.java
@@ -14,6 +14,8 @@
package com.google.gerrit.pgm.http.jetty;
+import static com.google.gerrit.httpd.GitOverHttpServlet.GIT_COMMAND_STATUS_HEADER;
+
import com.google.common.base.Strings;
import com.google.gerrit.httpd.GetUserFilter;
import com.google.gerrit.httpd.RequestMetricsFilter;
@@ -55,6 +57,7 @@
protected static final String P_CPU_TOTAL = "Cpu-Total";
protected static final String P_CPU_USER = "Cpu-User";
protected static final String P_MEMORY = "Memory";
+ protected static final String P_COMMAND_STATUS = "Command-Status";
private final AsyncAppender async;
@@ -117,6 +120,7 @@
set(event, P_LATENCY, System.currentTimeMillis() - req.getTimeStamp());
set(event, P_REFERER, req.getHeader("Referer"));
set(event, P_USER_AGENT, req.getHeader("User-Agent"));
+ set(event, P_COMMAND_STATUS, rsp.getHeader(GIT_COMMAND_STATUS_HEADER));
RequestMetricsFilter.Context ctx =
(RequestMetricsFilter.Context) req.getAttribute(RequestMetricsFilter.METRICS_CONTEXT);
diff --git a/java/com/google/gerrit/pgm/http/jetty/HttpLogJsonLayout.java b/java/com/google/gerrit/pgm/http/jetty/HttpLogJsonLayout.java
index 5f4ec43..54c587b 100644
--- a/java/com/google/gerrit/pgm/http/jetty/HttpLogJsonLayout.java
+++ b/java/com/google/gerrit/pgm/http/jetty/HttpLogJsonLayout.java
@@ -14,6 +14,7 @@
package com.google.gerrit.pgm.http.jetty;
+import static com.google.gerrit.pgm.http.jetty.HttpLog.P_COMMAND_STATUS;
import static com.google.gerrit.pgm.http.jetty.HttpLog.P_CONTENT_LENGTH;
import static com.google.gerrit.pgm.http.jetty.HttpLog.P_CPU_TOTAL;
import static com.google.gerrit.pgm.http.jetty.HttpLog.P_CPU_USER;
@@ -56,6 +57,7 @@
public String memory;
public String referer;
public String userAgent;
+ public String commandStatus;
public HttpJsonLogEntry(LoggingEvent event) {
this.host = getMdcString(event, P_HOST);
@@ -73,6 +75,7 @@
this.memory = getMdcString(event, P_MEMORY);
this.referer = getMdcString(event, P_REFERER);
this.userAgent = getMdcString(event, P_USER_AGENT);
+ this.commandStatus = getMdcString(event, P_COMMAND_STATUS);
}
}
}
diff --git a/java/com/google/gerrit/pgm/http/jetty/HttpLogLayout.java b/java/com/google/gerrit/pgm/http/jetty/HttpLogLayout.java
index e9e6866..ddc1b5e 100644
--- a/java/com/google/gerrit/pgm/http/jetty/HttpLogLayout.java
+++ b/java/com/google/gerrit/pgm/http/jetty/HttpLogLayout.java
@@ -80,6 +80,9 @@
buf.append(' ');
opt(buf, event, HttpLog.P_MEMORY);
+ buf.append(' ');
+ dq_opt(buf, event, HttpLog.P_COMMAND_STATUS);
+
buf.append('\n');
return buf.toString();
}
diff --git a/java/com/google/gerrit/pgm/init/BUILD b/java/com/google/gerrit/pgm/init/BUILD
index 62c9526..5849711 100644
--- a/java/com/google/gerrit/pgm/init/BUILD
+++ b/java/com/google/gerrit/pgm/init/BUILD
@@ -9,7 +9,6 @@
deps = [
"//java/com/google/gerrit/common:annotations",
"//java/com/google/gerrit/common:server",
- "//java/com/google/gerrit/elasticsearch",
"//java/com/google/gerrit/entities",
"//java/com/google/gerrit/exceptions",
"//java/com/google/gerrit/extensions:api",
diff --git a/java/com/google/gerrit/pgm/init/BaseInit.java b/java/com/google/gerrit/pgm/init/BaseInit.java
index 14147a4..4592cbb 100644
--- a/java/com/google/gerrit/pgm/init/BaseInit.java
+++ b/java/com/google/gerrit/pgm/init/BaseInit.java
@@ -33,7 +33,6 @@
import com.google.gerrit.pgm.init.api.LibraryDownload;
import com.google.gerrit.pgm.init.index.IndexManagerOnInit;
import com.google.gerrit.pgm.init.index.IndexModuleOnInit;
-import com.google.gerrit.pgm.init.index.elasticsearch.ElasticIndexModuleOnInit;
import com.google.gerrit.pgm.init.index.lucene.LuceneIndexModuleOnInit;
import com.google.gerrit.pgm.util.SiteProgram;
import com.google.gerrit.server.config.GerritServerConfigModule;
@@ -416,8 +415,6 @@
IndexType indexType = IndexModule.getIndexType(dbInjector);
if (indexType.isLucene()) {
modules.add(new LuceneIndexModuleOnInit());
- } else if (indexType.isElasticsearch()) {
- modules.add(new ElasticIndexModuleOnInit());
} else if (indexType.isFake()) {
try {
Class<?> clazz = Class.forName("com.google.gerrit.index.testing.FakeIndexModuleOnInit");
diff --git a/java/com/google/gerrit/pgm/init/ExternalIdsOnInit.java b/java/com/google/gerrit/pgm/init/ExternalIdsOnInit.java
index e2a1f04..9b4d5bb 100644
--- a/java/com/google/gerrit/pgm/init/ExternalIdsOnInit.java
+++ b/java/com/google/gerrit/pgm/init/ExternalIdsOnInit.java
@@ -21,6 +21,7 @@
import com.google.gerrit.server.account.externalids.ExternalIdFactory;
import com.google.gerrit.server.account.externalids.ExternalIdNotes;
import com.google.gerrit.server.config.AllUsersName;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.meta.MetaDataUpdate;
@@ -41,17 +42,20 @@
private final SitePaths site;
private final AllUsersName allUsers;
private final ExternalIdFactory externalIdFactory;
+ private final AuthConfig authConfig;
@Inject
public ExternalIdsOnInit(
InitFlags flags,
SitePaths site,
AllUsersNameOnInitProvider allUsers,
- ExternalIdFactory externalIdFactory) {
+ ExternalIdFactory externalIdFactory,
+ AuthConfig authConfig) {
this.flags = flags;
this.site = site;
this.allUsers = new AllUsersName(allUsers.get());
this.externalIdFactory = externalIdFactory;
+ this.authConfig = authConfig;
}
public synchronized void insert(String commitMessage, Collection<ExternalId> extIds)
@@ -60,7 +64,11 @@
if (path != null) {
try (Repository allUsersRepo = new FileRepository(path)) {
ExternalIdNotes extIdNotes =
- ExternalIdNotes.loadNoCacheUpdate(allUsers, allUsersRepo, externalIdFactory);
+ ExternalIdNotes.loadNoCacheUpdate(
+ allUsers,
+ allUsersRepo,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode());
extIdNotes.insert(extIds);
try (MetaDataUpdate metaDataUpdate =
new MetaDataUpdate(GitReferenceUpdated.DISABLED, allUsers, allUsersRepo)) {
diff --git a/java/com/google/gerrit/pgm/init/InitIndex.java b/java/com/google/gerrit/pgm/init/InitIndex.java
index 83d9261..a6254fd 100644
--- a/java/com/google/gerrit/pgm/init/InitIndex.java
+++ b/java/com/google/gerrit/pgm/init/InitIndex.java
@@ -39,7 +39,6 @@
private final SitePaths site;
private final InitFlags initFlags;
private final Section gerrit;
- private final Section.Factory sections;
@Inject
InitIndex(ConsoleUI ui, Section.Factory sections, SitePaths site, InitFlags initFlags) {
@@ -48,7 +47,6 @@
this.gerrit = sections.get("gerrit", null);
this.site = site;
this.initFlags = initFlags;
- this.sections = sections;
}
@Override
@@ -58,13 +56,6 @@
new IndexType(
index.select("Type", "type", IndexType.getDefault(), IndexType.getKnownTypes()));
- if (type.isElasticsearch()) {
- Section elasticsearch = sections.get("elasticsearch", null);
- elasticsearch.string("Index Prefix", "prefix", "gerrit_");
- elasticsearch.string("Server", "server", "http://localhost:9200");
- index.string("Result window size", "maxLimit", "10000");
- }
-
if ((site.isNew || isEmptySite()) && type.isLucene()) {
for (SchemaDefinitions<?> def : IndexModule.ALL_SCHEMA_DEFS) {
IndexUtils.setReady(site, def.getName(), def.getLatest().getVersion(), true);
diff --git a/java/com/google/gerrit/pgm/init/index/IndexModuleOnInit.java b/java/com/google/gerrit/pgm/init/index/IndexModuleOnInit.java
index 80de1e5..d1d0729 100644
--- a/java/com/google/gerrit/pgm/init/index/IndexModuleOnInit.java
+++ b/java/com/google/gerrit/pgm/init/index/IndexModuleOnInit.java
@@ -50,8 +50,7 @@
@Override
protected void configure() {
- // The AccountIndex implementations (LuceneAccountIndex and
- // ElasticAccountIndex) need AccountCache only for reading from the index.
+ // The LuceneAccountIndex needs AccountCache only for reading from the index.
// On init we only want to write to the index, hence we don't need the
// account cache.
bind(AccountCache.class).toProvider(Providers.of(null));
@@ -63,8 +62,8 @@
bind(AccountIndexCollection.class);
- // The GroupIndex implementations (LuceneGroupIndex and ElasticGroupIndex)
- // need GroupCache only for reading from the index. On init we only want to
+ // The LuceneGroupIndex needs GroupCache only for reading from the index. On init we only want
+ // to
// write to the index, hence we don't need the group cache.
bind(GroupCache.class).toProvider(Providers.of(null));
diff --git a/java/com/google/gerrit/pgm/init/index/elasticsearch/ElasticIndexModuleOnInit.java b/java/com/google/gerrit/pgm/init/index/elasticsearch/ElasticIndexModuleOnInit.java
deleted file mode 100644
index f086ab1..0000000
--- a/java/com/google/gerrit/pgm/init/index/elasticsearch/ElasticIndexModuleOnInit.java
+++ /dev/null
@@ -1,41 +0,0 @@
-// Copyright (C) 2017 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.pgm.init.index.elasticsearch;
-
-import com.google.gerrit.elasticsearch.ElasticAccountIndex;
-import com.google.gerrit.elasticsearch.ElasticGroupIndex;
-import com.google.gerrit.pgm.init.index.IndexModuleOnInit;
-import com.google.gerrit.server.index.account.AccountIndex;
-import com.google.gerrit.server.index.group.GroupIndex;
-import com.google.inject.AbstractModule;
-import com.google.inject.assistedinject.FactoryModuleBuilder;
-
-public class ElasticIndexModuleOnInit extends AbstractModule {
-
- @Override
- protected void configure() {
- install(
- new FactoryModuleBuilder()
- .implement(AccountIndex.class, ElasticAccountIndex.class)
- .build(AccountIndex.Factory.class));
-
- install(
- new FactoryModuleBuilder()
- .implement(GroupIndex.class, ElasticGroupIndex.class)
- .build(GroupIndex.Factory.class));
-
- install(new IndexModuleOnInit());
- }
-}
diff --git a/java/com/google/gerrit/pgm/init/index/lucene/LuceneIndexModuleOnInit.java b/java/com/google/gerrit/pgm/init/index/lucene/LuceneIndexModuleOnInit.java
index 12a44dc..078e648 100644
--- a/java/com/google/gerrit/pgm/init/index/lucene/LuceneIndexModuleOnInit.java
+++ b/java/com/google/gerrit/pgm/init/index/lucene/LuceneIndexModuleOnInit.java
@@ -19,6 +19,7 @@
import com.google.gerrit.pgm.init.index.IndexModuleOnInit;
import com.google.gerrit.server.index.account.AccountIndex;
import com.google.gerrit.server.index.group.GroupIndex;
+import com.google.gerrit.server.index.options.AutoFlush;
import com.google.inject.AbstractModule;
import com.google.inject.assistedinject.FactoryModuleBuilder;
@@ -36,5 +37,7 @@
.build(GroupIndex.Factory.class));
install(new IndexModuleOnInit());
+
+ bind(AutoFlush.class).toInstance(AutoFlush.DISABLED);
}
}
diff --git a/java/com/google/gerrit/pgm/util/BatchProgramModule.java b/java/com/google/gerrit/pgm/util/BatchProgramModule.java
index ce433d7..ae9e72f 100644
--- a/java/com/google/gerrit/pgm/util/BatchProgramModule.java
+++ b/java/com/google/gerrit/pgm/util/BatchProgramModule.java
@@ -133,7 +133,7 @@
// We're just running through each change
// once, so don't worry about cache removal.
bind(new TypeLiteral<DynamicSet<CacheRemovalListener>>() {}).toInstance(DynamicSet.emptySet());
- bind(new TypeLiteral<DynamicMap<Cache<?, ?>>>() {}).toInstance(DynamicMap.emptyMap());
+ DynamicMap.mapOf(binder(), new TypeLiteral<Cache<?, ?>>() {});
bind(new TypeLiteral<List<CommentLinkInfo>>() {})
.toProvider(CommentLinkProvider.class)
.in(SINGLETON);
@@ -216,7 +216,8 @@
bind(AccountVisibility.class).toProvider(AccountVisibilityProvider.class).in(SINGLETON);
ModuleOverloader.override(
- modules, LibModuleLoader.loadModules(parentInjector, LibModuleType.SYS_BATCH_MODULE))
+ modules,
+ LibModuleLoader.loadModules(parentInjector, LibModuleType.SYS_BATCH_MODULE_TYPE))
.stream()
.forEach(this::install);
}
diff --git a/java/com/google/gerrit/pgm/util/SiteProgram.java b/java/com/google/gerrit/pgm/util/SiteProgram.java
index ed6bce6..ff0b31e 100644
--- a/java/com/google/gerrit/pgm/util/SiteProgram.java
+++ b/java/com/google/gerrit/pgm/util/SiteProgram.java
@@ -137,7 +137,7 @@
return Guice.createInjector(
PRODUCTION,
ModuleOverloader.override(
- modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.DB_MODULE)));
+ modules, LibModuleLoader.loadModules(cfgInjector, LibModuleType.DB_MODULE_TYPE)));
} catch (CreationException ce) {
Message first = ce.getErrorMessages().iterator().next();
Throwable why = first.getCause();
diff --git a/java/com/google/gerrit/server/CreateGroupPermissionSyncer.java b/java/com/google/gerrit/server/CreateGroupPermissionSyncer.java
index 6c76de7..60f3d4b 100644
--- a/java/com/google/gerrit/server/CreateGroupPermissionSyncer.java
+++ b/java/com/google/gerrit/server/CreateGroupPermissionSyncer.java
@@ -121,7 +121,7 @@
});
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
}
diff --git a/java/com/google/gerrit/server/DynamicOptions.java b/java/com/google/gerrit/server/DynamicOptions.java
index db0aa70..7f9fbd2 100644
--- a/java/com/google/gerrit/server/DynamicOptions.java
+++ b/java/com/google/gerrit/server/DynamicOptions.java
@@ -225,7 +225,7 @@
Class<?> beanClass =
(bean instanceof BeanReceiver)
? ((BeanReceiver) bean).getExportedBeanReceiver()
- : getClass();
+ : bean.getClass();
for (String plugin : dynamicBeans.plugins()) {
Provider<DynamicBean> provider =
dynamicBeans.byPlugin(plugin).get(beanClass.getCanonicalName());
diff --git a/java/com/google/gerrit/server/LibModuleLoader.java b/java/com/google/gerrit/server/LibModuleLoader.java
index fd2c5e4..cf53c80 100644
--- a/java/com/google/gerrit/server/LibModuleLoader.java
+++ b/java/com/google/gerrit/server/LibModuleLoader.java
@@ -22,8 +22,10 @@
import com.google.inject.Key;
import com.google.inject.Module;
import com.google.inject.ProvisionException;
+import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.List;
+import java.util.Map;
import org.eclipse.jgit.lib.Config;
/** Loads configured Guice modules from {@code gerrit.installModule}. */
@@ -37,6 +39,33 @@
.collect(toList());
}
+ public static List<Module> loadReindexModules(
+ Injector parent, Map<String, Integer> versions, int threads, boolean replica) {
+ Config cfg = getConfig(parent);
+ return Arrays.stream(
+ cfg.getStringList(
+ "gerrit", null, "install" + LibModuleType.INDEX_MODULE_TYPE.getConfigKey()))
+ .map(m -> createReindexModule(m, versions, threads, replica))
+ .collect(toList());
+ }
+
+ private static Module createReindexModule(
+ String className, Map<String, Integer> versions, int threads, boolean replica) {
+ Class<Module> clazz = loadModule(className);
+ try {
+
+ Method m =
+ clazz.getMethod("singleVersionWithExplicitVersions", Map.class, int.class, boolean.class);
+
+ Module module = (Module) m.invoke(null, versions, threads, replica);
+ logger.atInfo().log("Installed module %s", className);
+ return module;
+ } catch (Exception e) {
+ logger.atSevere().withCause(e).log("Unable to load libModule for %s", className);
+ throw new IllegalStateException(e);
+ }
+ }
+
private static Config getConfig(Injector i) {
return i.getInstance(Key.get(Config.class, GerritServerConfig.class));
}
diff --git a/java/com/google/gerrit/server/LibModuleType.java b/java/com/google/gerrit/server/LibModuleType.java
index b9cb196..57206aa 100644
--- a/java/com/google/gerrit/server/LibModuleType.java
+++ b/java/com/google/gerrit/server/LibModuleType.java
@@ -18,13 +18,16 @@
public enum LibModuleType {
/** Module for the sysInjector. */
- SYS_MODULE("Module"),
+ SYS_MODULE_TYPE("Module"),
/** BatchModule for the sysInjector */
- SYS_BATCH_MODULE("BatchModule"),
+ SYS_BATCH_MODULE_TYPE("BatchModule"),
/** Module for the dbInjector. */
- DB_MODULE("DbModule");
+ DB_MODULE_TYPE("DbModule"),
+
+ /** Module for the implementation of the indexing backend. */
+ INDEX_MODULE_TYPE("IndexModule");
private final String configKey;
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalId.java b/java/com/google/gerrit/server/account/externalids/ExternalId.java
index 30f4094..d27ff0e 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalId.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalId.java
@@ -186,17 +186,26 @@
return scheme.equals(scheme());
}
+ @Memoized
+ public ObjectId sha1() {
+ return sha1(isCaseInsensitive());
+ }
+
/**
* Returns the SHA1 of the external ID that is used as note ID in the refs/meta/external-ids
* notes branch.
*/
@SuppressWarnings("deprecation") // Use Hashing.sha1 for compatibility.
- @Memoized
- public ObjectId sha1() {
- String keyString = isCaseInsensitive() ? get().toLowerCase(Locale.US) : get();
+ private ObjectId sha1(Boolean isCaseInsensitive) {
+ String keyString = isCaseInsensitive ? get().toLowerCase(Locale.US) : get();
return ObjectId.fromRaw(Hashing.sha1().hashString(keyString, UTF_8).asBytes());
}
+ @Memoized
+ public ObjectId caseSensitiveSha1() {
+ return sha1(false);
+ }
+
/**
* Exports this external ID key as string with the format "scheme:id", or "id" if scheme is
* null.
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdCaseSensitivityMigrator.java b/java/com/google/gerrit/server/account/externalids/ExternalIdCaseSensitivityMigrator.java
new file mode 100644
index 0000000..a59e935
--- /dev/null
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdCaseSensitivityMigrator.java
@@ -0,0 +1,138 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.account.externalids;
+
+import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_GERRIT;
+import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_USERNAME;
+
+import com.google.common.flogger.FluentLogger;
+import com.google.gerrit.exceptions.DuplicateKeyException;
+import com.google.gerrit.server.config.AllUsersName;
+import com.google.gerrit.server.git.GitRepositoryManager;
+import com.google.gerrit.server.git.meta.MetaDataUpdate;
+import com.google.inject.AbstractModule;
+import com.google.inject.Inject;
+import com.google.inject.Provider;
+import com.google.inject.assistedinject.Assisted;
+import com.google.inject.assistedinject.FactoryModuleBuilder;
+import java.io.IOException;
+import java.util.Collection;
+import java.util.Collections;
+import org.eclipse.jgit.errors.ConfigInvalidException;
+import org.eclipse.jgit.errors.RepositoryNotFoundException;
+import org.eclipse.jgit.lib.Repository;
+
+public class ExternalIdCaseSensitivityMigrator {
+
+ public static class ExternalIdCaseSensitivityMigratorModule extends AbstractModule {
+ @Override
+ public void configure() {
+ install(new FactoryModuleBuilder().build(ExternalIdCaseSensitivityMigrator.Factory.class));
+ }
+ }
+
+ private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+
+ public interface Factory {
+ ExternalIdCaseSensitivityMigrator create(
+ @Assisted("isUserNameCaseInsensitive") Boolean isUserNameCaseInsensitive,
+ @Assisted("dryRun") Boolean dryRun);
+ }
+
+ private GitRepositoryManager repoManager;
+ private AllUsersName allUsersName;
+ private Provider<MetaDataUpdate.Server> metaDataUpdateServerFactory;
+ private ExternalIdNotes.FactoryNoReindex externalIdNotesFactory;
+
+ private ExternalIdFactory externalIdFactory;
+ private Boolean isUserNameCaseInsensitive;
+ private Boolean dryRun;
+
+ @Inject
+ public ExternalIdCaseSensitivityMigrator(
+ GitRepositoryManager repoManager,
+ AllUsersName allUsersName,
+ Provider<MetaDataUpdate.Server> metaDataUpdateServerFactory,
+ ExternalIdNotes.FactoryNoReindex externalIdNotesFactory,
+ ExternalIdFactory externalIdFactory,
+ @Assisted("isUserNameCaseInsensitive") Boolean isUserNameCaseInsensitive,
+ @Assisted("dryRun") Boolean dryRun) {
+ this.repoManager = repoManager;
+ this.allUsersName = allUsersName;
+ this.metaDataUpdateServerFactory = metaDataUpdateServerFactory;
+ this.externalIdNotesFactory = externalIdNotesFactory;
+ this.externalIdFactory = externalIdFactory;
+
+ this.isUserNameCaseInsensitive = isUserNameCaseInsensitive;
+ this.dryRun = dryRun;
+ }
+
+ private void recomputeExternalIdNoteId(ExternalIdNotes extIdNotes, ExternalId extId)
+ throws DuplicateKeyException, IOException {
+ if (extId.isScheme(SCHEME_GERRIT) || extId.isScheme(SCHEME_USERNAME)) {
+ ExternalIdKeyFactory keyFactory =
+ new ExternalIdKeyFactory(
+ new ExternalIdKeyFactory.Config() {
+ @Override
+ public boolean isUserNameCaseInsensitive() {
+ return isUserNameCaseInsensitive;
+ }
+ });
+ ExternalId.Key updatedKey = keyFactory.create(extId.key().scheme(), extId.key().id());
+ ExternalId.Key oldKey =
+ keyFactory.create(extId.key().scheme(), extId.key().id(), !isUserNameCaseInsensitive);
+ if (!oldKey.sha1().getName().equals(updatedKey.sha1().getName())
+ && !extId.key().sha1().getName().equals(updatedKey.sha1().getName())) {
+ logger.atInfo().log("Converting note name of external ID: %s", oldKey);
+ ExternalId updatedExtId =
+ externalIdFactory.create(
+ updatedKey, extId.accountId(), extId.email(), extId.password(), extId.blobId());
+ ExternalId oldExtId =
+ externalIdFactory.create(
+ oldKey, extId.accountId(), extId.email(), extId.password(), extId.blobId());
+ extIdNotes.replace(
+ Collections.singleton(oldExtId),
+ Collections.singleton(updatedExtId),
+ (externalId) -> externalId.key().sha1());
+ }
+ }
+ }
+
+ public void migrate(Collection<ExternalId> todo, Runnable monitor)
+ throws RepositoryNotFoundException, IOException, ConfigInvalidException {
+ try (Repository repo = repoManager.openRepository(allUsersName)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(repo);
+ for (ExternalId extId : todo) {
+ recomputeExternalIdNoteId(extIdNotes, extId);
+ monitor.run();
+ }
+ if (!dryRun) {
+ try (MetaDataUpdate metaDataUpdate =
+ metaDataUpdateServerFactory.get().create(allUsersName)) {
+ metaDataUpdate.setMessage(
+ String.format(
+ "Migration to case %ssensitive usernames",
+ isUserNameCaseInsensitive ? "" : "in"));
+ extIdNotes.commit(metaDataUpdate);
+ } catch (Exception e) {
+ logger.atSevere().withCause(e).log(e.getMessage());
+ }
+ }
+ } catch (DuplicateExternalIdKeyException e) {
+ logger.atSevere().withCause(e).log(e.getMessage());
+ throw e;
+ }
+ }
+}
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdFactory.java b/java/com/google/gerrit/server/account/externalids/ExternalIdFactory.java
index 196cc97..b16f73f 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIdFactory.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdFactory.java
@@ -22,6 +22,7 @@
import com.google.gerrit.common.Nullable;
import com.google.gerrit.entities.Account;
import com.google.gerrit.server.account.HashedPassword;
+import com.google.gerrit.server.config.AuthConfig;
import java.util.Set;
import javax.inject.Inject;
import javax.inject.Singleton;
@@ -32,10 +33,12 @@
@Singleton
public class ExternalIdFactory {
private final ExternalIdKeyFactory externalIdKeyFactory;
+ private AuthConfig authConfig;
@Inject
- public ExternalIdFactory(ExternalIdKeyFactory externalIdKeyFactory) {
+ public ExternalIdFactory(ExternalIdKeyFactory externalIdKeyFactory, AuthConfig authConfig) {
this.externalIdKeyFactory = externalIdKeyFactory;
+ this.authConfig = authConfig;
}
/**
@@ -248,10 +251,23 @@
}
if (!externalIdKey.sha1().getName().equals(noteId)) {
- throw invalidConfig(
- noteId,
- String.format(
- "SHA1 of external ID '%s' does not match note ID '%s'", externalIdKeyStr, noteId));
+ if (!authConfig.isUserNameCaseInsensitiveMigrationMode()) {
+ throw invalidConfig(
+ noteId,
+ String.format(
+ "SHA1 of external ID '%s' does not match note ID '%s'", externalIdKeyStr, noteId));
+ }
+
+ if (!externalIdKey.caseSensitiveSha1().getName().equals(noteId)) {
+ throw invalidConfig(
+ noteId,
+ String.format(
+ "Neither case sensitive nor case insensitive SHA1 of external ID '%s' match note ID"
+ + " '%s'",
+ externalIdKeyStr, noteId));
+ }
+ externalIdKey =
+ externalIdKeyFactory.create(externalIdKey.scheme(), externalIdKey.id(), false);
}
String email =
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdKeyFactory.java b/java/com/google/gerrit/server/account/externalids/ExternalIdKeyFactory.java
index 95df4a9..68d8b0c 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIdKeyFactory.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdKeyFactory.java
@@ -65,9 +65,23 @@
* @return the created external ID key
*/
public ExternalId.Key create(@Nullable String scheme, String id) {
+ return create(scheme, id, isUserNameCaseInsensitive);
+ }
+
+ /**
+ * Creates an external ID key.
+ *
+ * @param scheme the scheme name, must not contain colons (':'). E.g. {@link
+ * ExternalId#SCHEME_USERNAME}.
+ * @param id the external ID, must not contain colons (':')
+ * @param userNameCaseInsensitive whether the external ID key is matched case insensitively
+ * @return the created external ID key
+ */
+ public ExternalId.Key create(
+ @Nullable String scheme, String id, boolean userNameCaseInsensitive) {
if (scheme != null
&& (scheme.equals(ExternalId.SCHEME_USERNAME) || scheme.equals(ExternalId.SCHEME_GERRIT))) {
- return ExternalId.Key.create(scheme, id, isUserNameCaseInsensitive);
+ return ExternalId.Key.create(scheme, id, userNameCaseInsensitive);
}
return ExternalId.Key.create(scheme, id, false);
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdNotes.java b/java/com/google/gerrit/server/account/externalids/ExternalIdNotes.java
index 2b9c00a9..aa37451 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIdNotes.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdNotes.java
@@ -37,6 +37,7 @@
import com.google.gerrit.metrics.MetricMaker;
import com.google.gerrit.server.account.AccountsUpdate;
import com.google.gerrit.server.config.AllUsersName;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.git.meta.MetaDataUpdate;
import com.google.gerrit.server.git.meta.VersionedMetaData;
import com.google.gerrit.server.index.account.AccountIndexer;
@@ -53,6 +54,7 @@
import java.util.List;
import java.util.Optional;
import java.util.Set;
+import java.util.function.Function;
import org.eclipse.jgit.errors.ConfigInvalidException;
import org.eclipse.jgit.lib.BlobBasedConfig;
import org.eclipse.jgit.lib.CommitBuilder;
@@ -100,18 +102,21 @@
protected final AllUsersName allUsersName;
protected final DynamicMap<ExternalIdUpsertPreprocessor> upsertPreprocessors;
protected final ExternalIdFactory externalIdFactory;
+ protected final AuthConfig authConfig;
protected ExternalIdNotesLoader(
ExternalIdCache externalIdCache,
MetricMaker metricMaker,
AllUsersName allUsersName,
DynamicMap<ExternalIdUpsertPreprocessor> upsertPreprocessors,
- ExternalIdFactory externalIdFactory) {
+ ExternalIdFactory externalIdFactory,
+ AuthConfig authConfig) {
this.externalIdCache = externalIdCache;
this.metricMaker = metricMaker;
this.allUsersName = allUsersName;
this.upsertPreprocessors = upsertPreprocessors;
this.externalIdFactory = externalIdFactory;
+ this.authConfig = authConfig;
}
/**
@@ -203,8 +208,15 @@
MetricMaker metricMaker,
AllUsersName allUsersName,
DynamicMap<ExternalIdUpsertPreprocessor> upsertPreprocessors,
- ExternalIdFactory externalIdFactory) {
- super(externalIdCache, metricMaker, allUsersName, upsertPreprocessors, externalIdFactory);
+ ExternalIdFactory externalIdFactory,
+ AuthConfig authConfig) {
+ super(
+ externalIdCache,
+ metricMaker,
+ allUsersName,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig);
this.accountIndexer = accountIndexer;
}
@@ -212,7 +224,12 @@
public ExternalIdNotes load(Repository allUsersRepo)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
- metricMaker, allUsersName, allUsersRepo, upsertPreprocessors, externalIdFactory)
+ metricMaker,
+ allUsersName,
+ allUsersRepo,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
.load();
}
@@ -220,7 +237,12 @@
public ExternalIdNotes load(Repository allUsersRepo, @Nullable ObjectId rev)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
- metricMaker, allUsersName, allUsersRepo, upsertPreprocessors, externalIdFactory)
+ metricMaker,
+ allUsersName,
+ allUsersRepo,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
.load(rev);
}
@@ -239,15 +261,27 @@
MetricMaker metricMaker,
AllUsersName allUsersName,
DynamicMap<ExternalIdUpsertPreprocessor> upsertPreprocessors,
- ExternalIdFactory externalIdFactory) {
- super(externalIdCache, metricMaker, allUsersName, upsertPreprocessors, externalIdFactory);
+ ExternalIdFactory externalIdFactory,
+ AuthConfig authConfig) {
+ super(
+ externalIdCache,
+ metricMaker,
+ allUsersName,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig);
}
@Override
public ExternalIdNotes load(Repository allUsersRepo)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
- metricMaker, allUsersName, allUsersRepo, upsertPreprocessors, externalIdFactory)
+ metricMaker,
+ allUsersName,
+ allUsersRepo,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
.setNoReindex()
.load();
}
@@ -256,7 +290,12 @@
public ExternalIdNotes load(Repository allUsersRepo, @Nullable ObjectId rev)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
- metricMaker, allUsersName, allUsersRepo, upsertPreprocessors, externalIdFactory)
+ metricMaker,
+ allUsersName,
+ allUsersRepo,
+ upsertPreprocessors,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
.setNoReindex()
.load(rev);
}
@@ -281,14 +320,16 @@
AllUsersName allUsersName,
Repository allUsersRepo,
@Nullable ObjectId rev,
- ExternalIdFactory externalIdFactory)
+ ExternalIdFactory externalIdFactory,
+ boolean isUserNameCaseInsensitiveMigrationMode)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
new DisabledMetricMaker(),
allUsersName,
allUsersRepo,
DynamicMap.emptyMap(),
- externalIdFactory)
+ externalIdFactory,
+ isUserNameCaseInsensitiveMigrationMode)
.setReadOnly()
.setNoCacheUpdate()
.setNoReindex()
@@ -306,14 +347,18 @@
* @return {@link ExternalIdNotes} instance that doesn't updates caches on save
*/
public static ExternalIdNotes loadNoCacheUpdate(
- AllUsersName allUsersName, Repository allUsersRepo, ExternalIdFactory externalIdFactory)
+ AllUsersName allUsersName,
+ Repository allUsersRepo,
+ ExternalIdFactory externalIdFactory,
+ boolean isUserNameCaseInsensitiveMigrationMode)
throws IOException, ConfigInvalidException {
return new ExternalIdNotes(
new DisabledMetricMaker(),
allUsersName,
allUsersRepo,
DynamicMap.emptyMap(),
- externalIdFactory)
+ externalIdFactory,
+ isUserNameCaseInsensitiveMigrationMode)
.setNoCacheUpdate()
.setNoReindex()
.load();
@@ -350,13 +395,27 @@
private boolean readOnly = false;
private boolean noCacheUpdate = false;
private boolean noReindex = false;
+ private boolean isUserNameCaseInsensitiveMigrationMode = false;
+ protected final Function<ExternalId, ObjectId> defaultNoteIdResolver =
+ (extId) -> {
+ ObjectId noteId = extId.key().sha1();
+ try {
+ if (isUserNameCaseInsensitiveMigrationMode && !noteMap.contains(noteId)) {
+ noteId = extId.key().caseSensitiveSha1();
+ }
+ } catch (IOException e) {
+ return noteId;
+ }
+ return noteId;
+ };
private ExternalIdNotes(
MetricMaker metricMaker,
AllUsersName allUsersName,
Repository allUsersRepo,
DynamicMap<ExternalIdUpsertPreprocessor> upsertPreprocessors,
- ExternalIdFactory externalIdFactory) {
+ ExternalIdFactory externalIdFactory,
+ boolean isUserNameCaseInsensitiveMigrationMode) {
this.updateCount =
metricMaker.newCounter(
"notedb/external_id_update_count",
@@ -378,6 +437,7 @@
.addTarget(ExternalIdNotes.class)
.build();
this.externalIdFactory = externalIdFactory;
+ this.isUserNameCaseInsensitiveMigrationMode = isUserNameCaseInsensitiveMigrationMode;
}
public ExternalIdNotes setAfterReadRevision(Runnable afterReadRevision) {
@@ -449,16 +509,26 @@
*/
public Optional<ExternalId> get(ExternalId.Key key) throws IOException, ConfigInvalidException {
checkLoaded();
+ ObjectId noteId = getNoteId(key);
+ if (noteMap.contains(noteId)) {
+
+ try (RevWalk rw = new RevWalk(repo)) {
+ ObjectId noteDataId = noteMap.get(noteId);
+ byte[] raw = readNoteData(rw, noteDataId);
+ return Optional.of(externalIdFactory.parse(noteId.name(), raw, noteDataId));
+ }
+ }
+ return Optional.empty();
+ }
+
+ protected ObjectId getNoteId(ExternalId.Key key) throws IOException {
ObjectId noteId = key.sha1();
- if (!noteMap.contains(noteId)) {
- return Optional.empty();
+
+ if (!noteMap.contains(noteId) && isUserNameCaseInsensitiveMigrationMode) {
+ noteId = key.caseSensitiveSha1();
}
- try (RevWalk rw = new RevWalk(repo)) {
- ObjectId noteDataId = noteMap.get(noteId);
- byte[] raw = readNoteData(rw, noteDataId);
- return Optional.of(externalIdFactory.parse(noteId.name(), raw, noteDataId));
- }
+ return noteId;
}
/**
@@ -651,6 +721,12 @@
cacheUpdates.add(cu -> cu.remove(removedExtIds));
}
+ public void replace(
+ Account.Id accountId, Collection<ExternalId.Key> toDelete, Collection<ExternalId> toAdd)
+ throws IOException, DuplicateExternalIdKeyException {
+ replace(accountId, toDelete, toAdd, defaultNoteIdResolver);
+ }
+
/**
* Replaces external IDs for an account by external ID keys.
*
@@ -663,7 +739,10 @@
* the specified account.
*/
public void replace(
- Account.Id accountId, Collection<ExternalId.Key> toDelete, Collection<ExternalId> toAdd)
+ Account.Id accountId,
+ Collection<ExternalId.Key> toDelete,
+ Collection<ExternalId> toAdd,
+ Function<ExternalId, ObjectId> noteIdResolver)
throws IOException, DuplicateExternalIdKeyException {
checkLoaded();
checkSameAccount(toAdd, accountId);
@@ -681,7 +760,7 @@
}
for (ExternalId extId : toAdd) {
- ExternalId insertedExtId = upsert(rw, inserter, noteMap, extId);
+ ExternalId insertedExtId = upsert(rw, inserter, noteMap, extId, noteIdResolver);
preprocessUpsert(insertedExtId);
updatedExtIds.add(insertedExtId);
}
@@ -757,6 +836,32 @@
replace(accountId, toDelete.stream().map(ExternalId::key).collect(toSet()), toAdd);
}
+ /**
+ * Replaces external IDs.
+ *
+ * <p>Deletion of external IDs is done before adding the new external IDs. This means if an
+ * external ID is specified for deletion and an external ID with the same key is specified to be
+ * added, the old external ID with that key is deleted first and then the new external ID is added
+ * (so the external ID for that key is replaced).
+ *
+ * @throws IllegalStateException is thrown if the specified external IDs belong to different
+ * accounts.
+ */
+ public void replace(
+ Collection<ExternalId> toDelete,
+ Collection<ExternalId> toAdd,
+ Function<ExternalId, ObjectId> noteIdResolver)
+ throws IOException, DuplicateExternalIdKeyException {
+ Account.Id accountId = checkSameAccount(Iterables.concat(toDelete, toAdd));
+ if (accountId == null) {
+ // toDelete and toAdd are empty -> nothing to do
+ return;
+ }
+
+ replace(
+ accountId, toDelete.stream().map(ExternalId::key).collect(toSet()), toAdd, noteIdResolver);
+ }
+
@Override
protected void onLoad() throws IOException, ConfigInvalidException {
if (revision != null) {
@@ -865,9 +970,26 @@
*/
private ExternalId upsert(RevWalk rw, ObjectInserter ins, NoteMap noteMap, ExternalId extId)
throws IOException, ConfigInvalidException {
+ return upsert(rw, ins, noteMap, extId, defaultNoteIdResolver);
+ }
+
+ /**
+ * Inserts or updates a new external ID and sets it in the note map.
+ *
+ * <p>If the external ID already exists, it is overwritten.
+ */
+ private ExternalId upsert(
+ RevWalk rw,
+ ObjectInserter ins,
+ NoteMap noteMap,
+ ExternalId extId,
+ Function<ExternalId, ObjectId> noteIdResolver)
+ throws IOException, ConfigInvalidException {
ObjectId noteId = extId.key().sha1();
Config c = new Config();
- if (noteMap.contains(noteId)) {
+ ObjectId resolvedNoteId = noteIdResolver.apply(extId);
+ if (noteMap.contains(resolvedNoteId)) {
+ noteId = resolvedNoteId;
ObjectId noteDataId = noteMap.get(noteId);
byte[] raw = readNoteData(rw, noteDataId);
try {
@@ -892,7 +1014,8 @@
*/
private void remove(RevWalk rw, NoteMap noteMap, ExternalId extId)
throws IOException, ConfigInvalidException {
- ObjectId noteId = extId.key().sha1();
+ ObjectId noteId = getNoteId(extId.key());
+
if (!noteMap.contains(noteId)) {
return;
}
@@ -919,7 +1042,8 @@
private ExternalId remove(
RevWalk rw, NoteMap noteMap, ExternalId.Key extIdKey, Account.Id expectedAccountId)
throws IOException, ConfigInvalidException {
- ObjectId noteId = extIdKey.sha1();
+ ObjectId noteId = getNoteId(extIdKey);
+
if (!noteMap.contains(noteId)) {
return null;
}
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdReader.java b/java/com/google/gerrit/server/account/externalids/ExternalIdReader.java
index 0d715ae..e5aace0 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIdReader.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdReader.java
@@ -23,6 +23,7 @@
import com.google.gerrit.metrics.MetricMaker;
import com.google.gerrit.metrics.Timer0;
import com.google.gerrit.server.config.AllUsersName;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@@ -70,13 +71,15 @@
private final Timer0 readAllLatency;
private final Timer0 readSingleLatency;
private final ExternalIdFactory externalIdFactory;
+ private final AuthConfig authConfig;
@Inject
ExternalIdReader(
GitRepositoryManager repoManager,
AllUsersName allUsersName,
MetricMaker metricMaker,
- ExternalIdFactory externalIdFactory) {
+ ExternalIdFactory externalIdFactory,
+ AuthConfig authConfig) {
this.repoManager = repoManager;
this.allUsersName = allUsersName;
this.readAllLatency =
@@ -92,6 +95,7 @@
.setCumulative()
.setUnit(Units.MILLISECONDS));
this.externalIdFactory = externalIdFactory;
+ this.authConfig = authConfig;
}
@VisibleForTesting
@@ -111,7 +115,13 @@
try (Timer0.Context ctx = readAllLatency.start();
Repository repo = repoManager.openRepository(allUsersName)) {
- return ExternalIdNotes.loadReadOnly(allUsersName, repo, null, externalIdFactory).all();
+ return ExternalIdNotes.loadReadOnly(
+ allUsersName,
+ repo,
+ null,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
+ .all();
}
}
@@ -130,7 +140,13 @@
try (Timer0.Context ctx = readAllLatency.start();
Repository repo = repoManager.openRepository(allUsersName)) {
- return ExternalIdNotes.loadReadOnly(allUsersName, repo, rev, externalIdFactory).all();
+ return ExternalIdNotes.loadReadOnly(
+ allUsersName,
+ repo,
+ rev,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
+ .all();
}
}
@@ -140,7 +156,13 @@
try (Timer0.Context ctx = readSingleLatency.start();
Repository repo = repoManager.openRepository(allUsersName)) {
- return ExternalIdNotes.loadReadOnly(allUsersName, repo, null, externalIdFactory).get(key);
+ return ExternalIdNotes.loadReadOnly(
+ allUsersName,
+ repo,
+ null,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
+ .get(key);
}
}
@@ -151,7 +173,13 @@
try (Timer0.Context ctx = readSingleLatency.start();
Repository repo = repoManager.openRepository(allUsersName)) {
- return ExternalIdNotes.loadReadOnly(allUsersName, repo, rev, externalIdFactory).get(key);
+ return ExternalIdNotes.loadReadOnly(
+ allUsersName,
+ repo,
+ rev,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode())
+ .get(key);
}
}
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIds.java b/java/com/google/gerrit/server/account/externalids/ExternalIds.java
index 4e1e524..9450ff5 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIds.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIds.java
@@ -19,6 +19,7 @@
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.gerrit.entities.Account;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import java.io.IOException;
@@ -35,11 +36,19 @@
public class ExternalIds {
private final ExternalIdReader externalIdReader;
private final ExternalIdCache externalIdCache;
+ private final AuthConfig authConfig;
+ private final ExternalIdKeyFactory externalIdKeyFactory;
@Inject
- public ExternalIds(ExternalIdReader externalIdReader, ExternalIdCache externalIdCache) {
+ public ExternalIds(
+ ExternalIdReader externalIdReader,
+ ExternalIdCache externalIdCache,
+ ExternalIdKeyFactory externalIdKeyFactory,
+ AuthConfig authConfig) {
this.externalIdReader = externalIdReader;
this.externalIdCache = externalIdCache;
+ this.externalIdKeyFactory = externalIdKeyFactory;
+ this.authConfig = authConfig;
}
/** Returns all external IDs. */
@@ -54,7 +63,15 @@
/** Returns the specified external ID. */
public Optional<ExternalId> get(ExternalId.Key key) throws IOException {
- return externalIdCache.byKey(key);
+ Optional<ExternalId> externalId = Optional.empty();
+ if (authConfig.isUserNameCaseInsensitiveMigrationMode()) {
+ externalId =
+ externalIdCache.byKey(externalIdKeyFactory.create(key.scheme(), key.id(), false));
+ }
+ if (!externalId.isPresent()) {
+ externalId = externalIdCache.byKey(key);
+ }
+ return externalId;
}
/** Returns the specified external ID from the given revision. */
diff --git a/java/com/google/gerrit/server/account/externalids/ExternalIdsConsistencyChecker.java b/java/com/google/gerrit/server/account/externalids/ExternalIdsConsistencyChecker.java
index cf0e5d3..4e67e3d 100644
--- a/java/com/google/gerrit/server/account/externalids/ExternalIdsConsistencyChecker.java
+++ b/java/com/google/gerrit/server/account/externalids/ExternalIdsConsistencyChecker.java
@@ -61,14 +61,14 @@
public List<ConsistencyProblemInfo> check() throws IOException, ConfigInvalidException {
try (Repository repo = repoManager.openRepository(allUsers)) {
- return check(ExternalIdNotes.loadReadOnly(allUsers, repo, null, externalIdFactory));
+ return check(ExternalIdNotes.loadReadOnly(allUsers, repo, null, externalIdFactory, false));
}
}
public List<ConsistencyProblemInfo> check(ObjectId rev)
throws IOException, ConfigInvalidException {
try (Repository repo = repoManager.openRepository(allUsers)) {
- return check(ExternalIdNotes.loadReadOnly(allUsers, repo, rev, externalIdFactory));
+ return check(ExternalIdNotes.loadReadOnly(allUsers, repo, rev, externalIdFactory, false));
}
}
diff --git a/java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java b/java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigratiorExecutor.java
similarity index 60%
copy from java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java
copy to java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigratiorExecutor.java
index 6451b0f..8a3e4f1 100644
--- a/java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java
+++ b/java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigratiorExecutor.java
@@ -1,4 +1,4 @@
-// Copyright (C) 2018 The Android Open Source Project
+// Copyright (C) 2021 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,11 +12,13 @@
// See the License for the specific language governing permissions and
// limitations under the License.
-package com.google.gerrit.elasticsearch.bulk;
+package com.google.gerrit.server.account.externalids;
-public class DeleteRequest extends ActionRequest {
+import static java.lang.annotation.RetentionPolicy.RUNTIME;
- public DeleteRequest(String id, String index) {
- super("delete", id, index);
- }
-}
+import com.google.inject.BindingAnnotation;
+import java.lang.annotation.Retention;
+
+@Retention(RUNTIME)
+@BindingAnnotation
+public @interface OnlineExternalIdCaseSensivityMigratiorExecutor {}
diff --git a/java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigrator.java b/java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigrator.java
new file mode 100644
index 0000000..72e7e90
--- /dev/null
+++ b/java/com/google/gerrit/server/account/externalids/OnlineExternalIdCaseSensivityMigrator.java
@@ -0,0 +1,119 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.account.externalids;
+
+import com.google.common.flogger.FluentLogger;
+import com.google.gerrit.server.config.GerritServerConfig;
+import com.google.gerrit.server.config.SitePath;
+import com.google.gerrit.server.index.ReindexerAlreadyRunningException;
+import com.google.gerrit.server.index.VersionManager;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.Collection;
+import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
+import org.eclipse.jgit.errors.ConfigInvalidException;
+import org.eclipse.jgit.lib.Config;
+import org.eclipse.jgit.lib.ProgressMonitor;
+import org.eclipse.jgit.lib.TextProgressMonitor;
+import org.eclipse.jgit.storage.file.FileBasedConfig;
+import org.eclipse.jgit.util.FS;
+
+@Singleton
+public class OnlineExternalIdCaseSensivityMigrator {
+ private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+
+ private Executor executor;
+ private ExternalIdCaseSensitivityMigrator.Factory migratorFactory;
+ private ExternalIds externalIds;
+ private VersionManager versionManager;
+ private Config globalConfig;
+ private Path sitePath;
+ private final TextProgressMonitor monitor = new TextProgressMonitor();
+ private boolean isUserNameCaseInsensitive;
+ private boolean isUserNameCaseInsensitiveMigrationMode;
+
+ @Inject
+ public OnlineExternalIdCaseSensivityMigrator(
+ @OnlineExternalIdCaseSensivityMigratiorExecutor ExecutorService executor,
+ ExternalIdCaseSensitivityMigrator.Factory migratorFactory,
+ ExternalIds externalIds,
+ VersionManager versionManager,
+ @GerritServerConfig Config globalConfig,
+ @SitePath Path sitePath) {
+ this.migratorFactory = migratorFactory;
+ this.externalIds = externalIds;
+ this.versionManager = versionManager;
+ this.globalConfig = globalConfig;
+ this.sitePath = sitePath;
+ this.executor = executor;
+ this.isUserNameCaseInsensitiveMigrationMode =
+ globalConfig.getBoolean("auth", "userNameCaseInsensitiveMigrationMode", false);
+ this.isUserNameCaseInsensitive =
+ globalConfig.getBoolean("auth", "userNameCaseInsensitive", false);
+ }
+
+ public void migrate() {
+ if (!isUserNameCaseInsensitive || !isUserNameCaseInsensitiveMigrationMode) {
+ logger.atSevere().log(
+ "External IDs online migration requires auth.userNameCaseInsensitive and"
+ + " auth.userNameCaseInsensitiveMigrationMode to be set to true. Skipping"
+ + " migration!");
+ return;
+ }
+ executor.execute(
+ () -> {
+ try {
+ Collection<ExternalId> todo = externalIds.all();
+ try {
+ monitor.beginTask("Converting external ID note names", todo.size());
+ migratorFactory
+ .create(isUserNameCaseInsensitive, false)
+ .migrate(todo, () -> monitor.update(1));
+ } finally {
+ monitor.endTask();
+ }
+ try {
+ updateGerritConfig();
+ monitor.beginTask("Reindex accounts", ProgressMonitor.UNKNOWN);
+ versionManager.startReindexer("accounts", true);
+ } finally {
+ monitor.endTask();
+ }
+ logger.atInfo().log("External IDs migration completed!");
+ } catch (IOException | ConfigInvalidException e) {
+ logger.atSevere().withCause(e).log(
+ "Exception during the external ids migration, cause %s", e.getMessage());
+ } catch (ReindexerAlreadyRunningException e) {
+ logger.atSevere().log("Failed to reindex external ids: %s", e.getMessage());
+ }
+ });
+ }
+
+ private void updateGerritConfig() throws IOException, ConfigInvalidException {
+ logger.atInfo().log(
+ "Setting auth.userNameCaseInsensitiveMigrationMode to false in gerrit.config.");
+
+ FileBasedConfig config =
+ new FileBasedConfig(
+ globalConfig, sitePath.resolve("etc/gerrit.config").toFile(), FS.DETECTED);
+ config.load();
+ config.setBoolean("auth", null, "userNameCaseInsensitiveMigrationMode", false);
+
+ config.save();
+ }
+}
diff --git a/java/com/google/gerrit/server/account/externalids/PasswordVerifier.java b/java/com/google/gerrit/server/account/externalids/PasswordVerifier.java
index 33443c1..eb2bea9 100644
--- a/java/com/google/gerrit/server/account/externalids/PasswordVerifier.java
+++ b/java/com/google/gerrit/server/account/externalids/PasswordVerifier.java
@@ -20,6 +20,7 @@
import com.google.common.flogger.FluentLogger;
import com.google.gerrit.common.Nullable;
import com.google.gerrit.server.account.HashedPassword;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.inject.Inject;
import java.util.Collection;
@@ -29,9 +30,12 @@
private final ExternalIdKeyFactory externalIdKeyFactory;
+ private AuthConfig authConfig;
+
@Inject
- public PasswordVerifier(ExternalIdKeyFactory externalIdKeyFactory) {
+ public PasswordVerifier(ExternalIdKeyFactory externalIdKeyFactory, AuthConfig authConfig) {
this.externalIdKeyFactory = externalIdKeyFactory;
+ this.authConfig = authConfig;
}
/** Returns {@code true} if there is an external ID matching both the username and password. */
@@ -40,13 +44,23 @@
if (password == null) {
return false;
}
+
for (ExternalId id : externalIds) {
// Only process the "username:$USER" entry, which is unique.
- if (!id.isScheme(SCHEME_USERNAME)
- || !id.key().equals(externalIdKeyFactory.create(SCHEME_USERNAME, username))) {
+ if (!id.isScheme(SCHEME_USERNAME)) {
continue;
}
+ if (!id.key().equals(externalIdKeyFactory.create(SCHEME_USERNAME, username))) {
+ if (!authConfig.isUserNameCaseInsensitiveMigrationMode()) {
+ continue;
+ }
+
+ if (!id.key().equals(externalIdKeyFactory.create(SCHEME_USERNAME, username, false))) {
+ continue;
+ }
+ }
+
String hashedStr = id.password();
if (!Strings.isNullOrEmpty(hashedStr)) {
try {
diff --git a/java/com/google/gerrit/server/api/projects/ProjectApiImpl.java b/java/com/google/gerrit/server/api/projects/ProjectApiImpl.java
index 6d7fc15..9521759 100644
--- a/java/com/google/gerrit/server/api/projects/ProjectApiImpl.java
+++ b/java/com/google/gerrit/server/api/projects/ProjectApiImpl.java
@@ -63,6 +63,7 @@
import com.google.gerrit.server.restapi.project.CheckAccess;
import com.google.gerrit.server.restapi.project.ChildProjectsCollection;
import com.google.gerrit.server.restapi.project.CommitsCollection;
+import com.google.gerrit.server.restapi.project.CommitsIncludedInRefs;
import com.google.gerrit.server.restapi.project.CreateAccessChange;
import com.google.gerrit.server.restapi.project.CreateProject;
import com.google.gerrit.server.restapi.project.DeleteBranches;
@@ -88,8 +89,11 @@
import com.google.inject.Provider;
import com.google.inject.assistedinject.Assisted;
import com.google.inject.assistedinject.AssistedInject;
+import java.util.Collection;
import java.util.Collections;
import java.util.List;
+import java.util.Map;
+import java.util.Set;
public class ProjectApiImpl implements ProjectApi {
interface Factory {
@@ -116,6 +120,7 @@
private final CreateAccessChange createAccessChange;
private final GetConfig getConfig;
private final PutConfig putConfig;
+ private final CommitsIncludedInRefs commitsIncludedInRefs;
private final Provider<ListBranches> listBranches;
private final Provider<ListTags> listTags;
private final DeleteBranches deleteBranches;
@@ -154,6 +159,7 @@
CreateAccessChange createAccessChange,
GetConfig getConfig,
PutConfig putConfig,
+ CommitsIncludedInRefs commitsIncludedInRefs,
Provider<ListBranches> listBranches,
Provider<ListTags> listTags,
DeleteBranches deleteBranches,
@@ -191,6 +197,7 @@
createAccessChange,
getConfig,
putConfig,
+ commitsIncludedInRefs,
listBranches,
listTags,
deleteBranches,
@@ -232,6 +239,7 @@
CreateAccessChange createAccessChange,
GetConfig getConfig,
PutConfig putConfig,
+ CommitsIncludedInRefs commitsIncludedInRefs,
Provider<ListBranches> listBranches,
Provider<ListTags> listTags,
DeleteBranches deleteBranches,
@@ -269,6 +277,7 @@
createAccessChange,
getConfig,
putConfig,
+ commitsIncludedInRefs,
listBranches,
listTags,
deleteBranches,
@@ -309,6 +318,7 @@
CreateAccessChange createAccessChange,
GetConfig getConfig,
PutConfig putConfig,
+ CommitsIncludedInRefs commitsIncludedInRefs,
Provider<ListBranches> listBranches,
Provider<ListTags> listTags,
DeleteBranches deleteBranches,
@@ -346,6 +356,7 @@
this.setAccess = setAccess;
this.getConfig = getConfig;
this.putConfig = putConfig;
+ this.commitsIncludedInRefs = commitsIncludedInRefs;
this.listBranches = listBranches;
this.listTags = listTags;
this.deleteBranches = deleteBranches;
@@ -483,6 +494,18 @@
}
@Override
+ public Map<String, Set<String>> commitsIn(Collection<String> commits, Collection<String> refs)
+ throws RestApiException {
+ try {
+ commitsIncludedInRefs.addCommits(commits);
+ commitsIncludedInRefs.addRefs(refs);
+ return commitsIncludedInRefs.apply(project).value();
+ } catch (Exception e) {
+ throw asRestApiException("Cannot list commits included in refs", e);
+ }
+ }
+
+ @Override
public ListRefsRequest<BranchInfo> branches() {
return new ListRefsRequest<BranchInfo>() {
@Override
diff --git a/java/com/google/gerrit/server/cache/CacheDisplay.java b/java/com/google/gerrit/server/cache/CacheDisplay.java
new file mode 100644
index 0000000..60f5186
--- /dev/null
+++ b/java/com/google/gerrit/server/cache/CacheDisplay.java
@@ -0,0 +1,129 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.cache;
+
+import com.google.common.base.Strings;
+import java.io.IOException;
+import java.io.Writer;
+import java.util.Collection;
+
+public class CacheDisplay {
+
+ private final Writer stdout;
+ private final int nw;
+ private final Collection<CacheInfo> caches;
+
+ public CacheDisplay(Writer stdout, int nw, Collection<CacheInfo> caches) {
+ this.stdout = stdout;
+ this.nw = nw;
+ this.caches = caches;
+ }
+
+ public CacheDisplay(Writer stdout, Collection<CacheInfo> caches) {
+ this(stdout, 30, caches);
+ }
+
+ public void displayCaches() throws IOException {
+ stdout.write(
+ String.format( //
+ "%1s %-" + nw + "s|%-21s| %-5s |%-9s|\n" //
+ ,
+ "" //
+ ,
+ "Name" //
+ ,
+ "Entries" //
+ ,
+ "AvgGet" //
+ ,
+ "Hit Ratio" //
+ ));
+ stdout.write(
+ String.format( //
+ "%1s %-" + nw + "s|%6s %6s %7s| %-5s |%-4s %-4s|\n" //
+ ,
+ "" //
+ ,
+ "" //
+ ,
+ "Mem" //
+ ,
+ "Disk" //
+ ,
+ "Space" //
+ ,
+ "" //
+ ,
+ "Mem" //
+ ,
+ "Disk" //
+ ));
+ stdout.write("--");
+ for (int i = 0; i < nw; i++) {
+ stdout.write('-');
+ }
+ stdout.write("+---------------------+---------+---------+\n");
+ printMemoryCoreCaches(caches);
+ printMemoryPluginCaches(caches);
+ printDiskCaches(caches);
+ stdout.write('\n');
+ }
+
+ private void printMemoryCoreCaches(Collection<CacheInfo> caches) throws IOException {
+ for (CacheInfo cache : caches) {
+ if (!cache.name.contains("-") && CacheInfo.CacheType.MEM.equals(cache.type)) {
+ printCache(cache);
+ }
+ }
+ }
+
+ private void printMemoryPluginCaches(Collection<CacheInfo> caches) throws IOException {
+ for (CacheInfo cache : caches) {
+ if (cache.name.contains("-") && CacheInfo.CacheType.MEM.equals(cache.type)) {
+ printCache(cache);
+ }
+ }
+ }
+
+ private void printDiskCaches(Collection<CacheInfo> caches) throws IOException {
+ for (CacheInfo cache : caches) {
+ if (CacheInfo.CacheType.DISK.equals(cache.type)) {
+ printCache(cache);
+ }
+ }
+ }
+
+ private void printCache(CacheInfo cache) throws IOException {
+ stdout.write(
+ String.format(
+ "%1s %-" + nw + "s|%6s %6s %7s| %7s |%4s %4s|\n",
+ CacheInfo.CacheType.DISK.equals(cache.type) ? "D" : "",
+ cache.name,
+ nullToEmpty(cache.entries.mem),
+ nullToEmpty(cache.entries.disk),
+ Strings.nullToEmpty(cache.entries.space),
+ Strings.nullToEmpty(cache.averageGet),
+ formatAsPercent(cache.hitRatio.mem),
+ formatAsPercent(cache.hitRatio.disk)));
+ }
+
+ private static String nullToEmpty(Long l) {
+ return l != null ? String.valueOf(l) : "";
+ }
+
+ private static String formatAsPercent(Integer i) {
+ return i != null ? i + "%" : "";
+ }
+}
diff --git a/java/com/google/gerrit/server/cache/CacheInfo.java b/java/com/google/gerrit/server/cache/CacheInfo.java
new file mode 100644
index 0000000..d6eb065
--- /dev/null
+++ b/java/com/google/gerrit/server/cache/CacheInfo.java
@@ -0,0 +1,133 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.cache;
+
+import com.google.common.cache.Cache;
+import com.google.common.cache.CacheStats;
+
+public class CacheInfo {
+
+ public String name;
+ public CacheType type;
+ public EntriesInfo entries;
+ public String averageGet;
+ public HitRatioInfo hitRatio;
+
+ public CacheInfo(Cache<?, ?> cache) {
+ this(null, cache);
+ }
+
+ public CacheInfo(String name, Cache<?, ?> cache) {
+ this.name = name;
+
+ CacheStats stat = cache.stats();
+
+ entries = new EntriesInfo();
+ entries.setMem(cache.size());
+
+ averageGet = duration(stat.averageLoadPenalty());
+
+ hitRatio = new HitRatioInfo();
+ hitRatio.setMem(stat.hitCount(), stat.requestCount());
+
+ if (cache instanceof PersistentCache) {
+ type = CacheType.DISK;
+ PersistentCache.DiskStats diskStats = ((PersistentCache) cache).diskStats();
+ entries.setDisk(diskStats.size());
+ entries.setSpace(diskStats.space());
+ hitRatio.setDisk(diskStats.hitCount(), diskStats.requestCount());
+ } else {
+ type = CacheType.MEM;
+ }
+ }
+
+ private static String duration(double ns) {
+ if (ns < 0.5) {
+ return null;
+ }
+ String suffix = "ns";
+ if (ns >= 1000.0) {
+ ns /= 1000.0;
+ suffix = "us";
+ }
+ if (ns >= 1000.0) {
+ ns /= 1000.0;
+ suffix = "ms";
+ }
+ if (ns >= 1000.0) {
+ ns /= 1000.0;
+ suffix = "s";
+ }
+ return String.format("%4.1f%s", ns, suffix).trim();
+ }
+
+ public static class EntriesInfo {
+ public Long mem;
+ public Long disk;
+ public String space;
+
+ public void setMem(long mem) {
+ this.mem = mem != 0 ? mem : null;
+ }
+
+ public void setDisk(long disk) {
+ this.disk = disk != 0 ? disk : null;
+ }
+
+ public void setSpace(double value) {
+ space = bytes(value);
+ }
+
+ private static String bytes(double value) {
+ value /= 1024;
+ String suffix = "k";
+
+ if (value > 1024) {
+ value /= 1024;
+ suffix = "m";
+ }
+ if (value > 1024) {
+ value /= 1024;
+ suffix = "g";
+ }
+ return String.format("%1$6.2f%2$s", value, suffix).trim();
+ }
+ }
+
+ public static class HitRatioInfo {
+ public Integer mem;
+ public Integer disk;
+
+ public void setMem(long value, long total) {
+ mem = percent(value, total);
+ }
+
+ public void setDisk(long value, long total) {
+ disk = percent(value, total);
+ }
+
+ private static Integer percent(long value, long total) {
+ if (total <= 0) {
+ return null;
+ }
+ return (int) ((100 * value) / total);
+ }
+ }
+
+ public enum CacheType {
+ MEM,
+ DISK
+ }
+}
diff --git a/java/com/google/gerrit/server/change/IncludedIn.java b/java/com/google/gerrit/server/change/IncludedIn.java
index c06ce82..94498d7 100644
--- a/java/com/google/gerrit/server/change/IncludedIn.java
+++ b/java/com/google/gerrit/server/change/IncludedIn.java
@@ -21,6 +21,7 @@
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.ListMultimap;
+import com.google.common.collect.Lists;
import com.google.common.collect.MultimapBuilder;
import com.google.gerrit.entities.Project;
import com.google.gerrit.extensions.api.changes.IncludedInInfo;
@@ -37,10 +38,15 @@
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.Collection;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
import org.eclipse.jgit.errors.IncorrectObjectTypeException;
import org.eclipse.jgit.errors.MissingObjectException;
+import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Ref;
+import org.eclipse.jgit.lib.RefDatabase;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
@@ -75,13 +81,26 @@
throw new ResourceConflictException(err.getMessage());
}
- IncludedInResolver.Result d = IncludedInResolver.resolve(r, rw, rev);
+ RefDatabase refDb = r.getRefDatabase();
+ Collection<Ref> tags = refDb.getRefsByPrefix(Constants.R_TAGS);
+ Collection<Ref> branches = refDb.getRefsByPrefix(Constants.R_HEADS);
+ List<Ref> allTagsAndBranches = Lists.newArrayListWithCapacity(tags.size() + branches.size());
+ allTagsAndBranches.addAll(tags);
+ allTagsAndBranches.addAll(branches);
+
+ Set<String> allMatchingTagsAndBranches =
+ rw.getMergedInto(rev, IncludedInUtil.getSortedRefs(allTagsAndBranches, rw)).stream()
+ .map(Ref::getName)
+ .collect(Collectors.toSet());
// Filter branches and tags according to their visbility by the user
ImmutableSortedSet<String> filteredBranches =
- sortedShortNames(filterReadableRefs(project, d.branches()));
+ sortedShortNames(
+ filterReadableRefs(
+ project, getMatchingRefNames(allMatchingTagsAndBranches, branches)));
ImmutableSortedSet<String> filteredTags =
- sortedShortNames(filterReadableRefs(project, d.tags()));
+ sortedShortNames(
+ filterReadableRefs(project, getMatchingRefNames(allMatchingTagsAndBranches, tags)));
ListMultimap<String, String> external = MultimapBuilder.hashKeys().arrayListValues().build();
externalIncludedIn.runEach(
@@ -115,6 +134,18 @@
}
}
+ /**
+ * Returns the short names of refs which are as well in the matchingRefs list as well as in the
+ * allRef list.
+ */
+ private static ImmutableList<Ref> getMatchingRefNames(
+ Set<String> matchingRefs, Collection<Ref> allRefs) {
+ return allRefs.stream()
+ .filter(r -> matchingRefs.contains(r.getName()))
+ .distinct()
+ .collect(toImmutableList());
+ }
+
private ImmutableSortedSet<String> sortedShortNames(Collection<String> refs) {
return refs.stream()
.map(Repository::shortenRefName)
diff --git a/java/com/google/gerrit/server/change/IncludedInRefs.java b/java/com/google/gerrit/server/change/IncludedInRefs.java
new file mode 100644
index 0000000..f069251
--- /dev/null
+++ b/java/com/google/gerrit/server/change/IncludedInRefs.java
@@ -0,0 +1,139 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.change;
+
+import static java.util.stream.Collectors.toSet;
+
+import com.google.gerrit.entities.Project;
+import com.google.gerrit.extensions.restapi.AuthException;
+import com.google.gerrit.extensions.restapi.BadRequestException;
+import com.google.gerrit.extensions.restapi.ResourceConflictException;
+import com.google.gerrit.extensions.restapi.ResourceNotFoundException;
+import com.google.gerrit.server.git.GitRepositoryManager;
+import com.google.gerrit.server.permissions.PermissionBackend;
+import com.google.gerrit.server.permissions.PermissionBackend.RefFilterOptions;
+import com.google.gerrit.server.permissions.PermissionBackendException;
+import com.google.inject.Inject;
+import java.io.IOException;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import org.eclipse.jgit.errors.IncorrectObjectTypeException;
+import org.eclipse.jgit.errors.MissingObjectException;
+import org.eclipse.jgit.lib.ObjectId;
+import org.eclipse.jgit.lib.Ref;
+import org.eclipse.jgit.lib.RefDatabase;
+import org.eclipse.jgit.lib.Repository;
+import org.eclipse.jgit.revwalk.RevCommit;
+import org.eclipse.jgit.revwalk.RevWalk;
+
+public class IncludedInRefs {
+ protected final GitRepositoryManager repoManager;
+ protected final PermissionBackend permissionBackend;
+
+ @Inject
+ IncludedInRefs(GitRepositoryManager repoManager, PermissionBackend permissionBackend) {
+ this.repoManager = repoManager;
+ this.permissionBackend = permissionBackend;
+ }
+
+ public Map<String, Set<String>> apply(
+ Project.NameKey project, Set<String> commits, Set<String> refNames)
+ throws ResourceConflictException, BadRequestException, IOException,
+ PermissionBackendException, ResourceNotFoundException, AuthException {
+ try (Repository repo = repoManager.openRepository(project)) {
+ Set<Ref> visibleRefs = getVisibleRefs(repo, refNames, project);
+
+ if (!visibleRefs.isEmpty()) {
+ try (RevWalk revWalk = new RevWalk(repo)) {
+ revWalk.setRetainBody(false);
+ Set<RevCommit> revCommits = getRevCommits(commits, revWalk);
+
+ if (!revCommits.isEmpty()) {
+ return commitsIncludedIn(
+ revCommits, IncludedInUtil.getSortedRefs(visibleRefs, revWalk), revWalk);
+ }
+ }
+ }
+ }
+ return Collections.EMPTY_MAP;
+ }
+
+ private Set<Ref> getVisibleRefs(Repository repo, Set<String> refNames, Project.NameKey project)
+ throws PermissionBackendException {
+ RefDatabase refDb = repo.getRefDatabase();
+ Set<Ref> refs = new HashSet<>();
+ for (String refName : refNames) {
+ try {
+ Ref ref = refDb.exactRef(refName);
+ if (ref != null) {
+ refs.add(ref);
+ }
+ } catch (IOException e) {
+ // Ignore and continue to process rest of the refs so as to keep
+ // the behavior similar to the ref not being visible to the user.
+ // This will ensure that there is no information leak about the
+ // ref when the ref is corrupted and is not visible to the user.
+ }
+ }
+ return filterReadableRefs(project, refs, repo);
+ }
+
+ private Set<RevCommit> getRevCommits(Set<String> commits, RevWalk revWalk) throws IOException {
+ Set<RevCommit> revCommits = new HashSet<>();
+ for (String commit : commits) {
+ try {
+ revCommits.add(revWalk.parseCommit(ObjectId.fromString(commit)));
+ } catch (MissingObjectException | IncorrectObjectTypeException | IllegalArgumentException e) {
+ // Ignore and continue to process the rest of the commits so as to keep
+ // the behavior similar to the commit not being included in any of the
+ // visible specified refs. This will ensure that there is no information
+ // leak about the commit when the commit is not visible to the user.
+ }
+ }
+ return revCommits;
+ }
+
+ private Map<String, Set<String>> commitsIncludedIn(
+ Collection<RevCommit> commits, Collection<Ref> refs, RevWalk revWalk) throws IOException {
+ Map<String, Set<String>> refsByCommit = new HashMap<>();
+ for (RevCommit commit : commits) {
+ List<Ref> matchingRefs = revWalk.getMergedInto(commit, refs);
+ if (matchingRefs.size() > 0) {
+ refsByCommit.put(
+ commit.getName(), matchingRefs.stream().map(Ref::getName).collect(toSet()));
+ }
+ }
+ return refsByCommit;
+ }
+
+ /**
+ * Filter readable refs according to the caller's refs visibility.
+ *
+ * @param project specific Gerrit project.
+ * @param inputRefs a list of refs
+ * @param repo repository opened for the Gerrit project.
+ * @return set of visible refs to the caller
+ */
+ private Set<Ref> filterReadableRefs(Project.NameKey project, Set<Ref> inputRefs, Repository repo)
+ throws PermissionBackendException {
+ PermissionBackend.ForProject perm = permissionBackend.currentUser().project(project);
+ return perm.filter(inputRefs, repo, RefFilterOptions.defaults()).stream().collect(toSet());
+ }
+}
diff --git a/java/com/google/gerrit/server/change/IncludedInResolver.java b/java/com/google/gerrit/server/change/IncludedInResolver.java
deleted file mode 100644
index b2b0a64..0000000
--- a/java/com/google/gerrit/server/change/IncludedInResolver.java
+++ /dev/null
@@ -1,161 +0,0 @@
-// Copyright (C) 2013 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.server.change;
-
-import static java.util.Comparator.comparing;
-import static java.util.stream.Collectors.toList;
-
-import com.google.auto.value.AutoValue;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.LinkedListMultimap;
-import com.google.common.collect.ListMultimap;
-import com.google.common.collect.Lists;
-import com.google.common.flogger.FluentLogger;
-import java.io.IOException;
-import java.util.Collection;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Set;
-import org.eclipse.jgit.errors.IncorrectObjectTypeException;
-import org.eclipse.jgit.errors.MissingObjectException;
-import org.eclipse.jgit.lib.Constants;
-import org.eclipse.jgit.lib.Ref;
-import org.eclipse.jgit.lib.RefDatabase;
-import org.eclipse.jgit.lib.Repository;
-import org.eclipse.jgit.revwalk.RevCommit;
-import org.eclipse.jgit.revwalk.RevFlag;
-import org.eclipse.jgit.revwalk.RevWalk;
-
-/** Resolve in which tags and branches a commit is included. */
-public class IncludedInResolver {
- private static final FluentLogger logger = FluentLogger.forEnclosingClass();
-
- public static Result resolve(Repository repo, RevWalk rw, RevCommit commit) throws IOException {
- RevFlag flag = newFlag(rw);
- try {
- return new IncludedInResolver(repo, rw, commit, flag).resolve();
- } finally {
- rw.disposeFlag(flag);
- }
- }
-
- private static RevFlag newFlag(RevWalk rw) {
- return rw.newFlag("CONTAINS_TARGET");
- }
-
- private final Repository repo;
- private final RevWalk rw;
- private final RevCommit target;
-
- private final RevFlag containsTarget;
- private ListMultimap<RevCommit, String> commitToRef;
- private List<RevCommit> tipsByCommitTime;
-
- private IncludedInResolver(
- Repository repo, RevWalk rw, RevCommit target, RevFlag containsTarget) {
- this.repo = repo;
- this.rw = rw;
- this.target = target;
- this.containsTarget = containsTarget;
- }
-
- private Result resolve() throws IOException {
- RefDatabase refDb = repo.getRefDatabase();
- Collection<Ref> tags = refDb.getRefsByPrefix(Constants.R_TAGS);
- Collection<Ref> branches = refDb.getRefsByPrefix(Constants.R_HEADS);
- List<Ref> allTagsAndBranches = Lists.newArrayListWithCapacity(tags.size() + branches.size());
- allTagsAndBranches.addAll(tags);
- allTagsAndBranches.addAll(branches);
- parseCommits(allTagsAndBranches);
- Set<String> allMatchingTagsAndBranches = includedIn(tipsByCommitTime, 0);
-
- return new AutoValue_IncludedInResolver_Result(
- getMatchingRefNames(allMatchingTagsAndBranches, branches),
- getMatchingRefNames(allMatchingTagsAndBranches, tags));
- }
-
- /** Resolves which tip refs include the target commit. */
- private Set<String> includedIn(Collection<RevCommit> tips, int limit)
- throws IOException, MissingObjectException, IncorrectObjectTypeException {
- Set<String> result = new HashSet<>();
- for (RevCommit tip : tips) {
- boolean commitFound = false;
- rw.resetRetain(RevFlag.UNINTERESTING, containsTarget);
- rw.markStart(tip);
- for (RevCommit commit : rw) {
- if (commit.equals(target) || commit.has(containsTarget)) {
- commitFound = true;
- tip.add(containsTarget);
- result.addAll(commitToRef.get(tip));
- break;
- }
- }
- if (!commitFound) {
- rw.markUninteresting(tip);
- } else if (0 < limit && limit < result.size()) {
- break;
- }
- }
- return result;
- }
-
- /**
- * Returns the short names of refs which are as well in the matchingRefs list as well as in the
- * allRef list.
- */
- private static ImmutableList<Ref> getMatchingRefNames(
- Set<String> matchingRefs, Collection<Ref> allRefs) {
- return allRefs.stream()
- .filter(r -> matchingRefs.contains(r.getName()))
- .distinct()
- .collect(ImmutableList.toImmutableList());
- }
-
- /** Parse commit of ref and store the relation between ref and commit. */
- private void parseCommits(Collection<Ref> refs) throws IOException {
- if (commitToRef != null) {
- return;
- }
- commitToRef = LinkedListMultimap.create();
- for (Ref ref : refs) {
- final RevCommit commit;
- try {
- commit = rw.parseCommit(ref.getObjectId());
- } catch (IncorrectObjectTypeException notCommit) {
- // Its OK for a tag reference to point to a blob or a tree, this
- // is common in the Linux kernel or git.git repository.
- //
- continue;
- } catch (MissingObjectException notHere) {
- // Log the problem with this branch, but keep processing.
- //
- logger.atWarning().log(
- "Reference %s in %s points to dangling object %s",
- ref.getName(), repo.getDirectory(), ref.getObjectId());
- continue;
- }
- commitToRef.put(commit, ref.getName());
- }
- tipsByCommitTime =
- commitToRef.keySet().stream().sorted(comparing(RevCommit::getCommitTime)).collect(toList());
- }
-
- @AutoValue
- public abstract static class Result {
- public abstract ImmutableList<Ref> branches();
-
- public abstract ImmutableList<Ref> tags();
- }
-}
diff --git a/java/com/google/gerrit/server/change/IncludedInUtil.java b/java/com/google/gerrit/server/change/IncludedInUtil.java
new file mode 100644
index 0000000..6f75e0f
--- /dev/null
+++ b/java/com/google/gerrit/server/change/IncludedInUtil.java
@@ -0,0 +1,49 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.change;
+
+import static java.util.Comparator.comparing;
+import static java.util.stream.Collectors.toList;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import org.eclipse.jgit.lib.Ref;
+import org.eclipse.jgit.revwalk.RevWalk;
+
+public class IncludedInUtil {
+
+ /**
+ * Sorts the collection of {@code Ref} instances by its tip commit time.
+ *
+ * @param refs collection to be sorted
+ * @param revWalk {@code RevWalk} instance for parsing ref's tip commit
+ * @return sorted list of refs
+ */
+ public static List<Ref> getSortedRefs(Collection<Ref> refs, RevWalk revWalk) {
+ return refs.stream()
+ .sorted(
+ comparing(
+ ref -> {
+ try {
+ return revWalk.parseCommit(ref.getObjectId()).getCommitTime();
+ } catch (IOException e) {
+ // Ignore and continue to sort
+ }
+ return 0;
+ }))
+ .collect(toList());
+ }
+}
diff --git a/java/com/google/gerrit/server/config/AuthConfig.java b/java/com/google/gerrit/server/config/AuthConfig.java
index 1760378..b6ffcee 100644
--- a/java/com/google/gerrit/server/config/AuthConfig.java
+++ b/java/com/google/gerrit/server/config/AuthConfig.java
@@ -65,6 +65,7 @@
private final SignedToken emailReg;
private final boolean allowRegisterNewEmail;
private final boolean userNameCaseInsensitive;
+ private final boolean userNameCaseInsensitiveMigrationMode;
private GitBasicAuthPolicy gitBasicAuthPolicy;
@Inject
@@ -97,6 +98,8 @@
userNameToLowerCase = cfg.getBoolean("auth", "userNameToLowerCase", false);
allowRegisterNewEmail = cfg.getBoolean("auth", "allowRegisterNewEmail", true);
userNameCaseInsensitive = cfg.getBoolean("auth", "userNameCaseInsensitive", false);
+ userNameCaseInsensitiveMigrationMode =
+ cfg.getBoolean("auth", "userNameCaseInsensitiveMigrationMode", false);
if (gitBasicAuthPolicy == GitBasicAuthPolicy.HTTP_LDAP
&& authType != AuthType.LDAP
@@ -244,6 +247,11 @@
return userNameCaseInsensitive;
}
+ /** Whether user name case insensitive migration is in progress */
+ public boolean isUserNameCaseInsensitiveMigrationMode() {
+ return userNameCaseInsensitiveMigrationMode;
+ }
+
public GitBasicAuthPolicy getGitBasicAuthPolicy() {
return gitBasicAuthPolicy;
}
diff --git a/java/com/google/gerrit/server/events/EventGsonProvider.java b/java/com/google/gerrit/server/events/EventGsonProvider.java
index 27be2f3..bd784cf 100644
--- a/java/com/google/gerrit/server/events/EventGsonProvider.java
+++ b/java/com/google/gerrit/server/events/EventGsonProvider.java
@@ -29,8 +29,8 @@
.registerTypeAdapter(Event.class, new EventDeserializer())
.registerTypeAdapter(Supplier.class, new SupplierSerializer())
.registerTypeAdapter(Supplier.class, new SupplierDeserializer())
- .registerTypeAdapter(Project.NameKey.class, new ProjectNameKeyAdapter())
.registerTypeAdapterFactory(EntitiesAdapterFactory.create())
+ .registerTypeHierarchyAdapter(Project.NameKey.class, new ProjectNameKeyAdapter())
.create();
}
}
diff --git a/java/com/google/gerrit/server/git/PermissionAwareReadOnlyRefDatabase.java b/java/com/google/gerrit/server/git/PermissionAwareReadOnlyRefDatabase.java
index 99a66f8..f66a089 100644
--- a/java/com/google/gerrit/server/git/PermissionAwareReadOnlyRefDatabase.java
+++ b/java/com/google/gerrit/server/git/PermissionAwareReadOnlyRefDatabase.java
@@ -14,7 +14,6 @@
package com.google.gerrit.server.git;
-import static java.util.stream.Collectors.toList;
import static java.util.stream.Collectors.toMap;
import com.google.common.base.Preconditions;
@@ -108,20 +107,11 @@
return Iterables.getOnlyElement(result);
}
+ // WARNING: This method is deprecated in JGit's RefDatabase and it will be removed on master.
+ // Do not add any logic here but rather enrich the getRefsByPrefix method below.
@Override
public Map<String, Ref> getRefs(String prefix) throws IOException {
- List<Ref> refs = getDelegate().getRefDatabase().getRefsByPrefix(prefix);
- if (refs.isEmpty()) {
- return Collections.emptyMap();
- }
-
- Collection<Ref> result;
- try {
- result = forProject.filter(refs, getDelegate(), RefFilterOptions.defaults());
- } catch (PermissionBackendException e) {
- throw new IOException("", e);
- }
- return buildPrefixRefMap(prefix, result);
+ return buildPrefixRefMap(prefix, getRefsByPrefix(prefix));
}
private Map<String, Ref> buildPrefixRefMap(String prefix, Collection<Ref> refs) {
@@ -138,26 +128,18 @@
@Override
public List<Ref> getRefsByPrefix(String prefix) throws IOException {
- Map<String, Ref> coarseRefs;
- int lastSlash = prefix.lastIndexOf('/');
- if (lastSlash == -1) {
- coarseRefs = getRefs(ALL);
- } else {
- coarseRefs = getRefs(prefix.substring(0, lastSlash + 1));
+ List<Ref> refs = getDelegate().getRefDatabase().getRefsByPrefix(prefix);
+ if (refs.isEmpty()) {
+ return Collections.emptyList();
}
- List<Ref> result;
- if (lastSlash + 1 == prefix.length()) {
- result = coarseRefs.values().stream().collect(toList());
- } else {
- String p = prefix.substring(lastSlash + 1);
- result =
- coarseRefs.entrySet().stream()
- .filter(e -> e.getKey().startsWith(p))
- .map(e -> e.getValue())
- .collect(toList());
+ Collection<Ref> result;
+ try {
+ result = forProject.filter(refs, getDelegate(), RefFilterOptions.defaults());
+ } catch (PermissionBackendException e) {
+ throw new IOException("", e);
}
- return Collections.unmodifiableList(result);
+ return result.stream().collect(Collectors.toList());
}
@Override
diff --git a/java/com/google/gerrit/server/git/meta/VersionedMetaData.java b/java/com/google/gerrit/server/git/meta/VersionedMetaData.java
index a42ab8f..61bd8a8 100644
--- a/java/com/google/gerrit/server/git/meta/VersionedMetaData.java
+++ b/java/com/google/gerrit/server/git/meta/VersionedMetaData.java
@@ -210,6 +210,27 @@
}
/**
+ * Update this metadata branch, recording a new commit on its reference. This method mutates its
+ * receiver.
+ *
+ * @param update helper information to define the update that will occur.
+ * @param objInserter Shared object inserter.
+ * @param objReader Shared object reader.
+ * @param revWalk Shared rev walk.
+ * @return the commit that was created
+ * @throws IOException if there is a storage problem and the update cannot be executed as
+ * requested or if it failed because of a concurrent update to the same reference
+ */
+ public RevCommit commit(
+ MetaDataUpdate update, ObjectInserter objInserter, ObjectReader objReader, RevWalk revWalk)
+ throws IOException {
+ try (BatchMetaDataUpdate batch = openUpdate(update, objInserter, objReader, revWalk)) {
+ batch.write(update.getCommitBuilder());
+ return batch.commit();
+ }
+ }
+
+ /**
* Creates a new commit and a new ref based on this commit. This method mutates its receiver.
*
* @param update helper information to define the update that will occur.
@@ -256,11 +277,39 @@
* @throws IOException if the update failed.
*/
public BatchMetaDataUpdate openUpdate(MetaDataUpdate update) throws IOException {
+ return openUpdate(update, null, null, null);
+ }
+
+ /**
+ * Open a batch of updates to the same metadata ref.
+ *
+ * <p>This allows making multiple commits to a single metadata ref, at the end of which is a
+ * single ref update. For batching together updates to multiple refs (each consisting of one or
+ * more commits against their respective refs), create the {@link MetaDataUpdate} with a {@link
+ * BatchRefUpdate}.
+ *
+ * <p>A ref update produced by this {@link BatchMetaDataUpdate} is only committed if there is no
+ * associated {@link BatchRefUpdate}. As a result, the configured ref updated event is not fired
+ * if there is an associated batch.
+ *
+ * <p>If object inserter, reader and revwalk are provided, then the updates are not flushed,
+ * allowing callers the flexibility to flush only once after several updates.
+ *
+ * @param update helper info about the update.
+ * @param objInserter Shared object inserter.
+ * @param objReader Shared object reader.
+ * @param revWalk Shared rev walk.
+ * @throws IOException if the update failed.
+ */
+ public BatchMetaDataUpdate openUpdate(
+ MetaDataUpdate update, ObjectInserter objInserter, ObjectReader objReader, RevWalk revWalk)
+ throws IOException {
final Repository db = update.getRepository();
- inserter = db.newObjectInserter();
- reader = inserter.newReader();
- final RevWalk rw = new RevWalk(reader);
+ inserter = objInserter == null ? db.newObjectInserter() : objInserter;
+ reader = objReader == null ? inserter.newReader() : objReader;
+ final RevWalk rw = revWalk == null ? new RevWalk(reader) : revWalk;
+
final RevTree tree = revision != null ? rw.parseTree(revision) : null;
newTree = readTree(tree);
return new BatchMetaDataUpdate() {
@@ -372,13 +421,16 @@
public void close() {
newTree = null;
- rw.close();
- if (inserter != null) {
+ if (revWalk == null) {
+ rw.close();
+ }
+
+ if (objInserter == null && inserter != null) {
inserter.close();
inserter = null;
}
- if (reader != null) {
+ if (objReader == null && reader != null) {
reader.close();
reader = null;
}
@@ -389,7 +441,9 @@
BatchRefUpdate bru = update.getBatch();
if (bru != null) {
bru.addCommand(new ReceiveCommand(oldId.toObjectId(), newId.toObjectId(), refName));
- inserter.flush();
+ if (objInserter == null) {
+ inserter.flush();
+ }
revision = rw.parseCommit(newId);
return revision;
}
diff --git a/java/com/google/gerrit/server/git/receive/ReceiveCommits.java b/java/com/google/gerrit/server/git/receive/ReceiveCommits.java
index d7f1bb2..89c025a 100644
--- a/java/com/google/gerrit/server/git/receive/ReceiveCommits.java
+++ b/java/com/google/gerrit/server/git/receive/ReceiveCommits.java
@@ -2864,7 +2864,7 @@
try (TraceTimer traceTimer = newTimer("readChangesForReplace")) {
replaceByChange.values().stream()
.map(r -> r.ontoChange)
- .map(id -> notesFactory.create(project.getNameKey(), id))
+ .map(id -> notesFactory.create(repo, project.getNameKey(), id))
.forEach(notes -> replaceByChange.get(notes.getChangeId()).notes = notes);
}
}
@@ -3285,7 +3285,7 @@
}
if (isConfig(cmd)) {
logger.atFine().log("Reloading project in cache");
- projectCache.evict(project);
+ projectCache.evictAndReindex(project);
ProjectState ps =
projectCache.get(project.getNameKey()).orElseThrow(illegalState(project.getNameKey()));
try {
diff --git a/java/com/google/gerrit/server/group/db/RenameGroupOp.java b/java/com/google/gerrit/server/group/db/RenameGroupOp.java
index 843b346..257bc16 100644
--- a/java/com/google/gerrit/server/group/db/RenameGroupOp.java
+++ b/java/com/google/gerrit/server/group/db/RenameGroupOp.java
@@ -123,7 +123,7 @@
//
GroupReference ref = config.getGroup(uuid);
if (ref == null || newName.equals(ref.getName())) {
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
return;
}
@@ -132,7 +132,7 @@
md.setMessage("Rename group " + oldName + " to " + newName + "\n");
try {
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
success = true;
} catch (IOException e) {
logger.atSevere().withCause(e).log(
diff --git a/java/com/google/gerrit/server/index/AbstractIndexModule.java b/java/com/google/gerrit/server/index/AbstractIndexModule.java
index 352971f..81c517f 100644
--- a/java/com/google/gerrit/server/index/AbstractIndexModule.java
+++ b/java/com/google/gerrit/server/index/AbstractIndexModule.java
@@ -33,6 +33,7 @@
* index implementations, such as {@link com.google.gerrit.lucene.LuceneIndexModule}.
*/
public abstract class AbstractIndexModule extends AbstractModule {
+ public static final String INDEX_MODULE = "index-module";
private final int threads;
private final Map<String, Integer> singleVersions;
diff --git a/java/com/google/gerrit/server/index/IndexModule.java b/java/com/google/gerrit/server/index/IndexModule.java
index a2ef070..9ad7cdb 100644
--- a/java/com/google/gerrit/server/index/IndexModule.java
+++ b/java/com/google/gerrit/server/index/IndexModule.java
@@ -53,6 +53,7 @@
import com.google.gerrit.server.index.group.GroupIndexer;
import com.google.gerrit.server.index.group.GroupIndexerImpl;
import com.google.gerrit.server.index.group.GroupSchemaDefinitions;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.gerrit.server.index.project.ProjectIndexDefinition;
import com.google.gerrit.server.index.project.ProjectIndexerImpl;
import com.google.inject.Inject;
@@ -150,6 +151,9 @@
}
DynamicSet.setOf(binder(), OnlineUpgradeListener.class);
+ OptionalBinder.newOptionalBinder(binder(), IsFirstInsertForEntry.class)
+ .setDefault()
+ .toInstance(IsFirstInsertForEntry.NO);
}
@Provides
@@ -215,13 +219,14 @@
return interactiveExecutor;
}
int threads = this.threads;
- if (threads < 0) {
- return MoreExecutors.newDirectExecutorService();
- } else if (threads == 0) {
+ if (threads == 0) {
threads =
config.getInt(
"index", null, "threads", Runtime.getRuntime().availableProcessors() / 2 + 1);
}
+ if (threads < 0) {
+ return MoreExecutors.newDirectExecutorService();
+ }
return MoreExecutors.listeningDecorator(
workQueue.createQueue(threads, "Index-Interactive", true));
}
@@ -234,11 +239,13 @@
if (batchExecutor != null) {
return batchExecutor;
}
- int threads = config.getInt("index", null, "batchThreads", 0);
+ int threads = this.threads;
+ if (threads == 0) {
+ threads =
+ config.getInt("index", null, "batchThreads", Runtime.getRuntime().availableProcessors());
+ }
if (threads < 0) {
return MoreExecutors.newDirectExecutorService();
- } else if (threads == 0) {
- threads = Runtime.getRuntime().availableProcessors();
}
return MoreExecutors.listeningDecorator(workQueue.createQueue(threads, "Index-Batch", true));
}
diff --git a/java/com/google/gerrit/server/index/account/AllAccountsIndexer.java b/java/com/google/gerrit/server/index/account/AllAccountsIndexer.java
index 63889b7..1f48e35 100644
--- a/java/com/google/gerrit/server/index/account/AllAccountsIndexer.java
+++ b/java/com/google/gerrit/server/index/account/AllAccountsIndexer.java
@@ -27,6 +27,7 @@
import com.google.gerrit.server.account.AccountState;
import com.google.gerrit.server.account.Accounts;
import com.google.gerrit.server.index.IndexExecutor;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import java.io.IOException;
@@ -50,15 +51,18 @@
private final ListeningExecutorService executor;
private final Accounts accounts;
private final AccountCache accountCache;
+ private final IsFirstInsertForEntry isFirstInsertForEntry;
@Inject
AllAccountsIndexer(
@IndexExecutor(BATCH) ListeningExecutorService executor,
Accounts accounts,
- AccountCache accountCache) {
+ AccountCache accountCache,
+ IsFirstInsertForEntry isFirstInsertForEntry) {
this.executor = executor;
this.accounts = accounts;
this.accountCache = accountCache;
+ this.isFirstInsertForEntry = isFirstInsertForEntry;
}
@Override
@@ -92,7 +96,11 @@
try {
Optional<AccountState> a = accountCache.get(id);
if (a.isPresent()) {
- index.replace(a.get());
+ if (isFirstInsertForEntry.equals(IsFirstInsertForEntry.YES)) {
+ index.insert(a.get());
+ } else {
+ index.replace(a.get());
+ }
} else {
index.delete(id);
}
diff --git a/java/com/google/gerrit/server/index/change/AllChangesIndexer.java b/java/com/google/gerrit/server/index/change/AllChangesIndexer.java
index f3ed758..3f71d39 100644
--- a/java/com/google/gerrit/server/index/change/AllChangesIndexer.java
+++ b/java/com/google/gerrit/server/index/change/AllChangesIndexer.java
@@ -20,15 +20,16 @@
import static com.google.common.util.concurrent.MoreExecutors.directExecutor;
import static com.google.gerrit.server.git.QueueProvider.QueueType.BATCH;
+import com.google.auto.value.AutoValue;
import com.google.common.base.Stopwatch;
+import com.google.common.collect.Sets;
import com.google.common.flogger.FluentLogger;
-import com.google.common.primitives.Ints;
+import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.ListenableFuture;
import com.google.common.util.concurrent.ListeningExecutorService;
import com.google.common.util.concurrent.UncheckedExecutionException;
import com.google.gerrit.entities.Change;
import com.google.gerrit.entities.Project;
-import com.google.gerrit.entities.RefNames;
import com.google.gerrit.index.SiteIndexer;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MultiProgressMonitor;
@@ -38,6 +39,7 @@
import com.google.gerrit.server.index.OnlineReindexMode;
import com.google.gerrit.server.notedb.ChangeNotes;
import com.google.gerrit.server.notedb.ChangeNotes.Factory.ChangeNotesResult;
+import com.google.gerrit.server.notedb.ChangeNotes.Factory.ScanResult;
import com.google.gerrit.server.project.ProjectCache;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.inject.Inject;
@@ -45,11 +47,13 @@
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
-import java.util.Objects;
+import java.util.Set;
import java.util.concurrent.Callable;
+import java.util.concurrent.ExecutionException;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.atomic.AtomicBoolean;
-import org.eclipse.jgit.errors.RepositoryNotFoundException;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
import org.eclipse.jgit.lib.ProgressMonitor;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.lib.TextProgressMonitor;
@@ -64,6 +68,15 @@
private static final int PROJECT_SLICE_MAX_REFS = 1000;
private final MultiProgressMonitor.Factory multiProgressMonitorFactory;
+
+ private static class ProjectsCollectionFailure extends Exception {
+ private static final long serialVersionUID = 1L;
+
+ public ProjectsCollectionFailure(String message) {
+ super(message);
+ }
+ }
+
private final ChangeData.Factory changeDataFactory;
private final GitRepositoryManager repoManager;
private final ListeningExecutorService executor;
@@ -89,99 +102,58 @@
this.projectCache = projectCache;
}
- private static class ProjectSlice {
- private final Project.NameKey name;
- private final int slice;
- private final int slices;
+ @AutoValue
+ public abstract static class ProjectSlice {
+ public abstract Project.NameKey name();
- ProjectSlice(Project.NameKey name, int slice, int slices) {
- this.name = name;
- this.slice = slice;
- this.slices = slices;
- }
+ public abstract int slice();
- public Project.NameKey getName() {
- return name;
- }
+ public abstract int slices();
- public int getSlice() {
- return slice;
- }
+ public abstract ScanResult scanResult();
- public int getSlices() {
- return slices;
+ private static ProjectSlice create(Project.NameKey name, int slice, int slices, ScanResult sr) {
+ return new AutoValue_AllChangesIndexer_ProjectSlice(name, slice, slices, sr);
}
}
@Override
public Result indexAll(ChangeIndex index) {
- ProgressMonitor pm = new TextProgressMonitor();
- pm.beginTask("Collecting projects", ProgressMonitor.UNKNOWN);
- List<ProjectSlice> projectSlices = new ArrayList<>();
- int changeCount = 0;
- Stopwatch sw = Stopwatch.createStarted();
- int projectsFailed = 0;
- for (Project.NameKey name : projectCache.all()) {
- try (Repository repo = repoManager.openRepository(name)) {
- // The simplest approach to distribute indexing would be to let each thread grab a project
- // and index it fully. But if a site has one big project and 100s of small projects, then
- // in the beginning all CPUs would be busy reindexing projects. But soon enough all small
- // projects have been reindexed, and only the thread that reindexes the big project is
- // still working. The other threads would idle. Reindexing the big project on a single
- // thread becomes the critical path. Bringing in more CPUs would not speed up things.
- //
- // To avoid such situations, we split big repos into smaller parts and let
- // the thread pool index these smaller parts. This splitting introduces an overhead in the
- // workload setup and there might be additional slow-downs from multiple threads
- // concurrently working on different parts of the same project. But for Wikimedia's Gerrit,
- // which had 2 big projects, many middle sized ones, and lots of smaller ones, the
- // splitting of repos into smaller parts reduced indexing time from 1.5 hours to 55 minutes
- // in 2020.
- int size = estimateSize(repo);
- changeCount += size;
- int slices = 1 + size / PROJECT_SLICE_MAX_REFS;
- if (slices > 1) {
- verboseWriter.println("Submitting " + name + " for indexing in " + slices + " slices");
- }
- for (int slice = 0; slice < slices; slice++) {
- projectSlices.add(new ProjectSlice(name, slice, slices));
- }
- } catch (IOException e) {
- logger.atSevere().withCause(e).log("Error collecting project %s", name);
- projectsFailed++;
- if (projectsFailed > projectCache.all().size() / 2) {
- logger.atSevere().log("Over 50%% of the projects could not be collected: aborted");
- return Result.create(sw, false, 0, 0);
- }
- }
- pm.update(1);
- }
- pm.endTask();
- setTotalWork(changeCount);
+ // The simplest approach to distribute indexing would be to let each thread grab a project
+ // and index it fully. But if a site has one big project and 100s of small projects, then
+ // in the beginning all CPUs would be busy reindexing projects. But soon enough all small
+ // projects have been reindexed, and only the thread that reindexes the big project is
+ // still working. The other threads would idle. Reindexing the big project on a single
+ // thread becomes the critical path. Bringing in more CPUs would not speed up things.
+ //
+ // To avoid such situations, we split big repos into smaller parts and let
+ // the thread pool index these smaller parts. This splitting introduces an overhead in the
+ // workload setup and there might be additional slow-downs from multiple threads
+ // concurrently working on different parts of the same project. But for Wikimedia's Gerrit,
+ // which had 2 big projects, many middle sized ones, and lots of smaller ones, the
+ // splitting of repos into smaller parts reduced indexing time from 1.5 hours to 55 minutes
+ // in 2020.
- // projectSlices are currently grouped by projects. First all slices for project1, followed
- // by all slices for project2, and so on. As workers pick tasks sequentially, multiple threads
- // would typically work concurrently on different slices of the same project. While this is not
- // a big issue, shuffling the list beforehand helps with ungrouping the project slices, so
- // different slices are less likely to be worked on concurrently.
+ Stopwatch sw = Stopwatch.createStarted();
+ List<ProjectSlice> projectSlices;
+ try {
+ projectSlices = new SliceCreator().create();
+ } catch (ProjectsCollectionFailure | InterruptedException | ExecutionException e) {
+ logger.atSevere().log(e.getMessage());
+ return Result.create(sw, false, 0, 0);
+ }
+
+ // Since project slices are created in parallel, they are somewhat shuffled already. However,
+ // the number of threads used to create the project slices doesn't guarantee good randomization.
+ // If the slices are not shuffled well, then multiple threads would typically work concurrently
+ // on different slices of the same project. While this is not a big issue, shuffling the list
+ // beforehand helps with ungrouping the project slices, so different slices are less likely to
+ // be worked on concurrently.
// This shuffling gave a 6% runtime reduction for Wikimedia's Gerrit in 2020.
Collections.shuffle(projectSlices);
return indexAll(index, projectSlices);
}
- private int estimateSize(Repository repo) throws IOException {
- // Estimate size based on IDs that show up in ref names. This is not perfect, since patch set
- // refs may exist for changes whose metadata was never successfully stored. But that's ok, as
- // the estimate is just used as a heuristic for sorting projects.
- long size =
- repo.getRefDatabase().getRefsByPrefix(RefNames.REFS_CHANGES).stream()
- .map(r -> Change.Id.fromRef(r.getName()))
- .filter(Objects::nonNull)
- .distinct()
- .count();
- return Ints.saturatedCast(size);
- }
-
private SiteIndexer.Result indexAll(ChangeIndex index, List<ProjectSlice> projectSlices) {
Stopwatch sw = Stopwatch.createStarted();
MultiProgressMonitor mpm =
@@ -195,9 +167,9 @@
AtomicBoolean ok = new AtomicBoolean(true);
for (ProjectSlice projectSlice : projectSlices) {
- Project.NameKey name = projectSlice.getName();
- int slice = projectSlice.getSlice();
- int slices = projectSlice.getSlices();
+ Project.NameKey name = projectSlice.name();
+ int slice = projectSlice.slice();
+ int slices = projectSlice.slices();
ListenableFuture<?> future =
executor.submit(
reindexProject(
@@ -205,6 +177,7 @@
name,
slice,
slices,
+ projectSlice.scanResult(),
doneTask,
failedTask));
String description = "project " + name + " (" + slice + "/" + slices + ")";
@@ -237,13 +210,21 @@
"Failed %s/%s changes (%s%%); not marking new index as ready",
nFailed, nTotal, Math.round(pctFailed));
ok.set(false);
+ } else if (nFailed > 0) {
+ logger.atWarning().log("Failed %s/%s changes", nFailed, nTotal);
}
return Result.create(sw, ok.get(), nDone, nFailed);
}
public Callable<Void> reindexProject(
ChangeIndexer indexer, Project.NameKey project, Task done, Task failed) {
- return reindexProject(indexer, project, 0, 1, done, failed);
+ try (Repository repo = repoManager.openRepository(project)) {
+ return reindexProject(
+ indexer, project, 0, 1, ChangeNotes.Factory.scanChangeIds(repo), done, failed);
+ } catch (IOException e) {
+ logger.atSevere().log(e.getMessage());
+ return null;
+ }
}
public Callable<Void> reindexProject(
@@ -251,9 +232,10 @@
Project.NameKey project,
int slice,
int slices,
+ ScanResult scanResult,
Task done,
Task failed) {
- return new ProjectIndexer(indexer, project, slice, slices, done, failed);
+ return new ProjectIndexer(indexer, project, slice, slices, scanResult, done, failed);
}
private class ProjectIndexer implements Callable<Void> {
@@ -261,6 +243,7 @@
private final Project.NameKey project;
private final int slice;
private final int slices;
+ private final ScanResult scanResult;
private final ProgressMonitor done;
private final ProgressMonitor failed;
@@ -269,32 +252,30 @@
Project.NameKey project,
int slice,
int slices,
+ ScanResult scanResult,
ProgressMonitor done,
ProgressMonitor failed) {
this.indexer = indexer;
this.project = project;
this.slice = slice;
this.slices = slices;
+ this.scanResult = scanResult;
this.done = done;
this.failed = failed;
}
@Override
public Void call() throws Exception {
- try (Repository repo = repoManager.openRepository(project)) {
- OnlineReindexMode.begin();
-
- // Order of scanning changes is undefined. This is ok if we assume that packfile locality is
- // not important for indexing, since sites should have a fully populated DiffSummary cache.
- // It does mean that reindexing after invalidating the DiffSummary cache will be expensive,
- // but the goal is to invalidate that cache as infrequently as we possibly can. And besides,
- // we don't have concrete proof that improving packfile locality would help.
- notesFactory.scan(repo, project, id -> (id.get() % slices) == slice).forEach(r -> index(r));
- } catch (RepositoryNotFoundException rnfe) {
- logger.atSevere().log("%s", rnfe.getMessage());
- } finally {
- OnlineReindexMode.end();
- }
+ OnlineReindexMode.begin();
+ // Order of scanning changes is undefined. This is ok if we assume that packfile locality is
+ // not important for indexing, since sites should have a fully populated DiffSummary cache.
+ // It does mean that reindexing after invalidating the DiffSummary cache will be expensive,
+ // but the goal is to invalidate that cache as infrequently as we possibly can. And besides,
+ // we don't have concrete proof that improving packfile locality would help.
+ notesFactory
+ .scan(scanResult, project, id -> (id.get() % slices) == slice)
+ .forEach(r -> index(r));
+ OnlineReindexMode.end();
return null;
}
@@ -334,4 +315,63 @@
return "Index all changes of project " + project.get();
}
}
+
+ private class SliceCreator {
+ final Set<ProjectSlice> projectSlices = Sets.newConcurrentHashSet();
+ final AtomicInteger changeCount = new AtomicInteger(0);
+ final AtomicInteger projectsFailed = new AtomicInteger(0);
+ final ProgressMonitor pm = new TextProgressMonitor();
+
+ private List<ProjectSlice> create()
+ throws ProjectsCollectionFailure, InterruptedException, ExecutionException {
+ List<ListenableFuture<?>> futures = new ArrayList<>();
+ pm.beginTask("Collecting projects", ProgressMonitor.UNKNOWN);
+ for (Project.NameKey name : projectCache.all()) {
+ futures.add(executor.submit(new ProjectSliceCreator(name)));
+ }
+
+ Futures.allAsList(futures).get();
+
+ if (projectsFailed.get() > projectCache.all().size() / 2) {
+ throw new ProjectsCollectionFailure(
+ "Over 50%% of the projects could not be collected: aborted");
+ }
+
+ pm.endTask();
+ setTotalWork(changeCount.get());
+ return projectSlices.stream().collect(Collectors.toList());
+ }
+
+ private class ProjectSliceCreator implements Callable<Void> {
+ final Project.NameKey name;
+
+ public ProjectSliceCreator(Project.NameKey name) {
+ this.name = name;
+ }
+
+ @Override
+ public Void call() throws IOException {
+ try (Repository repo = repoManager.openRepository(name)) {
+ ScanResult sr = ChangeNotes.Factory.scanChangeIds(repo);
+ int size = sr.all().size();
+ if (size > 0) {
+ changeCount.addAndGet(size);
+ int slices = 1 + size / PROJECT_SLICE_MAX_REFS;
+ if (slices > 1) {
+ verboseWriter.println(
+ "Submitting " + name + " for indexing in " + slices + " slices");
+ }
+ for (int slice = 0; slice < slices; slice++) {
+ projectSlices.add(ProjectSlice.create(name, slice, slices, sr));
+ }
+ }
+ } catch (IOException e) {
+ logger.atSevere().withCause(e).log("Error collecting project %s", name);
+ projectsFailed.incrementAndGet();
+ }
+ pm.update(1);
+ return null;
+ }
+ }
+ }
}
diff --git a/java/com/google/gerrit/server/index/change/ChangeField.java b/java/com/google/gerrit/server/index/change/ChangeField.java
index 21071f8..6c8e3fb 100644
--- a/java/com/google/gerrit/server/index/change/ChangeField.java
+++ b/java/com/google/gerrit/server/index/change/ChangeField.java
@@ -913,14 +913,22 @@
intRange(ChangeQueryBuilder.FIELD_ADDED)
.build(
cd -> cd.changedLines().isPresent() ? cd.changedLines().get().insertions : null,
- (cd, field) -> cd.setLinesInserted(field));
+ (cd, field) -> {
+ if (field != null) {
+ cd.setLinesInserted(field);
+ }
+ });
/** The number of deleted lines in this change. */
public static final FieldDef<ChangeData, Integer> DELETED =
intRange(ChangeQueryBuilder.FIELD_DELETED)
.build(
cd -> cd.changedLines().isPresent() ? cd.changedLines().get().deletions : null,
- (cd, field) -> cd.setLinesDeleted(field));
+ (cd, field) -> {
+ if (field != null) {
+ cd.setLinesDeleted(field);
+ }
+ });
/** The total number of modified lines in this change. */
public static final FieldDef<ChangeData, Integer> DELTA =
diff --git a/java/com/google/gerrit/server/index/change/ChangeIndexer.java b/java/com/google/gerrit/server/index/change/ChangeIndexer.java
index f7f0f33..8f68904 100644
--- a/java/com/google/gerrit/server/index/change/ChangeIndexer.java
+++ b/java/com/google/gerrit/server/index/change/ChangeIndexer.java
@@ -30,6 +30,7 @@
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.index.IndexExecutor;
import com.google.gerrit.server.index.StalenessCheckResult;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.gerrit.server.logging.Metadata;
import com.google.gerrit.server.logging.TraceContext;
import com.google.gerrit.server.logging.TraceContext.TraceTimer;
@@ -79,6 +80,7 @@
private final PluginSetContext<ChangeIndexedListener> indexedListeners;
private final StalenessChecker stalenessChecker;
private final boolean autoReindexIfStale;
+ private final IsFirstInsertForEntry isFirstInsertForEntry;
private final Map<Change.Id, IndexTask> queuedIndexTasks = new ConcurrentHashMap<>();
private final Set<ReindexIfStaleTask> queuedReindexIfStaleTasks =
@@ -94,7 +96,8 @@
StalenessChecker stalenessChecker,
@IndexExecutor(BATCH) ListeningExecutorService batchExecutor,
@Assisted ListeningExecutorService executor,
- @Assisted ChangeIndex index) {
+ @Assisted ChangeIndex index,
+ IsFirstInsertForEntry isFirstInsertForEntry) {
this.executor = executor;
this.changeDataFactory = changeDataFactory;
this.notesFactory = notesFactory;
@@ -105,6 +108,7 @@
this.autoReindexIfStale = autoReindexIfStale(cfg);
this.index = index;
this.indexes = null;
+ this.isFirstInsertForEntry = isFirstInsertForEntry;
}
@AssistedInject
@@ -117,7 +121,8 @@
StalenessChecker stalenessChecker,
@IndexExecutor(BATCH) ListeningExecutorService batchExecutor,
@Assisted ListeningExecutorService executor,
- @Assisted ChangeIndexCollection indexes) {
+ @Assisted ChangeIndexCollection indexes,
+ IsFirstInsertForEntry isFirstInsertForEntry) {
this.executor = executor;
this.changeDataFactory = changeDataFactory;
this.notesFactory = notesFactory;
@@ -128,6 +133,7 @@
this.autoReindexIfStale = autoReindexIfStale(cfg);
this.index = null;
this.indexes = indexes;
+ this.isFirstInsertForEntry = isFirstInsertForEntry;
}
private static boolean autoReindexIfStale(Config cfg) {
@@ -227,21 +233,25 @@
}
private void indexImpl(ChangeData cd) {
- logger.atFine().log("Replace change %d in index.", cd.getId().get());
+ logger.atFine().log("Reindex change %d in index.", cd.getId().get());
for (Index<?, ChangeData> i : getWriteIndexes()) {
try (TraceTimer traceTimer =
TraceContext.newTimer(
- "Replacing change in index",
+ "Reindexing change in index",
Metadata.builder()
.changeId(cd.getId().get())
.patchSetId(cd.currentPatchSet().number())
.indexVersion(i.getSchema().getVersion())
.build())) {
- i.replace(cd);
+ if (isFirstInsertForEntry.equals(IsFirstInsertForEntry.YES)) {
+ i.insert(cd);
+ } else {
+ i.replace(cd);
+ }
} catch (RuntimeException e) {
throw new StorageException(
String.format(
- "Failed to replace change %d in index version %d (current patch set = %d)",
+ "Failed to reindex change %d in index version %d (current patch set = %d)",
cd.getId().get(), i.getSchema().getVersion(), cd.currentPatchSet().number()),
e);
}
diff --git a/java/com/google/gerrit/server/index/group/AllGroupsIndexer.java b/java/com/google/gerrit/server/index/group/AllGroupsIndexer.java
index b3ef679..3773d435 100644
--- a/java/com/google/gerrit/server/index/group/AllGroupsIndexer.java
+++ b/java/com/google/gerrit/server/index/group/AllGroupsIndexer.java
@@ -30,6 +30,7 @@
import com.google.gerrit.server.group.db.Groups;
import com.google.gerrit.server.group.db.GroupsNoteDbConsistencyChecker;
import com.google.gerrit.server.index.IndexExecutor;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import java.io.IOException;
@@ -54,15 +55,18 @@
private final ListeningExecutorService executor;
private final GroupCache groupCache;
private final Groups groups;
+ private final IsFirstInsertForEntry isFirstInsertForEntry;
@Inject
AllGroupsIndexer(
@IndexExecutor(BATCH) ListeningExecutorService executor,
GroupCache groupCache,
- Groups groups) {
+ Groups groups,
+ IsFirstInsertForEntry isFirstInsertForEntry) {
this.executor = executor;
this.groupCache = groupCache;
this.groups = groups;
+ this.isFirstInsertForEntry = isFirstInsertForEntry;
}
@Override
@@ -96,9 +100,14 @@
executor.submit(
() -> {
try {
+ groupCache.evict(uuid);
InternalGroup internalGroup = reindexedGroups.get(uuid);
if (internalGroup != null) {
- index.replace(internalGroup);
+ if (isFirstInsertForEntry.equals(IsFirstInsertForEntry.YES)) {
+ index.insert(internalGroup);
+ } else {
+ index.replace(internalGroup);
+ }
} else {
index.delete(uuid);
diff --git a/java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java b/java/com/google/gerrit/server/index/options/AutoFlush.java
similarity index 69%
rename from java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java
rename to java/com/google/gerrit/server/index/options/AutoFlush.java
index 6451b0f..7b82edb 100644
--- a/java/com/google/gerrit/elasticsearch/bulk/DeleteRequest.java
+++ b/java/com/google/gerrit/server/index/options/AutoFlush.java
@@ -1,4 +1,4 @@
-// Copyright (C) 2018 The Android Open Source Project
+// Copyright (C) 2021 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,11 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
-package com.google.gerrit.elasticsearch.bulk;
+package com.google.gerrit.server.index.options;
-public class DeleteRequest extends ActionRequest {
-
- public DeleteRequest(String id, String index) {
- super("delete", id, index);
- }
+public enum AutoFlush {
+ ENABLED,
+ DISABLED
}
diff --git a/java/com/google/gerrit/server/index/options/IsFirstInsertForEntry.java b/java/com/google/gerrit/server/index/options/IsFirstInsertForEntry.java
new file mode 100644
index 0000000..f943309
--- /dev/null
+++ b/java/com/google/gerrit/server/index/options/IsFirstInsertForEntry.java
@@ -0,0 +1,26 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.index.options;
+
+/**
+ * This enum should be injected and checked to decide on which operation ({@link
+ * com.google.gerrit.index.Index#replace(Object) replace()} or {@link
+ * com.google.gerrit.index.Index#insert(Object) insert()}) should be performed on a specific {@link
+ * com.google.gerrit.index.Index index}.
+ */
+public enum IsFirstInsertForEntry {
+ YES,
+ NO
+}
diff --git a/java/com/google/gerrit/server/index/project/AllProjectsIndexer.java b/java/com/google/gerrit/server/index/project/AllProjectsIndexer.java
index 0e4b688..1c977d1 100644
--- a/java/com/google/gerrit/server/index/project/AllProjectsIndexer.java
+++ b/java/com/google/gerrit/server/index/project/AllProjectsIndexer.java
@@ -27,6 +27,7 @@
import com.google.gerrit.index.project.ProjectData;
import com.google.gerrit.index.project.ProjectIndex;
import com.google.gerrit.server.index.IndexExecutor;
+import com.google.gerrit.server.index.options.IsFirstInsertForEntry;
import com.google.gerrit.server.project.ProjectCache;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@@ -48,12 +49,16 @@
private final ListeningExecutorService executor;
private final ProjectCache projectCache;
+ private final IsFirstInsertForEntry isFirstInsertForEntry;
@Inject
AllProjectsIndexer(
- @IndexExecutor(BATCH) ListeningExecutorService executor, ProjectCache projectCache) {
+ @IndexExecutor(BATCH) ListeningExecutorService executor,
+ ProjectCache projectCache,
+ IsFirstInsertForEntry isFirstInsertForEntry) {
this.executor = executor;
this.projectCache = projectCache;
+ this.isFirstInsertForEntry = isFirstInsertForEntry;
}
@Override
@@ -79,8 +84,13 @@
() -> {
try {
projectCache.evict(name);
- index.replace(
- projectCache.get(name).orElseThrow(illegalState(name)).toProjectData());
+ ProjectData projectData =
+ projectCache.get(name).orElseThrow(illegalState(name)).toProjectData();
+ if (isFirstInsertForEntry.equals(IsFirstInsertForEntry.YES)) {
+ index.insert(projectData);
+ } else {
+ index.replace(projectData);
+ }
verboseWriter.println("Reindexed " + desc);
done.incrementAndGet();
} catch (Exception e) {
diff --git a/java/com/google/gerrit/server/mail/send/OutgoingEmail.java b/java/com/google/gerrit/server/mail/send/OutgoingEmail.java
index 33fd09c..d2e878f 100644
--- a/java/com/google/gerrit/server/mail/send/OutgoingEmail.java
+++ b/java/com/google/gerrit/server/mail/send/OutgoingEmail.java
@@ -469,9 +469,13 @@
* username. If no username is set, this function returns null.
*
* @param accountId user to fetch.
- * @return name/email of account, username, or null if unset.
+ * @return name/email of account, username, or null if unset or the accountId is null.
*/
- protected String getUserNameEmailFor(Account.Id accountId) {
+ protected String getUserNameEmailFor(@Nullable Account.Id accountId) {
+ if (accountId == null) {
+ return null;
+ }
+
Optional<AccountState> accountState = args.accountCache.get(accountId);
if (!accountState.isPresent()) {
return null;
diff --git a/java/com/google/gerrit/server/notedb/AbstractChangeNotes.java b/java/com/google/gerrit/server/notedb/AbstractChangeNotes.java
index d71f9ff..158972f 100644
--- a/java/com/google/gerrit/server/notedb/AbstractChangeNotes.java
+++ b/java/com/google/gerrit/server/notedb/AbstractChangeNotes.java
@@ -37,8 +37,6 @@
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
-import org.eclipse.jgit.revwalk.RevCommit;
-import org.eclipse.jgit.revwalk.RevWalk;
/** View of contents at a single ref related to some change. * */
public abstract class AbstractChangeNotes<T> {
@@ -140,6 +138,15 @@
}
public T load() {
+ try (Repository repo = args.repoManager.openRepository(getProjectName())) {
+ load(repo);
+ return self();
+ } catch (IOException e) {
+ throw new StorageException(e);
+ }
+ }
+
+ public T load(Repository repo) {
if (loaded) {
return self();
}
@@ -148,7 +155,6 @@
throw new StorageException("Reading from NoteDb is disabled");
}
try (Timer0.Context timer = args.metrics.readLatency.start();
- Repository repo = args.repoManager.openRepository(getProjectName());
// Call openHandle even if reading is disabled, to trigger
// auto-rebuilding before this object may get passed to a ChangeUpdate.
LoadHandle handle = openHandle(repo, revision)) {
@@ -173,17 +179,16 @@
* <p>Implementations may override this method to provide auto-rebuilding behavior.
*
* @param repo open repository.
+ * @param id SHA1 of the entity to read from the repository. The SHA1 is not sanity checked and is
+ * assumed to be valid. If null, lookup SHA1 from the /meta ref.
* @return handle for reading the entity.
* @throws NoSuchChangeException change does not exist.
- * @throws MissingMetaObjectException specified SHA1 isn't reachable from meta branch.
* @throws IOException a repo-level error occurred.
*/
protected LoadHandle openHandle(Repository repo, @Nullable ObjectId id)
- throws NoSuchChangeException, IOException, MissingMetaObjectException {
+ throws NoSuchChangeException, IOException {
if (id == null) {
id = readRef(repo);
- } else {
- verifyMetaId(repo, id);
}
return new LoadHandle(repo, id);
@@ -226,20 +231,4 @@
protected final T self() {
return (T) this;
}
-
- private void verifyMetaId(Repository repo, ObjectId id)
- throws IOException, MissingMetaObjectException {
- try (RevWalk rw = new RevWalk(repo)) {
- Ref ref = repo.getRefDatabase().exactRef(getRefName());
- RevCommit tip = rw.parseCommit(ref.getObjectId());
- rw.markStart(tip);
- for (RevCommit rev : rw) {
- if (id.equals(rev)) {
- return;
- }
- }
- }
-
- throw new MissingMetaObjectException(id.getName() + " not reachable from " + getRefName());
- }
}
diff --git a/java/com/google/gerrit/server/notedb/ChangeNotes.java b/java/com/google/gerrit/server/notedb/ChangeNotes.java
index bbfe8dd..3dae2b8 100644
--- a/java/com/google/gerrit/server/notedb/ChangeNotes.java
+++ b/java/com/google/gerrit/server/notedb/ChangeNotes.java
@@ -69,12 +69,14 @@
import java.sql.Timestamp;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.function.Predicate;
import java.util.stream.Stream;
import org.eclipse.jgit.errors.ConfigInvalidException;
+import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
@@ -113,11 +115,43 @@
this.projectCache = projectCache;
}
+ @AutoValue
+ public abstract static class ScanResult {
+ abstract ImmutableSet<Change.Id> fromPatchSetRefs();
+
+ abstract ImmutableSet<Change.Id> fromMetaRefs();
+
+ public SetView<Change.Id> all() {
+ return Sets.union(fromPatchSetRefs(), fromMetaRefs());
+ }
+ }
+
+ public static ScanResult scanChangeIds(Repository repo) throws IOException {
+ ImmutableSet.Builder<Change.Id> fromPs = ImmutableSet.builder();
+ ImmutableSet.Builder<Change.Id> fromMeta = ImmutableSet.builder();
+ for (Ref r : repo.getRefDatabase().getRefsByPrefix(RefNames.REFS_CHANGES)) {
+ Change.Id id = Change.Id.fromRef(r.getName());
+ if (id != null) {
+ (r.getName().endsWith(RefNames.META_SUFFIX) ? fromMeta : fromPs).add(id);
+ }
+ }
+ return new AutoValue_ChangeNotes_Factory_ScanResult(fromPs.build(), fromMeta.build());
+ }
+
public ChangeNotes createChecked(Change c) {
return createChecked(c.getProject(), c.getId());
}
public ChangeNotes createChecked(
+ Repository repo,
+ Project.NameKey project,
+ Change.Id changeId,
+ @Nullable ObjectId metaRevId) {
+ Change change = newChange(project, changeId);
+ return new ChangeNotes(args, change, true, null, metaRevId).load(repo);
+ }
+
+ public ChangeNotes createChecked(
Project.NameKey project, Change.Id changeId, @Nullable ObjectId metaRevId) {
Change change = newChange(project, changeId);
return new ChangeNotes(args, change, true, null, metaRevId).load();
@@ -137,6 +171,22 @@
return new ChangeNotes(args, newChange(project, changeId), true, null).load();
}
+ public ChangeNotes create(Repository repository, Project.NameKey project, Change.Id changeId) {
+ checkArgument(project != null, "project is required");
+ return new ChangeNotes(args, newChange(project, changeId), true, null).load(repository);
+ }
+
+ /**
+ * Create change notes for a change that was loaded from index. This method should only be used
+ * when database access is harmful and potentially stale data from the index is acceptable.
+ *
+ * @param change change loaded from secondary index
+ * @return change notes
+ */
+ public ChangeNotes createFromIndexedChange(Change change) {
+ return new ChangeNotes(args, change, true, null);
+ }
+
public ChangeNotes createForBatchUpdate(Change change, boolean shouldExist) {
return new ChangeNotes(args, change, shouldExist, null).load();
}
@@ -181,13 +231,14 @@
}
public List<ChangeNotes> create(
+ Repository repo,
Project.NameKey project,
Collection<Change.Id> changeIds,
Predicate<ChangeNotes> predicate) {
List<ChangeNotes> notes = new ArrayList<>();
for (Change.Id cid : changeIds) {
try {
- ChangeNotes cn = create(project, cid);
+ ChangeNotes cn = create(repo, project, cid);
if (cn.getChange() != null && predicate.test(cn)) {
notes.add(cn);
}
@@ -200,6 +251,28 @@
return notes;
}
+ /* TODO: This is now unused in the Gerrit code-base, however it is kept in the code
+ /* because it is a public method in a stable branch.
+ * It can be removed in master branch where we have more flexibility to change the API
+ * interface.
+ */
+ public List<ChangeNotes> create(
+ Project.NameKey project,
+ Collection<Change.Id> changeIds,
+ Predicate<ChangeNotes> predicate) {
+ try (Repository repo = args.repoManager.openRepository(project)) {
+ return create(repo, project, changeIds, predicate);
+ } catch (RepositoryNotFoundException e) {
+ // The repository does not exist, hence it does not contain
+ // any change.
+ } catch (IOException e) {
+ logger.atWarning().withCause(e).log(
+ "Unable to open project=%s when trying to retrieve changeId=%s from NoteDb",
+ project, changeIds);
+ }
+ return Collections.emptyList();
+ }
+
public ListMultimap<Project.NameKey, ChangeNotes> create(Predicate<ChangeNotes> predicate)
throws IOException {
ListMultimap<Project.NameKey, ChangeNotes> m =
@@ -224,8 +297,11 @@
public Stream<ChangeNotesResult> scan(
Repository repo, Project.NameKey project, Predicate<Change.Id> changeIdPredicate)
throws IOException {
- ScanResult sr = scanChangeIds(repo);
+ return scan(scanChangeIds(repo), project, changeIdPredicate);
+ }
+ public Stream<ChangeNotesResult> scan(
+ ScanResult sr, Project.NameKey project, Predicate<Change.Id> changeIdPredicate) {
Stream<Change.Id> idStream = sr.all().stream();
if (changeIdPredicate != null) {
idStream = idStream.filter(changeIdPredicate);
@@ -297,29 +373,6 @@
@Nullable
abstract ChangeNotes maybeNotes();
}
-
- @AutoValue
- abstract static class ScanResult {
- abstract ImmutableSet<Change.Id> fromPatchSetRefs();
-
- abstract ImmutableSet<Change.Id> fromMetaRefs();
-
- SetView<Change.Id> all() {
- return Sets.union(fromPatchSetRefs(), fromMetaRefs());
- }
- }
-
- private static ScanResult scanChangeIds(Repository repo) throws IOException {
- ImmutableSet.Builder<Change.Id> fromPs = ImmutableSet.builder();
- ImmutableSet.Builder<Change.Id> fromMeta = ImmutableSet.builder();
- for (Ref r : repo.getRefDatabase().getRefsByPrefix(RefNames.REFS_CHANGES)) {
- Change.Id id = Change.Id.fromRef(r.getName());
- if (id != null) {
- (r.getName().endsWith(RefNames.META_SUFFIX) ? fromMeta : fromPs).add(id);
- }
- }
- return new AutoValue_ChangeNotes_Factory_ScanResult(fromPs.build(), fromMeta.build());
- }
}
private final boolean shouldExist;
diff --git a/java/com/google/gerrit/server/notedb/DraftCommentNotes.java b/java/com/google/gerrit/server/notedb/DraftCommentNotes.java
index 9b403e8..4988406 100644
--- a/java/com/google/gerrit/server/notedb/DraftCommentNotes.java
+++ b/java/com/google/gerrit/server/notedb/DraftCommentNotes.java
@@ -59,7 +59,7 @@
}
DraftCommentNotes(Args args, Change.Id changeId, Account.Id author, @Nullable Ref ref) {
- super(args, changeId);
+ super(args, changeId, null);
this.author = requireNonNull(author);
this.ref = ref;
if (ref != null) {
diff --git a/java/com/google/gerrit/server/notedb/RobotCommentNotes.java b/java/com/google/gerrit/server/notedb/RobotCommentNotes.java
index fe05643..d53b2ca 100644
--- a/java/com/google/gerrit/server/notedb/RobotCommentNotes.java
+++ b/java/com/google/gerrit/server/notedb/RobotCommentNotes.java
@@ -47,7 +47,7 @@
@Inject
RobotCommentNotes(Args args, @Assisted Change change) {
- super(args, change.getId());
+ super(args, change.getId(), null);
this.change = change;
}
diff --git a/java/com/google/gerrit/server/patch/gitfilediff/GitFileDiffCacheImpl.java b/java/com/google/gerrit/server/patch/gitfilediff/GitFileDiffCacheImpl.java
index a44c5fa..0888f3f 100644
--- a/java/com/google/gerrit/server/patch/gitfilediff/GitFileDiffCacheImpl.java
+++ b/java/com/google/gerrit/server/patch/gitfilediff/GitFileDiffCacheImpl.java
@@ -172,7 +172,7 @@
ConfigUtil.getTimeUnit(
cfg,
"cache",
- "diff",
+ GIT_DIFF,
"timeout",
TimeUnit.MILLISECONDS.convert(5, TimeUnit.SECONDS),
TimeUnit.MILLISECONDS);
diff --git a/java/com/google/gerrit/server/permissions/DefaultRefFilter.java b/java/com/google/gerrit/server/permissions/DefaultRefFilter.java
index 8c54742..e523d76 100644
--- a/java/com/google/gerrit/server/permissions/DefaultRefFilter.java
+++ b/java/com/google/gerrit/server/permissions/DefaultRefFilter.java
@@ -135,7 +135,7 @@
return ImmutableList.of();
}
if (RefNames.isRefsChanges(refName)) {
- boolean isChangeRefVisisble = canSeeSingleChangeRef(refName);
+ boolean isChangeRefVisisble = canSeeSingleChangeRef(repo, refName);
if (isChangeRefVisisble) {
logger.atFinest().log("Change ref %s is visible", refName);
return ImmutableList.copyOf(refs);
@@ -375,7 +375,8 @@
* with refs-in-wants is used as that enables Gerrit to skip traditional advertisement of all
* visible refs.
*/
- private boolean canSeeSingleChangeRef(String refName) throws PermissionBackendException {
+ private boolean canSeeSingleChangeRef(Repository repo, String refName)
+ throws PermissionBackendException {
// We are treating just a single change ref. We are therefore not going through regular ref
// filtering, but use NoteDb directly. This makes it so that we can always serve this ref
// even if the change is not part of the set of most recent changes that
@@ -389,7 +390,7 @@
}
ChangeNotes notes;
try {
- notes = changeNotesFactory.create(projectState.getNameKey(), cId);
+ notes = changeNotesFactory.create(repo, projectState.getNameKey(), cId);
} catch (StorageException e) {
throw new PermissionBackendException("can't construct change notes", e);
}
diff --git a/java/com/google/gerrit/server/project/NullProjectCache.java b/java/com/google/gerrit/server/project/NullProjectCache.java
index 1d5f5b7..d19a726 100644
--- a/java/com/google/gerrit/server/project/NullProjectCache.java
+++ b/java/com/google/gerrit/server/project/NullProjectCache.java
@@ -42,11 +42,6 @@
}
@Override
- public void evict(Project p) {
- throw new UnsupportedOperationException();
- }
-
- @Override
public void evict(NameKey p) {
throw new UnsupportedOperationException();
}
@@ -80,4 +75,14 @@
public void onCreateProject(NameKey newProjectName) throws IOException {
throw new UnsupportedOperationException();
}
+
+ @Override
+ public void evictAndReindex(Project p) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void evictAndReindex(NameKey p) {
+ throw new UnsupportedOperationException();
+ }
}
diff --git a/java/com/google/gerrit/server/project/ProjectCache.java b/java/com/google/gerrit/server/project/ProjectCache.java
index fee7105..fb0a4ec 100644
--- a/java/com/google/gerrit/server/project/ProjectCache.java
+++ b/java/com/google/gerrit/server/project/ProjectCache.java
@@ -59,18 +59,25 @@
Optional<ProjectState> get(@Nullable Project.NameKey projectName) throws StorageException;
/**
+ * Invalidate the cached information about the given project.
+ *
+ * @param p the NameKey of the project that is being evicted
+ */
+ void evict(Project.NameKey p);
+
+ /**
* Invalidate the cached information about the given project, and triggers reindexing for it
*
* @param p project that is being evicted
*/
- void evict(Project p);
+ void evictAndReindex(Project p);
/**
* Invalidate the cached information about the given project, and triggers reindexing for it
*
* @param p the NameKey of the project that is being evicted
*/
- void evict(Project.NameKey p);
+ void evictAndReindex(Project.NameKey p);
/**
* Remove information about the given project from the cache. It will no longer be returned from
diff --git a/java/com/google/gerrit/server/project/ProjectCacheImpl.java b/java/com/google/gerrit/server/project/ProjectCacheImpl.java
index 7e2ccca..31bbff5 100644
--- a/java/com/google/gerrit/server/project/ProjectCacheImpl.java
+++ b/java/com/google/gerrit/server/project/ProjectCacheImpl.java
@@ -214,16 +214,21 @@
}
@Override
- public void evict(Project p) {
- evict(p.getNameKey());
- }
-
- @Override
public void evict(Project.NameKey p) {
if (p != null) {
logger.atFine().log("Evict project '%s'", p.get());
inMemoryProjectCache.invalidate(p);
}
+ }
+
+ @Override
+ public void evictAndReindex(Project p) {
+ evictAndReindex(p.getNameKey());
+ }
+
+ @Override
+ public void evictAndReindex(Project.NameKey p) {
+ evict(p);
indexer.get().index(p);
}
@@ -244,7 +249,7 @@
} finally {
listLock.unlock();
}
- evict(name);
+ evictAndReindex(name);
}
@Override
diff --git a/java/com/google/gerrit/server/query/change/InternalChangeQuery.java b/java/com/google/gerrit/server/query/change/InternalChangeQuery.java
index 39ffab6..3f8bfda 100644
--- a/java/com/google/gerrit/server/query/change/InternalChangeQuery.java
+++ b/java/com/google/gerrit/server/query/change/InternalChangeQuery.java
@@ -194,6 +194,7 @@
List<ChangeNotes> notes =
notesFactory.create(
+ repo,
branch.project(),
changeIds,
cn -> {
diff --git a/java/com/google/gerrit/server/restapi/change/GetChange.java b/java/com/google/gerrit/server/restapi/change/GetChange.java
index 740b8cb..a81171a 100644
--- a/java/com/google/gerrit/server/restapi/change/GetChange.java
+++ b/java/com/google/gerrit/server/restapi/change/GetChange.java
@@ -18,6 +18,7 @@
import com.google.common.collect.Streams;
import com.google.gerrit.common.Nullable;
import com.google.gerrit.entities.Change;
+import com.google.gerrit.entities.RefNames;
import com.google.gerrit.extensions.client.ListChangesOption;
import com.google.gerrit.extensions.client.ListOption;
import com.google.gerrit.extensions.common.ChangeInfo;
@@ -26,6 +27,7 @@
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.PreconditionFailedException;
import com.google.gerrit.extensions.restapi.Response;
+import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.server.DynamicOptions;
import com.google.gerrit.server.DynamicOptions.DynamicBean;
@@ -34,15 +36,21 @@
import com.google.gerrit.server.change.ChangeResource;
import com.google.gerrit.server.change.PluginDefinedAttributesFactories;
import com.google.gerrit.server.change.RevisionResource;
+import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.notedb.MissingMetaObjectException;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.inject.Inject;
+import java.io.IOException;
import java.util.Collection;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.Map;
import org.eclipse.jgit.errors.InvalidObjectIdException;
import org.eclipse.jgit.lib.ObjectId;
+import org.eclipse.jgit.lib.Ref;
+import org.eclipse.jgit.lib.Repository;
+import org.eclipse.jgit.revwalk.RevCommit;
+import org.eclipse.jgit.revwalk.RevWalk;
import org.kohsuke.args4j.Option;
public class GetChange
@@ -53,6 +61,7 @@
private final DynamicSet<ChangePluginDefinedInfoFactory> pdiFactories;
private final EnumSet<ListChangesOption> options = EnumSet.noneOf(ListChangesOption.class);
private final Map<String, DynamicBean> dynamicBeans = new HashMap<>();
+ private final GitRepositoryManager repoMgr;
@Option(name = "-o", usage = "Output options")
public void addOption(ListChangesOption o) {
@@ -73,9 +82,13 @@
}
@Inject
- GetChange(ChangeJson.Factory json, DynamicSet<ChangePluginDefinedInfoFactory> pdiFactories) {
+ GetChange(
+ ChangeJson.Factory json,
+ DynamicSet<ChangePluginDefinedInfoFactory> pdiFactories,
+ GitRepositoryManager repoMgr) {
this.json = json;
this.pdiFactories = pdiFactories;
+ this.repoMgr = repoMgr;
}
@Override
@@ -89,10 +102,11 @@
}
@Override
- public Response<ChangeInfo> apply(ChangeResource rsrc)
- throws BadRequestException, PreconditionFailedException {
+ public Response<ChangeInfo> apply(ChangeResource rsrc) throws RestApiException {
try {
- return Response.withMustRevalidate(newChangeJson().format(rsrc.getChange(), getMetaRevId()));
+ Change change = rsrc.getChange();
+ ObjectId changeMetaRevId = getMetaRevId(change);
+ return Response.withMustRevalidate(newChangeJson().format(change, changeMetaRevId));
} catch (MissingMetaObjectException e) {
throw new PreconditionFailedException(e.getMessage());
}
@@ -103,7 +117,7 @@
}
@Nullable
- private ObjectId getMetaRevId() throws BadRequestException {
+ private ObjectId getMetaRevId(Change change) throws RestApiException {
if (metaRevId.isEmpty()) {
return null;
}
@@ -111,11 +125,13 @@
// It might be interesting to also allow {SHA1}^^, so callers can walk back into history
// without having to fetch the entire /meta ref. If we do so, we have to be careful that
// the error messages can't be abused to fetch hidden data.
+ ObjectId metaRevObjectId;
try {
- return ObjectId.fromString(metaRevId);
+ metaRevObjectId = ObjectId.fromString(metaRevId);
} catch (InvalidObjectIdException e) {
throw new BadRequestException("invalid meta SHA1: " + metaRevId, e);
}
+ return verifyMetaId(change, metaRevObjectId);
}
private ChangeJson newChangeJson() {
@@ -127,4 +143,34 @@
return PluginDefinedAttributesFactories.createAll(
cds, this, Streams.stream(pdiFactories.entries()));
}
+
+ private ObjectId verifyMetaId(Change change, @Nullable ObjectId id) throws RestApiException {
+ if (id == null) {
+ return null;
+ }
+
+ String changeMetaRefName = RefNames.changeMetaRef(change.getId());
+ try (Repository repo = repoMgr.openRepository(change.getProject());
+ RevWalk rw = new RevWalk(repo)) {
+ rw.setRetainBody(false);
+ Ref ref = repo.getRefDatabase().exactRef(changeMetaRefName);
+ RevCommit tip = rw.parseCommit(ref.getObjectId());
+ rw.markStart(tip);
+ for (RevCommit rev : rw) {
+ if (id.equals(rev)) {
+ return id;
+ }
+ }
+ } catch (IOException e) {
+ throw RestApiException.wrap(
+ "I/O error while reading meta-ref id="
+ + id.getName()
+ + " from change "
+ + change.getChangeId(),
+ e);
+ }
+
+ throw new PreconditionFailedException(
+ id.getName() + " not reachable from " + changeMetaRefName);
+ }
}
diff --git a/java/com/google/gerrit/server/restapi/change/GetDetail.java b/java/com/google/gerrit/server/restapi/change/GetDetail.java
index c6bbf53..b365e57 100644
--- a/java/com/google/gerrit/server/restapi/change/GetDetail.java
+++ b/java/com/google/gerrit/server/restapi/change/GetDetail.java
@@ -17,8 +17,8 @@
import com.google.gerrit.extensions.client.ListChangesOption;
import com.google.gerrit.extensions.common.ChangeInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
-import com.google.gerrit.extensions.restapi.PreconditionFailedException;
import com.google.gerrit.extensions.restapi.Response;
+import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.server.DynamicOptions;
import com.google.gerrit.server.DynamicOptions.DynamicBean;
@@ -60,8 +60,7 @@
}
@Override
- public Response<ChangeInfo> apply(ChangeResource rsrc)
- throws BadRequestException, PreconditionFailedException {
+ public Response<ChangeInfo> apply(ChangeResource rsrc) throws RestApiException {
return delegate.apply(rsrc);
}
}
diff --git a/java/com/google/gerrit/server/restapi/change/GetMetaDiff.java b/java/com/google/gerrit/server/restapi/change/GetMetaDiff.java
index 08d51e7..81d97e2 100644
--- a/java/com/google/gerrit/server/restapi/change/GetMetaDiff.java
+++ b/java/com/google/gerrit/server/restapi/change/GetMetaDiff.java
@@ -25,6 +25,7 @@
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.PreconditionFailedException;
import com.google.gerrit.extensions.restapi.Response;
+import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.server.DynamicOptions;
import com.google.gerrit.server.DynamicOptions.DynamicBean;
@@ -98,13 +99,13 @@
@Override
public Response<ChangeInfoDifference> apply(ChangeResource resource)
- throws BadRequestException, PreconditionFailedException, IOException {
+ throws RestApiException, IOException {
return Response.ok(
ChangeInfoDiffer.getDifference(getOldChangeInfo(resource), getNewChangeInfo(resource)));
}
private ChangeInfo getOldChangeInfo(ChangeResource resource)
- throws BadRequestException, IOException, PreconditionFailedException {
+ throws RestApiException, IOException {
GetChange getChange = createGetChange();
getChange.setMetaRevId(getOldMetaRevId(resource));
ChangeInfo oldChangeInfo;
@@ -137,7 +138,7 @@
}
private ChangeInfo getNewChangeInfo(ChangeResource resource)
- throws BadRequestException, PreconditionFailedException, IOException {
+ throws RestApiException, IOException {
GetChange getChange = createGetChange();
getChange.setMetaRevId(getNewMetaRevId(resource));
return getChange.apply(resource).value();
diff --git a/java/com/google/gerrit/server/restapi/change/Move.java b/java/com/google/gerrit/server/restapi/change/Move.java
index 86ec6c8..17fe1ce 100644
--- a/java/com/google/gerrit/server/restapi/change/Move.java
+++ b/java/com/google/gerrit/server/restapi/change/Move.java
@@ -200,9 +200,6 @@
RevWalk revWalk = new RevWalk(repo)) {
RevCommit currPatchsetRevCommit =
revWalk.parseCommit(psUtil.current(ctx.getNotes()).commitId());
- if (currPatchsetRevCommit.getParentCount() > 1) {
- throw new ResourceConflictException("Merge commit cannot be moved");
- }
ObjectId refId = repo.resolve(input.destinationBranch);
// Check if destination ref exists in project repo
diff --git a/java/com/google/gerrit/server/restapi/config/GetCache.java b/java/com/google/gerrit/server/restapi/config/GetCache.java
index 93600ea..5dd3d3d 100644
--- a/java/com/google/gerrit/server/restapi/config/GetCache.java
+++ b/java/com/google/gerrit/server/restapi/config/GetCache.java
@@ -16,6 +16,7 @@
import com.google.gerrit.extensions.restapi.Response;
import com.google.gerrit.extensions.restapi.RestReadView;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gerrit.server.config.CacheResource;
import com.google.inject.Singleton;
@@ -23,7 +24,7 @@
public class GetCache implements RestReadView<CacheResource> {
@Override
- public Response<ListCaches.CacheInfo> apply(CacheResource rsrc) {
- return Response.ok(new ListCaches.CacheInfo(rsrc.getName(), rsrc.getCache()));
+ public Response<CacheInfo> apply(CacheResource rsrc) {
+ return Response.ok(new CacheInfo(rsrc.getName(), rsrc.getCache()));
}
}
diff --git a/java/com/google/gerrit/server/restapi/config/ListCaches.java b/java/com/google/gerrit/server/restapi/config/ListCaches.java
index ccafbe8..ffc65c9 100644
--- a/java/com/google/gerrit/server/restapi/config/ListCaches.java
+++ b/java/com/google/gerrit/server/restapi/config/ListCaches.java
@@ -22,7 +22,6 @@
import static java.util.stream.Collectors.joining;
import com.google.common.cache.Cache;
-import com.google.common.cache.CacheStats;
import com.google.common.collect.Streams;
import com.google.gerrit.extensions.annotations.RequiresAnyCapability;
import com.google.gerrit.extensions.registration.DynamicMap;
@@ -30,7 +29,7 @@
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.extensions.restapi.Response;
import com.google.gerrit.extensions.restapi.RestReadView;
-import com.google.gerrit.server.cache.PersistentCache;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gerrit.server.config.ConfigResource;
import com.google.inject.Inject;
import java.util.Map;
@@ -87,118 +86,4 @@
}
return Response.ok(cacheNames.collect(toImmutableList()));
}
-
- public enum CacheType {
- MEM,
- DISK
- }
-
- public static class CacheInfo {
- public String name;
- public CacheType type;
- public EntriesInfo entries;
- public String averageGet;
- public HitRatioInfo hitRatio;
-
- public CacheInfo(Cache<?, ?> cache) {
- this(null, cache);
- }
-
- public CacheInfo(String name, Cache<?, ?> cache) {
- this.name = name;
-
- CacheStats stat = cache.stats();
-
- entries = new EntriesInfo();
- entries.setMem(cache.size());
-
- averageGet = duration(stat.averageLoadPenalty());
-
- hitRatio = new HitRatioInfo();
- hitRatio.setMem(stat.hitCount(), stat.requestCount());
-
- if (cache instanceof PersistentCache) {
- type = CacheType.DISK;
- PersistentCache.DiskStats diskStats = ((PersistentCache) cache).diskStats();
- entries.setDisk(diskStats.size());
- entries.setSpace(diskStats.space());
- hitRatio.setDisk(diskStats.hitCount(), diskStats.requestCount());
- } else {
- type = CacheType.MEM;
- }
- }
-
- private static String duration(double ns) {
- if (ns < 0.5) {
- return null;
- }
- String suffix = "ns";
- if (ns >= 1000.0) {
- ns /= 1000.0;
- suffix = "us";
- }
- if (ns >= 1000.0) {
- ns /= 1000.0;
- suffix = "ms";
- }
- if (ns >= 1000.0) {
- ns /= 1000.0;
- suffix = "s";
- }
- return String.format("%4.1f%s", ns, suffix).trim();
- }
- }
-
- public static class EntriesInfo {
- public Long mem;
- public Long disk;
- public String space;
-
- public void setMem(long mem) {
- this.mem = mem != 0 ? mem : null;
- }
-
- public void setDisk(long disk) {
- this.disk = disk != 0 ? disk : null;
- }
-
- public void setSpace(double value) {
- space = bytes(value);
- }
-
- private static String bytes(double value) {
- value /= 1024;
- String suffix = "k";
-
- if (value > 1024) {
- value /= 1024;
- suffix = "m";
- }
- if (value > 1024) {
- value /= 1024;
- suffix = "g";
- }
- return String.format("%1$6.2f%2$s", value, suffix).trim();
- }
- }
-
- public static class HitRatioInfo {
- public Integer mem;
- public Integer disk;
-
- public void setMem(long value, long total) {
- mem = percent(value, total);
- }
-
- public void setDisk(long value, long total) {
- disk = percent(value, total);
- }
-
- private static Integer percent(long value, long total) {
- if (total <= 0) {
- return null;
- }
- return (int) ((100 * value) / total);
- }
- }
}
diff --git a/java/com/google/gerrit/server/restapi/project/CommitsIncludedInRefs.java b/java/com/google/gerrit/server/restapi/project/CommitsIncludedInRefs.java
new file mode 100644
index 0000000..05ec28e
--- /dev/null
+++ b/java/com/google/gerrit/server/restapi/project/CommitsIncludedInRefs.java
@@ -0,0 +1,85 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.restapi.project;
+
+import com.google.gerrit.extensions.restapi.AuthException;
+import com.google.gerrit.extensions.restapi.BadRequestException;
+import com.google.gerrit.extensions.restapi.ResourceConflictException;
+import com.google.gerrit.extensions.restapi.ResourceNotFoundException;
+import com.google.gerrit.extensions.restapi.Response;
+import com.google.gerrit.extensions.restapi.RestReadView;
+import com.google.gerrit.server.change.IncludedInRefs;
+import com.google.gerrit.server.permissions.PermissionBackendException;
+import com.google.gerrit.server.project.ProjectResource;
+import com.google.inject.Inject;
+import java.io.IOException;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import org.kohsuke.args4j.Option;
+
+public class CommitsIncludedInRefs implements RestReadView<ProjectResource> {
+ protected final IncludedInRefs includedInRefs;
+
+ protected Set<String> commits = new HashSet<>();
+ protected Set<String> refs = new HashSet<>();
+
+ @Option(
+ name = "--commit",
+ aliases = {"-c"},
+ required = true,
+ metaVar = "COMMIT",
+ usage = "commit sha1")
+ protected void addCommit(String commit) {
+ commits.add(commit);
+ }
+
+ @Option(
+ name = "--ref",
+ aliases = {"-r"},
+ required = true,
+ metaVar = "REF",
+ usage = "full ref name")
+ protected void addRef(String ref) {
+ refs.add(ref);
+ }
+
+ public void addCommits(Collection<String> commits) {
+ this.commits.addAll(commits);
+ }
+
+ public void addRefs(Collection<String> refs) {
+ this.refs.addAll(refs);
+ }
+
+ @Inject
+ public CommitsIncludedInRefs(IncludedInRefs includedInRefs) {
+ this.includedInRefs = includedInRefs;
+ }
+
+ @Override
+ public Response<Map<String, Set<String>>> apply(ProjectResource resource)
+ throws ResourceConflictException, BadRequestException, IOException,
+ PermissionBackendException, ResourceNotFoundException, AuthException {
+ if (commits.isEmpty()) {
+ throw new BadRequestException("commit is required");
+ }
+ if (refs.isEmpty()) {
+ throw new BadRequestException("ref is required");
+ }
+ return Response.ok(includedInRefs.apply(resource.getNameKey(), commits, refs));
+ }
+}
diff --git a/java/com/google/gerrit/server/restapi/project/CreateLabel.java b/java/com/google/gerrit/server/restapi/project/CreateLabel.java
index 4eceee5..01686ff 100644
--- a/java/com/google/gerrit/server/restapi/project/CreateLabel.java
+++ b/java/com/google/gerrit/server/restapi/project/CreateLabel.java
@@ -107,7 +107,7 @@
config.commit(md);
- projectCache.evict(rsrc.getProjectState().getProject());
+ projectCache.evictAndReindex(rsrc.getProjectState().getProject());
return Response.created(LabelDefinitionJson.format(rsrc.getNameKey(), labelType));
}
diff --git a/java/com/google/gerrit/server/restapi/project/DeleteLabel.java b/java/com/google/gerrit/server/restapi/project/DeleteLabel.java
index 531640c..8a1927a 100644
--- a/java/com/google/gerrit/server/restapi/project/DeleteLabel.java
+++ b/java/com/google/gerrit/server/restapi/project/DeleteLabel.java
@@ -90,7 +90,7 @@
config.commit(md);
}
- projectCache.evict(rsrc.getProject().getProjectState().getProject());
+ projectCache.evictAndReindex(rsrc.getProject().getProjectState().getProject());
return Response.none();
}
diff --git a/java/com/google/gerrit/server/restapi/project/GetAccess.java b/java/com/google/gerrit/server/restapi/project/GetAccess.java
index b572db3..651e7f0 100644
--- a/java/com/google/gerrit/server/restapi/project/GetAccess.java
+++ b/java/com/google/gerrit/server/restapi/project/GetAccess.java
@@ -153,12 +153,12 @@
if (config.updateGroupNames(groupBackend)) {
md.setMessage("Update group names\n");
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
projectState = projectCache.get(projectName).orElseThrow(illegalState(projectName));
perm = permissionBackend.currentUser().project(projectName);
} else if (config.getRevision() != null
&& !config.getRevision().equals(projectState.getConfig().getRevision().orElse(null))) {
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
projectState = projectCache.get(projectName).orElseThrow(illegalState(projectName));
perm = permissionBackend.currentUser().project(projectName);
}
@@ -331,7 +331,7 @@
}
AccountGroup.UUID group = r.getGroup().getUUID();
if (group != null) {
- pInfo.rules.put(group.get(), info);
+ pInfo.rules.putIfAbsent(group.get(), info); // First entry for the group wins
loadGroup(groups, group);
}
}
diff --git a/java/com/google/gerrit/server/restapi/project/PostLabels.java b/java/com/google/gerrit/server/restapi/project/PostLabels.java
index a56cfe6..6cb912e 100644
--- a/java/com/google/gerrit/server/restapi/project/PostLabels.java
+++ b/java/com/google/gerrit/server/restapi/project/PostLabels.java
@@ -139,7 +139,7 @@
if (dirty) {
config.commit(md);
- projectCache.evict(rsrc.getProjectState().getProject());
+ projectCache.evictAndReindex(rsrc.getProjectState().getProject());
}
}
diff --git a/java/com/google/gerrit/server/restapi/project/ProjectRestApiModule.java b/java/com/google/gerrit/server/restapi/project/ProjectRestApiModule.java
index 410a88c8..e50a494 100644
--- a/java/com/google/gerrit/server/restapi/project/ProjectRestApiModule.java
+++ b/java/com/google/gerrit/server/restapi/project/ProjectRestApiModule.java
@@ -99,6 +99,7 @@
get(FILE_KIND, "content").to(GetContent.class);
child(PROJECT_KIND, "commits").to(CommitsCollection.class);
+ get(PROJECT_KIND, "commits:in").to(CommitsIncludedInRefs.class);
get(COMMIT_KIND).to(GetCommit.class);
get(COMMIT_KIND, "in").to(CommitIncludedIn.class);
child(COMMIT_KIND, "files").to(FilesInCommitCollection.class);
diff --git a/java/com/google/gerrit/server/restapi/project/PutConfig.java b/java/com/google/gerrit/server/restapi/project/PutConfig.java
index fa8638e..30d667c 100644
--- a/java/com/google/gerrit/server/restapi/project/PutConfig.java
+++ b/java/com/google/gerrit/server/restapi/project/PutConfig.java
@@ -163,7 +163,7 @@
md.setMessage("Modified project settings\n");
try {
projectConfig.commit(md);
- projectCache.evict(projectConfig.getProject());
+ projectCache.evictAndReindex(projectConfig.getProject());
md.getRepository().setGitwebDescription(projectConfig.getProject().getDescription());
} catch (IOException e) {
if (e.getCause() instanceof ConfigInvalidException) {
diff --git a/java/com/google/gerrit/server/restapi/project/PutDescription.java b/java/com/google/gerrit/server/restapi/project/PutDescription.java
index a65c626..ec42035 100644
--- a/java/com/google/gerrit/server/restapi/project/PutDescription.java
+++ b/java/com/google/gerrit/server/restapi/project/PutDescription.java
@@ -84,7 +84,7 @@
md.setAuthor(user);
md.setMessage(msg);
config.commit(md);
- cache.evict(resource.getProjectState().getProject());
+ cache.evictAndReindex(resource.getProjectState().getProject());
md.getRepository().setGitwebDescription(config.getProject().getDescription());
return Strings.isNullOrEmpty(config.getProject().getDescription())
diff --git a/java/com/google/gerrit/server/restapi/project/SetAccess.java b/java/com/google/gerrit/server/restapi/project/SetAccess.java
index 794cae8..07dbeca 100644
--- a/java/com/google/gerrit/server/restapi/project/SetAccess.java
+++ b/java/com/google/gerrit/server/restapi/project/SetAccess.java
@@ -125,7 +125,7 @@
}
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
createGroupPermissionSyncer.syncIfNeeded();
} catch (InvalidNameException e) {
throw new BadRequestException(e.toString());
diff --git a/java/com/google/gerrit/server/restapi/project/SetDefaultDashboard.java b/java/com/google/gerrit/server/restapi/project/SetDefaultDashboard.java
index 5aef76a..853d7df 100644
--- a/java/com/google/gerrit/server/restapi/project/SetDefaultDashboard.java
+++ b/java/com/google/gerrit/server/restapi/project/SetDefaultDashboard.java
@@ -115,7 +115,7 @@
md.setAuthor(rsrc.getUser().asIdentifiedUser());
md.setMessage(msg);
config.commit(md);
- cache.evict(rsrc.getProjectState().getProject());
+ cache.evictAndReindex(rsrc.getProjectState().getProject());
if (target != null) {
Response<DashboardInfo> response = get.get().apply(target);
diff --git a/java/com/google/gerrit/server/restapi/project/SetLabel.java b/java/com/google/gerrit/server/restapi/project/SetLabel.java
index d4fcc4f..79bb4ee 100644
--- a/java/com/google/gerrit/server/restapi/project/SetLabel.java
+++ b/java/com/google/gerrit/server/restapi/project/SetLabel.java
@@ -99,7 +99,7 @@
config.getLabelSections().get(newName.isEmpty() ? labelType.getName() : newName);
config.commit(md);
- projectCache.evict(rsrc.getProject().getProjectState().getProject());
+ projectCache.evictAndReindex(rsrc.getProject().getProjectState().getProject());
}
}
return Response.ok(LabelDefinitionJson.format(rsrc.getProject().getNameKey(), labelType));
diff --git a/java/com/google/gerrit/server/restapi/project/SetParent.java b/java/com/google/gerrit/server/restapi/project/SetParent.java
index 91c29f5..ef31dc5 100644
--- a/java/com/google/gerrit/server/restapi/project/SetParent.java
+++ b/java/com/google/gerrit/server/restapi/project/SetParent.java
@@ -114,7 +114,7 @@
md.setAuthor(user);
md.setMessage(msg);
config.commit(md);
- cache.evict(rsrc.getProjectState().getProject());
+ cache.evictAndReindex(rsrc.getProjectState().getProject());
Project.NameKey parent = config.getProject().getParent(allProjects);
requireNonNull(parent);
diff --git a/java/com/google/gerrit/server/submit/SubmitStrategyOp.java b/java/com/google/gerrit/server/submit/SubmitStrategyOp.java
index ab8bedb..de10fe7 100644
--- a/java/com/google/gerrit/server/submit/SubmitStrategyOp.java
+++ b/java/com/google/gerrit/server/submit/SubmitStrategyOp.java
@@ -479,7 +479,7 @@
// TODO(dborowitz): Move to BatchUpdate? Would also allow us to run once
// per project even if multiple changes to refs/meta/config are submitted.
if (RefNames.REFS_CONFIG.equals(getDest().branch())) {
- args.projectCache.evict(getProject());
+ args.projectCache.evictAndReindex(getProject());
ProjectState p =
args.projectCache.get(getProject()).orElseThrow(illegalState(getProject()));
try (Repository git = args.repoManager.openRepository(getProject())) {
diff --git a/java/com/google/gerrit/sshd/commands/ExternalIdCaseSensitivityMigrationCommand.java b/java/com/google/gerrit/sshd/commands/ExternalIdCaseSensitivityMigrationCommand.java
new file mode 100644
index 0000000..aa147f0
--- /dev/null
+++ b/java/com/google/gerrit/sshd/commands/ExternalIdCaseSensitivityMigrationCommand.java
@@ -0,0 +1,53 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.sshd.commands;
+
+import com.google.gerrit.common.data.GlobalCapability;
+import com.google.gerrit.extensions.annotations.RequiresCapability;
+import com.google.gerrit.server.account.externalids.OnlineExternalIdCaseSensivityMigrator;
+import com.google.gerrit.server.config.GerritServerConfig;
+import com.google.gerrit.sshd.CommandMetaData;
+import com.google.gerrit.sshd.SshCommand;
+import com.google.inject.Inject;
+import org.eclipse.jgit.lib.Config;
+
+@RequiresCapability(GlobalCapability.ADMINISTRATE_SERVER)
+@CommandMetaData(
+ name = "migrate-externalids-to-insensitive",
+ description = "Migrate external-ids to case insensitive")
+public class ExternalIdCaseSensitivityMigrationCommand extends SshCommand {
+
+ @Inject OnlineExternalIdCaseSensivityMigrator onlineExternalIdCaseSensivityMigrator;
+ @Inject @GerritServerConfig private Config globalConfig;
+
+ @Override
+ public void run() throws UnloggedFailure, Failure, Exception {
+ Boolean isUserNameCaseInsensitiveMigrationMode =
+ globalConfig.getBoolean("auth", "userNameCaseInsensitiveMigrationMode", false);
+ Boolean isUserNameCaseInsensitive =
+ globalConfig.getBoolean("auth", "userNameCaseInsensitive", false);
+
+ if (!isUserNameCaseInsensitive || !isUserNameCaseInsensitiveMigrationMode) {
+ die(
+ "External IDs online migration requires auth.userNameCaseInsensitive and"
+ + " auth.userNameCaseInsensitiveMigrationMode to be set to true. Cannot start"
+ + " migration!");
+ }
+ onlineExternalIdCaseSensivityMigrator.migrate();
+ stdout.println(
+ "External ids case insensitivity migration started. To check if it's completed look for"
+ + " \"External IDs migration completed!\" message in the Gerrit server logs");
+ }
+}
diff --git a/java/com/google/gerrit/sshd/commands/ExternalIdCommandsModule.java b/java/com/google/gerrit/sshd/commands/ExternalIdCommandsModule.java
new file mode 100644
index 0000000..57bf9e5
--- /dev/null
+++ b/java/com/google/gerrit/sshd/commands/ExternalIdCommandsModule.java
@@ -0,0 +1,40 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.sshd.commands;
+
+import com.google.gerrit.server.account.externalids.OnlineExternalIdCaseSensivityMigratiorExecutor;
+import com.google.gerrit.server.account.externalids.OnlineExternalIdCaseSensivityMigrator;
+import com.google.gerrit.server.git.WorkQueue;
+import com.google.gerrit.sshd.CommandModule;
+import com.google.gerrit.sshd.Commands;
+import com.google.inject.Provides;
+import com.google.inject.Singleton;
+import java.util.concurrent.ExecutorService;
+
+public class ExternalIdCommandsModule extends CommandModule {
+
+ @Override
+ protected void configure() {
+ bind(OnlineExternalIdCaseSensivityMigrator.class);
+ command(Commands.named("gerrit"), ExternalIdCaseSensitivityMigrationCommand.class);
+ }
+
+ @Provides
+ @Singleton
+ @OnlineExternalIdCaseSensivityMigratiorExecutor
+ public ExecutorService OnlineExternalIdCaseSensivityMigratiorExecutor(WorkQueue queues) {
+ return queues.createQueue(1, "MigrateExternalIdCase", true);
+ }
+}
diff --git a/java/com/google/gerrit/sshd/commands/ShowCaches.java b/java/com/google/gerrit/sshd/commands/ShowCaches.java
index 979be1b..02956f7 100644
--- a/java/com/google/gerrit/sshd/commands/ShowCaches.java
+++ b/java/com/google/gerrit/sshd/commands/ShowCaches.java
@@ -24,6 +24,8 @@
import com.google.gerrit.extensions.events.LifecycleListener;
import com.google.gerrit.extensions.restapi.AuthException;
import com.google.gerrit.server.CurrentUser;
+import com.google.gerrit.server.cache.CacheDisplay;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gerrit.server.config.ConfigResource;
import com.google.gerrit.server.permissions.GlobalPermission;
import com.google.gerrit.server.permissions.PermissionBackend;
@@ -35,8 +37,6 @@
import com.google.gerrit.server.restapi.config.GetSummary.TaskSummaryInfo;
import com.google.gerrit.server.restapi.config.GetSummary.ThreadSummaryInfo;
import com.google.gerrit.server.restapi.config.ListCaches;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheInfo;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheType;
import com.google.gerrit.server.util.time.TimeUtil;
import com.google.gerrit.sshd.CommandMetaData;
import com.google.gerrit.sshd.SshCommand;
@@ -123,52 +123,8 @@
stdout.format("%-25s %-20s uptime %16s\n", "", "", uptime(now.getTime() - serverStarted));
stdout.print('\n');
- stdout.print(
- String.format( //
- "%1s %-" + nw + "s|%-21s| %-5s |%-9s|\n" //
- ,
- "" //
- ,
- "Name" //
- ,
- "Entries" //
- ,
- "AvgGet" //
- ,
- "Hit Ratio" //
- ));
- stdout.print(
- String.format( //
- "%1s %-" + nw + "s|%6s %6s %7s| %-5s |%-4s %-4s|\n" //
- ,
- "" //
- ,
- "" //
- ,
- "Mem" //
- ,
- "Disk" //
- ,
- "Space" //
- ,
- "" //
- ,
- "Mem" //
- ,
- "Disk" //
- ));
- stdout.print("--");
- for (int i = 0; i < nw; i++) {
- stdout.print('-');
- }
- stdout.print("+---------------------+---------+---------+\n");
-
try {
- Collection<CacheInfo> caches = getCaches();
- printMemoryCoreCaches(caches);
- printMemoryPluginCaches(caches);
- printDiskCaches(caches);
- stdout.print('\n');
+ new CacheDisplay(stdout, nw, getCaches()).displayCaches();
boolean showJvm;
try {
@@ -209,52 +165,6 @@
return caches.values();
}
- private void printMemoryCoreCaches(Collection<CacheInfo> caches) {
- for (CacheInfo cache : caches) {
- if (!cache.name.contains("-") && CacheType.MEM.equals(cache.type)) {
- printCache(cache);
- }
- }
- }
-
- private void printMemoryPluginCaches(Collection<CacheInfo> caches) {
- for (CacheInfo cache : caches) {
- if (cache.name.contains("-") && CacheType.MEM.equals(cache.type)) {
- printCache(cache);
- }
- }
- }
-
- private void printDiskCaches(Collection<CacheInfo> caches) {
- for (CacheInfo cache : caches) {
- if (CacheType.DISK.equals(cache.type)) {
- printCache(cache);
- }
- }
- }
-
- private void printCache(CacheInfo cache) {
- stdout.print(
- String.format(
- "%1s %-" + nw + "s|%6s %6s %7s| %7s |%4s %4s|\n",
- CacheType.DISK.equals(cache.type) ? "D" : "",
- cache.name,
- nullToEmpty(cache.entries.mem),
- nullToEmpty(cache.entries.disk),
- Strings.nullToEmpty(cache.entries.space),
- Strings.nullToEmpty(cache.averageGet),
- formatAsPercent(cache.hitRatio.mem),
- formatAsPercent(cache.hitRatio.disk)));
- }
-
- private static String nullToEmpty(Long l) {
- return l != null ? String.valueOf(l) : "";
- }
-
- private static String formatAsPercent(Integer i) {
- return i != null ? String.valueOf(i) + "%" : "";
- }
-
private void memSummary(MemSummaryInfo memSummary) {
stdout.format(
"Mem: %s total = %s used + %s free + %s buffers\n",
diff --git a/java/com/google/gerrit/testing/InMemoryModule.java b/java/com/google/gerrit/testing/InMemoryModule.java
index a45e906..b9daa13 100644
--- a/java/com/google/gerrit/testing/InMemoryModule.java
+++ b/java/com/google/gerrit/testing/InMemoryModule.java
@@ -248,8 +248,6 @@
// For custom index types, callers must provide their own module.
if (indexType.isLucene()) {
install(luceneIndexModule());
- } else if (indexType.isElasticsearch()) {
- install(elasticIndexModule());
} else if (indexType.isFake()) {
install(fakeIndexModule());
}
@@ -324,10 +322,6 @@
return indexModule("com.google.gerrit.lucene.LuceneIndexModule");
}
- private Module elasticIndexModule() {
- return indexModule("com.google.gerrit.elasticsearch.ElasticIndexModule");
- }
-
private Module fakeIndexModule() {
return indexModule("com.google.gerrit.index.testing.FakeIndexModule");
}
diff --git a/java/com/google/gerrit/testing/IndexConfig.java b/java/com/google/gerrit/testing/IndexConfig.java
index d68dcad..13b346f 100644
--- a/java/com/google/gerrit/testing/IndexConfig.java
+++ b/java/com/google/gerrit/testing/IndexConfig.java
@@ -43,16 +43,6 @@
return cfg;
}
- public static Config createForElasticsearch() {
- Config cfg = create();
-
- // For some reason enabling the staleness checker increases the flakiness of the Elasticsearch
- // tests. Hence disable the staleness checker.
- cfg.setBoolean("index", null, "autoReindexIfStale", false);
-
- return cfg;
- }
-
public static Config createForFake() {
return create();
}
diff --git a/java/com/google/gerrit/testing/TestLoggingActivator.java b/java/com/google/gerrit/testing/TestLoggingActivator.java
index fc462bf..c7e22c9 100644
--- a/java/com/google/gerrit/testing/TestLoggingActivator.java
+++ b/java/com/google/gerrit/testing/TestLoggingActivator.java
@@ -59,13 +59,6 @@
.put("org.eclipse.jgit.internal.storage.file.FileSnapshot", Level.WARNING)
.put("org.eclipse.jgit.util.FS", Level.WARNING)
.put("org.eclipse.jgit.util.SystemReader", Level.WARNING)
-
- // Silence non-critical messages from Elasticsearch.
- .put("org.elasticsearch", Level.WARNING)
-
- // Silence non-critical messages from Docker for Elasticsearch query tests.
- .put("org.testcontainers", Level.WARNING)
- .put("com.github.dockerjava.core", Level.WARNING)
.build();
private static Level getGerritLogLevel() {
diff --git a/javatests/com/google/gerrit/acceptance/ProjectResetterTest.java b/javatests/com/google/gerrit/acceptance/ProjectResetterTest.java
index ff9bac9..7d04558 100644
--- a/javatests/com/google/gerrit/acceptance/ProjectResetterTest.java
+++ b/javatests/com/google/gerrit/acceptance/ProjectResetterTest.java
@@ -231,7 +231,7 @@
updateRef(repo2, metaConfig);
}
- verify(projectCache, only()).evict(project2);
+ verify(projectCache, only()).evictAndReindex(project2);
}
@Test
@@ -248,7 +248,7 @@
createRef(repo2, RefNames.REFS_CONFIG);
}
- verify(projectCache, only()).evict(project2);
+ verify(projectCache, only()).evictAndReindex(project2);
}
@Test
diff --git a/javatests/com/google/gerrit/acceptance/api/accounts/AccountIT.java b/javatests/com/google/gerrit/acceptance/api/accounts/AccountIT.java
index 1da2176c..915e759 100644
--- a/javatests/com/google/gerrit/acceptance/api/accounts/AccountIT.java
+++ b/javatests/com/google/gerrit/acceptance/api/accounts/AccountIT.java
@@ -135,6 +135,7 @@
import com.google.gerrit.server.account.externalids.ExternalIdKeyFactory;
import com.google.gerrit.server.account.externalids.ExternalIdNotes;
import com.google.gerrit.server.account.externalids.ExternalIds;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.meta.MetaDataUpdate;
import com.google.gerrit.server.index.account.AccountIndexer;
@@ -233,6 +234,7 @@
@Inject private PluginSetContext<ExceptionHook> exceptionHooks;
@Inject private ExternalIdKeyFactory externalIdKeyFactory;
@Inject private ExternalIdFactory externalIdFactory;
+ @Inject private AuthConfig authConfig;
@Inject protected Emails emails;
@@ -2495,7 +2497,11 @@
// stale.
try (Repository repo = repoManager.openRepository(allUsers)) {
ExternalIdNotes extIdNotes =
- ExternalIdNotes.loadNoCacheUpdate(allUsers, repo, externalIdFactory);
+ ExternalIdNotes.loadNoCacheUpdate(
+ allUsers,
+ repo,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode());
ExternalId.Key key = externalIdKeyFactory.create("foo", "foo");
extIdNotes.insert(externalIdFactory.create(key, accountId));
@@ -2504,14 +2510,24 @@
}
assertStaleAccountAndReindex(accountId);
- extIdNotes = ExternalIdNotes.loadNoCacheUpdate(allUsers, repo, externalIdFactory);
+ extIdNotes =
+ ExternalIdNotes.loadNoCacheUpdate(
+ allUsers,
+ repo,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode());
extIdNotes.upsert(externalIdFactory.createWithEmail(key, accountId, "foo@example.com"));
try (MetaDataUpdate update = metaDataUpdateFactory.create(allUsers)) {
extIdNotes.commit(update);
}
assertStaleAccountAndReindex(accountId);
- extIdNotes = ExternalIdNotes.loadNoCacheUpdate(allUsers, repo, externalIdFactory);
+ extIdNotes =
+ ExternalIdNotes.loadNoCacheUpdate(
+ allUsers,
+ repo,
+ externalIdFactory,
+ authConfig.isUserNameCaseInsensitiveMigrationMode());
extIdNotes.delete(accountId, key);
try (MetaDataUpdate update = metaDataUpdateFactory.create(allUsers)) {
extIdNotes.commit(update);
diff --git a/javatests/com/google/gerrit/acceptance/api/accounts/AgreementsIT.java b/javatests/com/google/gerrit/acceptance/api/accounts/AgreementsIT.java
index 5279ba1..9b77b01 100644
--- a/javatests/com/google/gerrit/acceptance/api/accounts/AgreementsIT.java
+++ b/javatests/com/google/gerrit/acceptance/api/accounts/AgreementsIT.java
@@ -71,7 +71,7 @@
config.updateProject(
p -> p.setBooleanConfig(BooleanProjectConfig.USE_CONTRIBUTOR_AGREEMENTS, value));
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
}
diff --git a/javatests/com/google/gerrit/acceptance/api/project/AccessIT.java b/javatests/com/google/gerrit/acceptance/api/project/AccessIT.java
index bf428f9..df5a094 100644
--- a/javatests/com/google/gerrit/acceptance/api/project/AccessIT.java
+++ b/javatests/com/google/gerrit/acceptance/api/project/AccessIT.java
@@ -24,6 +24,8 @@
import static com.google.gerrit.testing.GerritJUnit.assertThrows;
import static com.google.gerrit.truth.ConfigSubject.assertThat;
import static com.google.gerrit.truth.MapSubject.assertThatMap;
+import static java.util.Arrays.asList;
+import static org.junit.Assert.assertEquals;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.ExtensionRegistry;
@@ -64,6 +66,7 @@
import com.google.gerrit.server.schema.GrantRevertPermission;
import com.google.inject.Inject;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import org.eclipse.jgit.internal.storage.dfs.InMemoryRepository;
import org.eclipse.jgit.junit.TestRepository;
@@ -946,6 +949,90 @@
assertThat(thrown).hasMessageThat().contains("Invalid Name: " + invalidRef);
}
+ @Test
+ public void grantAllowAndDenyForSameGroup() throws Exception {
+ GroupReference registeredUsers = systemGroupBackend.getGroup(REGISTERED_USERS);
+ String access = "access";
+ List<String> allowThenDeny =
+ asList(registeredUsers.toConfigValue(), "deny " + registeredUsers.toConfigValue());
+ // Clone repository to forcefully add permission
+ TestRepository<InMemoryRepository> allProjectsRepo = cloneProject(allProjects, admin);
+
+ // Fetch permission ref
+ GitUtil.fetch(allProjectsRepo, "refs/meta/config:cfg");
+ allProjectsRepo.reset("cfg");
+
+ // Load current permissions
+ String config =
+ gApi.projects()
+ .name(allProjects.get())
+ .branch(RefNames.REFS_CONFIG)
+ .file(ProjectConfig.PROJECT_CONFIG)
+ .asString();
+
+ // Append and push allowThenDeny permissions
+ Config cfg = new Config();
+ cfg.fromText(config);
+ cfg.setStringList(access, AccessSection.HEADS, Permission.READ, allowThenDeny);
+ config = cfg.toText();
+ PushOneCommit push =
+ pushFactory.create(
+ admin.newIdent(), allProjectsRepo, "Subject", ProjectConfig.PROJECT_CONFIG, config);
+ push.to(RefNames.REFS_CONFIG).assertOkStatus();
+
+ ProjectAccessInfo pai = gApi.projects().name(allProjects.get()).access();
+ Map<String, AccessSectionInfo> local = pai.local;
+ AccessSectionInfo heads = local.get(AccessSection.HEADS);
+ Map<String, PermissionInfo> permissions = heads.permissions;
+ PermissionInfo read = permissions.get(Permission.READ);
+ Map<String, PermissionRuleInfo> rules = read.rules;
+ assertEquals(
+ rules.get(registeredUsers.getUUID().get()),
+ new PermissionRuleInfo(PermissionRuleInfo.Action.ALLOW, false));
+ }
+
+ @Test
+ public void grantDenyAndAllowForSameGroup() throws Exception {
+ GroupReference registeredUsers = systemGroupBackend.getGroup(REGISTERED_USERS);
+ String access = "access";
+ List<String> denyThenAllow =
+ asList("deny " + registeredUsers.toConfigValue(), registeredUsers.toConfigValue());
+ // Clone repository to forcefully add permission
+ TestRepository<InMemoryRepository> allProjectsRepo = cloneProject(allProjects, admin);
+
+ // Fetch permission ref
+ GitUtil.fetch(allProjectsRepo, "refs/meta/config:cfg");
+ allProjectsRepo.reset("cfg");
+
+ // Load current permissions
+ String config =
+ gApi.projects()
+ .name(allProjects.get())
+ .branch(RefNames.REFS_CONFIG)
+ .file(ProjectConfig.PROJECT_CONFIG)
+ .asString();
+
+ // Append and push denyThenAllow permissions
+ Config cfg = new Config();
+ cfg.fromText(config);
+ cfg.setStringList(access, AccessSection.HEADS, Permission.READ, denyThenAllow);
+ config = cfg.toText();
+ PushOneCommit push =
+ pushFactory.create(
+ admin.newIdent(), allProjectsRepo, "Subject", ProjectConfig.PROJECT_CONFIG, config);
+ push.to(RefNames.REFS_CONFIG).assertOkStatus();
+
+ ProjectAccessInfo pai = gApi.projects().name(allProjects.get()).access();
+ Map<String, AccessSectionInfo> local = pai.local;
+ AccessSectionInfo heads = local.get(AccessSection.HEADS);
+ Map<String, PermissionInfo> permissions = heads.permissions;
+ PermissionInfo read = permissions.get(Permission.READ);
+ Map<String, PermissionRuleInfo> rules = read.rules;
+ assertEquals(
+ rules.get(registeredUsers.getUUID().get()),
+ new PermissionRuleInfo(PermissionRuleInfo.Action.DENY, false));
+ }
+
private ProjectApi pApi() throws Exception {
return gApi.projects().name(newProjectName.get());
}
diff --git a/javatests/com/google/gerrit/acceptance/api/project/ProjectIT.java b/javatests/com/google/gerrit/acceptance/api/project/ProjectIT.java
index c42628c..9c74c48 100644
--- a/javatests/com/google/gerrit/acceptance/api/project/ProjectIT.java
+++ b/javatests/com/google/gerrit/acceptance/api/project/ProjectIT.java
@@ -16,20 +16,25 @@
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
+import static com.google.gerrit.acceptance.GitUtil.pushHead;
import static com.google.gerrit.acceptance.testsuite.project.TestProjectUpdate.allow;
import static com.google.gerrit.acceptance.testsuite.project.TestProjectUpdate.block;
+import static com.google.gerrit.server.group.SystemGroupBackend.REGISTERED_USERS;
import static com.google.gerrit.server.project.ProjectState.INHERITED_FROM_GLOBAL;
import static com.google.gerrit.server.project.ProjectState.INHERITED_FROM_PARENT;
import static com.google.gerrit.server.project.ProjectState.OVERRIDDEN_BY_GLOBAL;
import static com.google.gerrit.server.project.ProjectState.OVERRIDDEN_BY_PARENT;
import static com.google.gerrit.testing.GerritJUnit.assertThrows;
import static java.util.stream.Collectors.toSet;
+import static org.eclipse.jgit.lib.Constants.R_HEADS;
+import static org.eclipse.jgit.lib.Constants.R_TAGS;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
import com.google.common.util.concurrent.AtomicLongMap;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.ExtensionRegistry;
@@ -41,6 +46,7 @@
import com.google.gerrit.acceptance.testsuite.project.ProjectOperations;
import com.google.gerrit.acceptance.testsuite.request.RequestScopeOperations;
import com.google.gerrit.entities.AccountGroup;
+import com.google.gerrit.entities.BranchNameKey;
import com.google.gerrit.entities.Change;
import com.google.gerrit.entities.GroupReference;
import com.google.gerrit.entities.LabelId;
@@ -74,9 +80,13 @@
import com.google.inject.AbstractModule;
import com.google.inject.Inject;
import com.google.inject.Module;
+import java.util.Arrays;
+import java.util.Collections;
import java.util.HashMap;
+import java.util.List;
import java.util.Map;
import java.util.Optional;
+import java.util.Set;
import org.eclipse.jgit.internal.storage.dfs.InMemoryRepository;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.lib.Config;
@@ -92,6 +102,7 @@
private static final String JIRA = "jira";
private static final String JIRA_LINK = "http://jira.example.com/?id=$2";
private static final String JIRA_MATCH = "(jira\\\\s+#?)(\\\\d+)";
+ private static final String R_HEADS_MASTER = R_HEADS + "master";
@Inject private ProjectOperations projectOperations;
@Inject private RequestScopeOperations requestScopeOperations;
@@ -989,7 +1000,7 @@
ProjectConfig config = projectConfigFactory.read(md);
config.renameGroup(AccountGroup.uuid(group.id), newName);
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
Optional<String> afterRename =
@@ -1000,6 +1011,156 @@
assertThat(afterRename).hasValue(newName);
}
+ @Test
+ public void commitsIncludedInRefsEmptyCommitAndEmptyRefs() throws Exception {
+ BadRequestException thrown =
+ assertThrows(
+ BadRequestException.class,
+ () -> getCommitsIncludedInRefs(Collections.emptyList(), Arrays.asList(R_HEADS_MASTER)));
+ assertThat(thrown).hasMessageThat().contains("commit is required");
+ PushOneCommit.Result result = createChange();
+ thrown =
+ assertThrows(
+ BadRequestException.class,
+ () -> getCommitsIncludedInRefs(result.getCommit().getName(), Collections.emptyList()));
+ assertThat(thrown).hasMessageThat().contains("ref is required");
+ }
+
+ @Test
+ public void commitsIncludedInRefsNonExistentCommits() throws Exception {
+ assertThat(
+ getCommitsIncludedInRefs(
+ Arrays.asList("foo", "4fa12ab8f257034ec793dacb2ae2752ae2e9f5f3"),
+ Arrays.asList(R_HEADS_MASTER)))
+ .isEmpty();
+ }
+
+ @Test
+ public void commitsIncludedInRefsNonExistentRefs() throws Exception {
+ PushOneCommit.Result change = createChange();
+ assertThat(
+ getCommitsIncludedInRefs(
+ Arrays.asList(change.getCommit().getName()), Arrays.asList(R_HEADS + "foo")))
+ .isEmpty();
+ }
+
+ @Test
+ public void commitsIncludedInRefsOpenChange() throws Exception {
+ PushOneCommit.Result change1 = createChange();
+ testRepo.reset(change1.getCommit().getParent(0));
+ PushOneCommit.Result change2 = createChange();
+
+ Map<String, Set<String>> refsByCommit =
+ getCommitsIncludedInRefs(
+ Arrays.asList(change1.getCommit().getName(), change2.getCommit().getName()),
+ Arrays.asList(
+ R_HEADS_MASTER, change1.getPatchSet().refName(), change2.getPatchSet().refName()));
+ assertThat(refsByCommit.get(change2.getCommit().getName()))
+ .containsExactly(change2.getPatchSet().refName());
+ assertThat(refsByCommit.get(change1.getCommit().getName()))
+ .containsExactly(change1.getPatchSet().refName());
+ }
+
+ @Test
+ public void commitsIncludedInRefsMergedChange() throws Exception {
+ String branchWithoutChanges = R_HEADS + "branch-without-changes";
+ String tagWithChange1 = R_TAGS + "tag-with-change1";
+ String branchWithChange1 = R_HEADS + "branch-with-change1";
+
+ createBranch(BranchNameKey.create(project, "branch-without-changes"));
+ PushOneCommit.Result change1 = createAndSubmitChange("refs/for/master");
+
+ assertThat(
+ getCommitsIncludedInRefs(change1.getCommit().getName(), Arrays.asList(R_HEADS_MASTER)))
+ .containsExactly(R_HEADS_MASTER);
+ assertThat(
+ getCommitsIncludedInRefs(
+ change1.getCommit().getName(), Arrays.asList(R_HEADS_MASTER, branchWithoutChanges)))
+ .containsExactly(R_HEADS_MASTER);
+
+ pushHead(testRepo, tagWithChange1, false, false);
+ createBranch(BranchNameKey.create(project, "branch-with-change1"));
+
+ PushOneCommit.Result change2 = createAndSubmitChange("refs/for/master");
+ Map<String, Set<String>> refsByCommit =
+ getCommitsIncludedInRefs(
+ Arrays.asList(change1.getCommit().getName(), change2.getCommit().getName()),
+ Arrays.asList(R_HEADS_MASTER, tagWithChange1, branchWithoutChanges, branchWithChange1));
+ assertThat(refsByCommit.get(change1.getCommit().getName()))
+ .containsExactly(R_HEADS_MASTER, tagWithChange1, branchWithChange1);
+ assertThat(refsByCommit.get(change2.getCommit().getName())).containsExactly(R_HEADS_MASTER);
+ }
+
+ @Test
+ public void commitsIncludedInRefsMergedChangeNonTipCommit() throws Exception {
+ String branchWithChange1 = R_HEADS + "branch-with-change1";
+ String tagWithChange1 = R_TAGS + "tag-with-change1";
+ String branchWithChange1Change2 = R_HEADS + "branch-with-change1-change2";
+ String tagWithChange1Change2 = R_TAGS + "tag-with-change1-change2";
+
+ createBranch(BranchNameKey.create(project, "branch-with-change1"));
+ PushOneCommit.Result change1 = createAndSubmitChange("refs/for/branch-with-change1");
+ pushHead(testRepo, tagWithChange1, false, false);
+ createBranch(BranchNameKey.create(project, "branch-with-change1-change2"));
+ PushOneCommit.Result change2 = createAndSubmitChange("refs/for/branch-with-change1-change2");
+ pushHead(testRepo, tagWithChange1Change2, false, false);
+
+ Map<String, Set<String>> refsByCommit =
+ getCommitsIncludedInRefs(
+ Arrays.asList(change1.getCommit().getName(), change2.getCommit().getName()),
+ Arrays.asList(
+ branchWithChange1,
+ branchWithChange1Change2,
+ tagWithChange1,
+ tagWithChange1Change2));
+ assertThat(refsByCommit.get(change1.getCommit().getName()))
+ .containsExactly(
+ branchWithChange1, branchWithChange1Change2, tagWithChange1, tagWithChange1Change2);
+ assertThat(refsByCommit.get(change2.getCommit().getName()))
+ .containsExactly(branchWithChange1Change2, tagWithChange1Change2);
+ }
+
+ @Test
+ public void commitsIncludedInRefsMergedChangeFilterNonVisibleBranchRef() throws Exception {
+ String nonVisibleBranch = R_HEADS + "non-visible-branch";
+ PushOneCommit.Result change = createAndSubmitChange("refs/for/master");
+ createBranch(BranchNameKey.create(project, "non-visible-branch"));
+ blockReadPermission(nonVisibleBranch);
+
+ assertThat(
+ getCommitsIncludedInRefs(
+ change.getCommit().getName(), Arrays.asList(R_HEADS_MASTER, nonVisibleBranch)))
+ .containsExactly(R_HEADS_MASTER);
+ }
+
+ @Test
+ public void commitsIncludedInRefsMergedChangeFilterNonVisibleTagRef() throws Exception {
+ String nonVisibleTag = R_TAGS + "non-visible-tag";
+ PushOneCommit.Result change = createAndSubmitChange("refs/for/master");
+ pushHead(testRepo, nonVisibleTag, false, false);
+ // Tag permissions are controlled by read permissions on branches. Blocking read permission
+ // on master so that tag-with-change becomes non-visible
+ blockReadPermission(R_HEADS_MASTER);
+
+ assertThat(
+ getCommitsIncludedInRefs(
+ change.getCommit().getName(), Arrays.asList(R_HEADS_MASTER, nonVisibleTag)))
+ .isNull();
+ }
+
+ @Test
+ public void commitsIncludedInRefsFilterNonVisibleChangeRef() throws Exception {
+ PushOneCommit.Result change = createChange("refs/for/master");
+ // change ref permissions are controlled by read permissions on destination branch.
+ // Blocking read permission on master so that refs/changes/01/1/1 becomes non-visible
+ blockReadPermission(R_HEADS_MASTER);
+
+ assertThat(
+ getCommitsIncludedInRefs(
+ change.getCommit().getName(), Arrays.asList(change.getPatchSet().refName())))
+ .isNull();
+ }
+
private CommentLinkInfo commentLinkInfo(String name, String match, String link) {
CommentLinkInfo info = new CommentLinkInfo();
info.name = name;
@@ -1039,6 +1200,30 @@
return getConfig(project);
}
+ private Set<String> getCommitsIncludedInRefs(String commit, List<String> refs) throws Exception {
+ return getCommitsIncludedInRefs(Lists.newArrayList(commit), refs).get(commit);
+ }
+
+ private Map<String, Set<String>> getCommitsIncludedInRefs(List<String> commits, List<String> refs)
+ throws Exception {
+ return gApi.projects().name(project.get()).commitsIn(commits, refs);
+ }
+
+ private PushOneCommit.Result createAndSubmitChange(String branch) throws Exception {
+ PushOneCommit.Result r = createChange(branch);
+ approve(r.getChangeId());
+ gApi.changes().id(r.getChangeId()).current().submit();
+ return r;
+ }
+
+ private void blockReadPermission(String ref) {
+ projectOperations
+ .project(project)
+ .forUpdate()
+ .add(block(Permission.READ).ref(ref).group(REGISTERED_USERS))
+ .update();
+ }
+
private ConfigInput createTestConfigInput() {
ConfigInput input = new ConfigInput();
input.description = "some description";
diff --git a/javatests/com/google/gerrit/acceptance/api/revision/RevisionDiffIT.java b/javatests/com/google/gerrit/acceptance/api/revision/RevisionDiffIT.java
index d9531f0..656451d 100644
--- a/javatests/com/google/gerrit/acceptance/api/revision/RevisionDiffIT.java
+++ b/javatests/com/google/gerrit/acceptance/api/revision/RevisionDiffIT.java
@@ -103,7 +103,7 @@
public void setUp() throws Exception {
// Reduce flakiness of tests. (If tests aren't fast enough, we would use a fall-back
// computation, which might yield different results.)
- baseConfig.setString("cache", "diff", "timeout", "1 minute");
+ baseConfig.setString("cache", "git_file_diff", "timeout", "1 minute");
baseConfig.setString("cache", "diff_intraline", "timeout", "1 minute");
intraline = baseConfig.getBoolean(TEST_PARAMETER_MARKER, "intraline", false);
diff --git a/javatests/com/google/gerrit/acceptance/git/AbstractSubmoduleSubscription.java b/javatests/com/google/gerrit/acceptance/git/AbstractSubmoduleSubscription.java
index 415aa79..c3bcbd3 100644
--- a/javatests/com/google/gerrit/acceptance/git/AbstractSubmoduleSubscription.java
+++ b/javatests/com/google/gerrit/acceptance/git/AbstractSubmoduleSubscription.java
@@ -226,7 +226,7 @@
ObjectId oldId = pc.getRevision();
ObjectId newId = pc.commit(md);
assertThat(newId).isNotEqualTo(oldId);
- projectCache.evict(pc.getProject());
+ projectCache.evictAndReindex(pc.getProject());
}
}
diff --git a/javatests/com/google/gerrit/acceptance/pgm/BUILD b/javatests/com/google/gerrit/acceptance/pgm/BUILD
index 5b18d02..9fab01b 100644
--- a/javatests/com/google/gerrit/acceptance/pgm/BUILD
+++ b/javatests/com/google/gerrit/acceptance/pgm/BUILD
@@ -4,7 +4,6 @@
acceptance_tests(
srcs = glob(
["*IT.java"],
- exclude = ["ElasticReindexIT.java"],
),
group = "pgm",
labels = [
@@ -21,24 +20,6 @@
],
)
-acceptance_tests(
- srcs = ["ElasticReindexIT.java"],
- group = "elastic",
- labels = [
- "docker",
- "elastic",
- "pgm",
- "no_windows",
- ],
- vm_args = ["-Xmx1024m"],
- deps = [
- ":util",
- "//java/com/google/gerrit/elasticsearch",
- "//java/com/google/gerrit/server/schema",
- "//javatests/com/google/gerrit/elasticsearch:elasticsearch_test_utils",
- ],
-)
-
java_library(
name = "util",
testonly = True,
diff --git a/javatests/com/google/gerrit/acceptance/pgm/ElasticReindexIT.java b/javatests/com/google/gerrit/acceptance/pgm/ElasticReindexIT.java
deleted file mode 100644
index 0632241..0000000
--- a/javatests/com/google/gerrit/acceptance/pgm/ElasticReindexIT.java
+++ /dev/null
@@ -1,28 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.acceptance.pgm;
-
-import static com.google.gerrit.elasticsearch.ElasticTestUtils.getConfig;
-
-import com.google.gerrit.elasticsearch.ElasticVersion;
-import com.google.gerrit.testing.ConfigSuite;
-import org.eclipse.jgit.lib.Config;
-
-public class ElasticReindexIT extends AbstractReindexTests {
- @ConfigSuite.Default
- public static Config elasticsearchV7() {
- return getConfig(ElasticVersion.V7_8);
- }
-}
diff --git a/javatests/com/google/gerrit/acceptance/rest/DynamicOptionsBeanParseListenerIT.java b/javatests/com/google/gerrit/acceptance/rest/DynamicOptionsBeanParseListenerIT.java
new file mode 100644
index 0000000..39f1e8d
--- /dev/null
+++ b/javatests/com/google/gerrit/acceptance/rest/DynamicOptionsBeanParseListenerIT.java
@@ -0,0 +1,67 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.acceptance.rest;
+
+import static com.google.common.truth.Truth.assertThat;
+
+import com.google.gerrit.acceptance.AbstractDaemonTest;
+import com.google.gerrit.acceptance.RestResponse;
+import com.google.gerrit.extensions.annotations.Exports;
+import com.google.gerrit.json.OutputFormat;
+import com.google.gerrit.server.DynamicOptions;
+import com.google.gerrit.server.restapi.project.ListProjects;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import com.google.inject.AbstractModule;
+import java.util.Map;
+import org.junit.Test;
+
+public class DynamicOptionsBeanParseListenerIT extends AbstractDaemonTest {
+ private static final Gson GSON = OutputFormat.JSON.newGson();
+
+ @Test
+ public void testBeanParseListener() throws Exception {
+ createProjectOverAPI("project1", project, true, null);
+ createProjectOverAPI("project2", project, true, null);
+ try (AutoCloseable ignored = installPlugin("my-plugin", PluginModule.class)) {
+ assertThat(getProjects(adminRestSession.get("/projects/"))).hasSize(1);
+ }
+ }
+
+ protected Map<String, Object> getProjects(RestResponse res) throws Exception {
+ res.assertOK();
+ return GSON.fromJson(res.getReader(), new TypeToken<Map<String, Object>>() {}.getType());
+ }
+
+ protected static class ListProjectsBeanListener implements DynamicOptions.BeanParseListener {
+ @Override
+ public void onBeanParseStart(String plugin, Object bean) {
+ ListProjects listProjects = (ListProjects) bean;
+ listProjects.setLimit(1);
+ }
+
+ @Override
+ public void onBeanParseEnd(String plugin, Object bean) {}
+ }
+
+ protected static class PluginModule extends AbstractModule {
+ @Override
+ public void configure() {
+ bind(DynamicOptions.DynamicBean.class)
+ .annotatedWith(Exports.named(ListProjects.class))
+ .to(ListProjectsBeanListener.class);
+ }
+ }
+}
diff --git a/javatests/com/google/gerrit/acceptance/rest/account/ExternalIdIT.java b/javatests/com/google/gerrit/acceptance/rest/account/ExternalIdIT.java
index cd123aa..1e89a85 100644
--- a/javatests/com/google/gerrit/acceptance/rest/account/ExternalIdIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/account/ExternalIdIT.java
@@ -93,6 +93,10 @@
import org.junit.Test;
public class ExternalIdIT extends AbstractDaemonTest {
+ private static final boolean IS_USER_NAME_CASE_INSENSITIVE_MIGRATION_MODE = false;
+ private static final boolean CASE_SENSITIVE_USERNAME = false;
+ private static final boolean CASE_INSENSITIVE_USERNAME = true;
+
@Inject @ServerInitiated private Provider<AccountsUpdate> accountsUpdateProvider;
@Inject private ExternalIds externalIds;
@Inject private ExternalIdReader externalIdReader;
@@ -852,6 +856,131 @@
@Test
@GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void createCaseInsensitiveMigrationModeExternalIdBeforeTheMigration() throws Exception {
+ Account.Id accountId = Account.id(66);
+ boolean isUserNameCaseInsensitive = false;
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JaneDoe", accountId, isUserNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JaneDoe", accountId, isUserNameCaseInsensitive);
+
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "JaneDoe")).isEqualTo(accountId.get());
+ assertThat(getExternalId(extIdNotes, SCHEME_GERRIT, "janedoe").isPresent()).isFalse();
+
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "JaneDoe")).isEqualTo(accountId.get());
+ assertThat(getExternalId(extIdNotes, SCHEME_USERNAME, "janedoe").isPresent()).isFalse();
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void createCaseInsensitiveMigrationModeExternalIdAccountAfterTheMigration()
+ throws Exception {
+ Account.Id accountId = Account.id(66);
+ boolean isUserNameCaseInsensitive = true;
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JaneDoe", accountId, isUserNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JaneDoe", accountId, isUserNameCaseInsensitive);
+
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "JaneDoe")).isEqualTo(accountId.get());
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "janedoe")).isEqualTo(accountId.get());
+
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "JaneDoe")).isEqualTo(accountId.get());
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "janedoe")).isEqualTo(accountId.get());
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldTolerateDuplicateExternalIdsWhenInMigrationMode() throws Exception {
+ Account.Id firstAccountId = Account.id(1);
+ Account.Id secondAccountId = Account.id(2);
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "janedoe", firstAccountId, CASE_SENSITIVE_USERNAME);
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JaneDoe", secondAccountId, CASE_SENSITIVE_USERNAME);
+
+ ExternalId.Key firstAccountExternalId =
+ externalIdKeyFactory.create(SCHEME_GERRIT, "janedoe", CASE_INSENSITIVE_USERNAME);
+ assertThat(externalIds.get(firstAccountExternalId).get().accountId())
+ .isEqualTo(firstAccountId);
+
+ ExternalId.Key secondAccountExternalId =
+ externalIdKeyFactory.create(SCHEME_GERRIT, "JaneDoe", CASE_INSENSITIVE_USERNAME);
+ assertThat(externalIds.get(secondAccountExternalId).get().accountId())
+ .isEqualTo(secondAccountId);
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void createCaseInsensitiveMigrationModeExternalIdAccountDuringTheMigration()
+ throws Exception {
+ Account.Id accountId = Account.id(66);
+ boolean userNameCaseInsensitive = true;
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JonDoe", accountId, !userNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, !userNameCaseInsensitive);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JaneDoe", accountId, userNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JaneDoe", accountId, userNameCaseInsensitive);
+
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "JonDoe")).isEqualTo(accountId.get());
+ assertThat(getExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isFalse();
+
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "JonDoe")).isEqualTo(accountId.get());
+
+ assertThat(getExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "JaneDoe")).isEqualTo(accountId.get());
+
+ assertThat(getAccountId(extIdNotes, SCHEME_GERRIT, "janedoe")).isEqualTo(accountId.get());
+
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "JaneDoe")).isEqualTo(accountId.get());
+ assertThat(getAccountId(extIdNotes, SCHEME_USERNAME, "janedoe")).isEqualTo(accountId.get());
+ }
+ }
+
+ protected int getAccountId(ExternalIdNotes extIdNotes, String scheme, String id)
+ throws IOException, ConfigInvalidException {
+ return getExternalId(extIdNotes, scheme, id).get().accountId().get();
+ }
+
+ protected Optional<ExternalId> getExternalId(ExternalIdNotes extIdNotes, String scheme, String id)
+ throws IOException, ConfigInvalidException {
+ return extIdNotes.get(externalIdKeyFactory.create(scheme, id));
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
public void createCaseSensitiveExternalId_SchemeWithoutUsername() throws Exception {
try (Repository allUsersRepo = repoManager.openRepository(allUsers);
MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
@@ -869,10 +998,8 @@
ExternalId extId = externalIdFactory.create(scheme, id, accountId);
extIdNotes.insert(extId);
extIdNotes.commit(md);
- assertThat(extIdNotes.get(externalIdKeyFactory.create(scheme, id)).get().accountId().get())
- .isEqualTo(accountId.get());
- assertThat(extIdNotes.get(externalIdKeyFactory.create(scheme, id.toLowerCase())).isPresent())
- .isFalse();
+ assertThat(getAccountId(extIdNotes, scheme, id)).isEqualTo(accountId.get());
+ assertThat(getExternalId(extIdNotes, scheme, id.toLowerCase()).isPresent()).isFalse();
}
private void testCaseInsensitiveExternalIdKey(
@@ -881,15 +1008,29 @@
ExternalId extId = externalIdFactory.create(scheme, id, accountId);
extIdNotes.insert(extId);
extIdNotes.commit(md);
- assertThat(extIdNotes.get(externalIdKeyFactory.create(scheme, id)).get().accountId().get())
- .isEqualTo(accountId.get());
- assertThat(
- extIdNotes
- .get(externalIdKeyFactory.create(scheme, id.toLowerCase()))
- .get()
- .accountId()
- .get())
- .isEqualTo(accountId.get());
+ assertThat(getAccountId(extIdNotes, scheme, id)).isEqualTo(accountId.get());
+ assertThat(getAccountId(extIdNotes, scheme, id.toLowerCase())).isEqualTo(accountId.get());
+ }
+
+ /**
+ * Create external id object
+ *
+ * <p>This method skips gerrit.config auth.userNameCaseInsensitiveMigrationMode and allow to
+ * create case sensitive/insensitive external id
+ */
+ protected void createExternalId(
+ MetaDataUpdate md,
+ ExternalIdNotes extIdNotes,
+ String scheme,
+ String id,
+ Account.Id accountId,
+ boolean isUserNameCaseInsensitive)
+ throws IOException {
+ ExternalId extId =
+ externalIdFactory.create(
+ externalIdKeyFactory.create(scheme, id, isUserNameCaseInsensitive), accountId);
+ extIdNotes.insert(extId);
+ extIdNotes.commit(md);
}
private boolean isPartialCacheReloadingEnabled() {
@@ -918,7 +1059,8 @@
try (Repository repo = repoManager.openRepository(allUsers)) {
// Inserting an external ID "behind Gerrit's back" means that the caches are not updated.
ExternalIdNotes extIdNotes =
- ExternalIdNotes.loadNoCacheUpdate(allUsers, repo, externalIdFactory);
+ ExternalIdNotes.loadNoCacheUpdate(
+ allUsers, repo, externalIdFactory, IS_USER_NAME_CASE_INSENSITIVE_MIGRATION_MODE);
extIdNotes.insert(extId);
try (MetaDataUpdate metaDataUpdate =
new MetaDataUpdate(GitReferenceUpdated.DISABLED, null, repo)) {
diff --git a/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedChildRestApiBindingsIT.java b/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedChildRestApiBindingsIT.java
index 23a1d23..d2c1430 100644
--- a/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedChildRestApiBindingsIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedChildRestApiBindingsIT.java
@@ -56,8 +56,7 @@
@Inject private TestCommentHelper testCommentHelper;
/** Resource to bind a child collection. */
- public static final TypeLiteral<RestView<TestPluginResource>> TEST_KIND =
- new TypeLiteral<RestView<TestPluginResource>>() {};
+ public static final TypeLiteral<RestView<TestPluginResource>> TEST_KIND = new TypeLiteral<>() {};
private static final String PLUGIN_NAME = "my-plugin";
diff --git a/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedRootRestApiBindingsIT.java b/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedRootRestApiBindingsIT.java
index 16dc294..24ce605 100644
--- a/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedRootRestApiBindingsIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/binding/PluginProvidedRootRestApiBindingsIT.java
@@ -60,8 +60,7 @@
public class PluginProvidedRootRestApiBindingsIT extends AbstractDaemonTest {
/** Resource to bind a child collection. */
- public static final TypeLiteral<RestView<TestPluginResource>> TEST_KIND =
- new TypeLiteral<RestView<TestPluginResource>>() {};
+ public static final TypeLiteral<RestView<TestPluginResource>> TEST_KIND = new TypeLiteral<>() {};
private static final String PLUGIN_NAME = "my-plugin";
diff --git a/javatests/com/google/gerrit/acceptance/rest/change/MoveChangeIT.java b/javatests/com/google/gerrit/acceptance/rest/change/MoveChangeIT.java
index 6ed0bf8..d5c7610 100644
--- a/javatests/com/google/gerrit/acceptance/rest/change/MoveChangeIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/change/MoveChangeIT.java
@@ -176,11 +176,9 @@
BranchNameKey newBranch =
BranchNameKey.create(r1.getChange().change().getProject(), "moveTest");
createBranch(newBranch);
- ResourceConflictException thrown =
- assertThrows(
- ResourceConflictException.class,
- () -> move(GitUtil.getChangeId(testRepo, c).get(), newBranch.branch()));
- assertThat(thrown).hasMessageThat().contains("Merge commit cannot be moved");
+ String changeId = GitUtil.getChangeId(testRepo, c).get();
+ move(changeId, newBranch.branch());
+ assertThat(gApi.changes().id(changeId).get().branch).isEqualTo("moveTest");
}
@Test
diff --git a/javatests/com/google/gerrit/acceptance/rest/config/CacheOperationsIT.java b/javatests/com/google/gerrit/acceptance/rest/config/CacheOperationsIT.java
index ef5e7dc..ab8e4d8 100644
--- a/javatests/com/google/gerrit/acceptance/rest/config/CacheOperationsIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/config/CacheOperationsIT.java
@@ -25,7 +25,7 @@
import com.google.gerrit.acceptance.RestResponse;
import com.google.gerrit.acceptance.testsuite.project.ProjectOperations;
import com.google.gerrit.common.data.GlobalCapability;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheInfo;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gerrit.server.restapi.config.PostCaches;
import com.google.inject.Inject;
import java.util.Arrays;
diff --git a/javatests/com/google/gerrit/acceptance/rest/config/FlushCacheIT.java b/javatests/com/google/gerrit/acceptance/rest/config/FlushCacheIT.java
index a161ec4..164f683 100644
--- a/javatests/com/google/gerrit/acceptance/rest/config/FlushCacheIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/config/FlushCacheIT.java
@@ -23,7 +23,7 @@
import com.google.gerrit.acceptance.RestResponse;
import com.google.gerrit.acceptance.testsuite.project.ProjectOperations;
import com.google.gerrit.common.data.GlobalCapability;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheInfo;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.inject.Inject;
import org.junit.Test;
diff --git a/javatests/com/google/gerrit/acceptance/rest/config/GetCacheIT.java b/javatests/com/google/gerrit/acceptance/rest/config/GetCacheIT.java
index 247d63b..8765360 100644
--- a/javatests/com/google/gerrit/acceptance/rest/config/GetCacheIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/config/GetCacheIT.java
@@ -18,8 +18,7 @@
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.RestResponse;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheInfo;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheType;
+import com.google.gerrit.server.cache.CacheInfo;
import org.junit.Test;
public class GetCacheIT extends AbstractDaemonTest {
@@ -31,7 +30,7 @@
CacheInfo result = newGson().fromJson(r.getReader(), CacheInfo.class);
assertThat(result.name).isEqualTo("accounts");
- assertThat(result.type).isEqualTo(CacheType.MEM);
+ assertThat(result.type).isEqualTo(CacheInfo.CacheType.MEM);
assertThat(result.entries.mem).isAtLeast(1L);
assertThat(result.averageGet).isNotNull();
assertThat(result.averageGet).endsWith("s");
diff --git a/javatests/com/google/gerrit/acceptance/rest/config/ListCachesIT.java b/javatests/com/google/gerrit/acceptance/rest/config/ListCachesIT.java
index 8baeffc..be21436 100644
--- a/javatests/com/google/gerrit/acceptance/rest/config/ListCachesIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/config/ListCachesIT.java
@@ -21,8 +21,7 @@
import com.google.common.io.BaseEncoding;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.RestResponse;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheInfo;
-import com.google.gerrit.server.restapi.config.ListCaches.CacheType;
+import com.google.gerrit.server.cache.CacheInfo;
import com.google.gson.reflect.TypeToken;
import java.util.Arrays;
import java.util.List;
@@ -40,7 +39,7 @@
assertThat(result).containsKey("accounts");
CacheInfo accountsCacheInfo = result.get("accounts");
- assertThat(accountsCacheInfo.type).isEqualTo(CacheType.MEM);
+ assertThat(accountsCacheInfo.type).isEqualTo(CacheInfo.CacheType.MEM);
assertThat(accountsCacheInfo.entries.mem).isAtLeast(1L);
assertThat(accountsCacheInfo.averageGet).isNotNull();
assertThat(accountsCacheInfo.averageGet).endsWith("s");
diff --git a/javatests/com/google/gerrit/acceptance/rest/project/FilesInCommitIT.java b/javatests/com/google/gerrit/acceptance/rest/project/FilesInCommitIT.java
index 74ba48e..fa8b79a 100644
--- a/javatests/com/google/gerrit/acceptance/rest/project/FilesInCommitIT.java
+++ b/javatests/com/google/gerrit/acceptance/rest/project/FilesInCommitIT.java
@@ -36,7 +36,7 @@
@Before
public void setUp() throws Exception {
- baseConfig.setString("cache", "diff", "timeout", "1 minute");
+ baseConfig.setString("cache", "git_file_diff", "timeout", "1 minute");
ObjectId headCommit = testRepo.getRepository().resolve("HEAD");
addCommit(
diff --git a/javatests/com/google/gerrit/acceptance/server/account/externalids/BUILD b/javatests/com/google/gerrit/acceptance/server/account/externalids/BUILD
new file mode 100644
index 0000000..32676fe
--- /dev/null
+++ b/javatests/com/google/gerrit/acceptance/server/account/externalids/BUILD
@@ -0,0 +1,7 @@
+load("//javatests/com/google/gerrit/acceptance:tests.bzl", "acceptance_tests")
+
+acceptance_tests(
+ srcs = glob(["*IT.java"]),
+ group = "server_externalids",
+ labels = ["server"],
+)
diff --git a/javatests/com/google/gerrit/acceptance/server/account/externalids/OnlineExternalIdCaseSensivityMigratorIT.java b/javatests/com/google/gerrit/acceptance/server/account/externalids/OnlineExternalIdCaseSensivityMigratorIT.java
new file mode 100644
index 0000000..4eac16f
--- /dev/null
+++ b/javatests/com/google/gerrit/acceptance/server/account/externalids/OnlineExternalIdCaseSensivityMigratorIT.java
@@ -0,0 +1,293 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.acceptance.server.account.externalids;
+
+import static com.google.common.truth.Truth.assertThat;
+import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_GERRIT;
+import static com.google.gerrit.server.account.externalids.ExternalId.SCHEME_USERNAME;
+import static com.google.gerrit.testing.GerritJUnit.assertThrows;
+
+import com.google.common.util.concurrent.MoreExecutors;
+import com.google.gerrit.acceptance.AbstractDaemonTest;
+import com.google.gerrit.acceptance.config.GerritConfig;
+import com.google.gerrit.entities.Account;
+import com.google.gerrit.server.account.externalids.DuplicateExternalIdKeyException;
+import com.google.gerrit.server.account.externalids.ExternalId;
+import com.google.gerrit.server.account.externalids.ExternalIdCaseSensitivityMigrator;
+import com.google.gerrit.server.account.externalids.ExternalIdFactory;
+import com.google.gerrit.server.account.externalids.ExternalIdKeyFactory;
+import com.google.gerrit.server.account.externalids.ExternalIdNotes;
+import com.google.gerrit.server.account.externalids.OnlineExternalIdCaseSensivityMigratiorExecutor;
+import com.google.gerrit.server.account.externalids.OnlineExternalIdCaseSensivityMigrator;
+import com.google.gerrit.server.git.meta.MetaDataUpdate;
+import com.google.inject.AbstractModule;
+import com.google.inject.Inject;
+import com.google.inject.Module;
+import com.google.inject.assistedinject.FactoryModuleBuilder;
+import java.io.IOException;
+import java.util.Optional;
+import java.util.concurrent.ExecutorService;
+import org.eclipse.jgit.errors.ConfigInvalidException;
+import org.eclipse.jgit.errors.RepositoryNotFoundException;
+import org.eclipse.jgit.lib.Repository;
+import org.junit.Test;
+
+public class OnlineExternalIdCaseSensivityMigratorIT extends AbstractDaemonTest {
+ private Account.Id accountId = Account.id(66);
+ private boolean isUserNameCaseInsensitive = false;
+
+ @Inject private ExternalIdNotes.Factory externalIdNotesFactory;
+ @Inject private ExternalIdKeyFactory externalIdKeyFactory;
+ @Inject private ExternalIdFactory externalIdFactory;
+ @Inject private OnlineExternalIdCaseSensivityMigrator objectUnderTest;
+
+ @Override
+ public Module createModule() {
+ return new AbstractModule() {
+ @Override
+ protected void configure() {
+ install(new FactoryModuleBuilder().build(ExternalIdCaseSensitivityMigrator.Factory.class));
+ bind(ExecutorService.class)
+ .annotatedWith(OnlineExternalIdCaseSensivityMigratiorExecutor.class)
+ .toInstance(MoreExecutors.newDirectExecutorService());
+ }
+ };
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldMigrateExternalId() throws IOException, ConfigInvalidException {
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JonDoe", accountId, isUserNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isFalse();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+
+ objectUnderTest.migrate();
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isTrue();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isTrue();
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldNotThrowExceptionDuringTheMigrationForExternalIdsWithCaseInsensitiveSha1()
+ throws IOException, ConfigInvalidException {
+
+ final boolean caseInsensitiveUserName = true;
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(md, extIdNotes, SCHEME_GERRIT, "JonDoe", accountId, caseInsensitiveUserName);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, caseInsensitiveUserName);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isTrue();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isTrue();
+
+ objectUnderTest.migrate();
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldNotCreateDuplicateExternaIdNotesWhenUpdatingAccount()
+ throws IOException, ConfigInvalidException {
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_GERRIT, "JonDoe", accountId, isUserNameCaseInsensitive);
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+ ExternalId extId =
+ externalIdFactory.create(
+ externalIdKeyFactory.create(SCHEME_USERNAME, "JonDoe", true),
+ accountId,
+ "test@email.com",
+ "w1m9Bg85GQ4hijLNxW+6xAfj4r9wyk9rzVQelIHxuQ");
+ extIdNotes.upsert(extId);
+ extIdNotes.commit(md);
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isFalse();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").get().email())
+ .isEqualTo("test@email.com");
+
+ objectUnderTest.migrate();
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_GERRIT, "jondoe").isPresent()).isTrue();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isFalse();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isTrue();
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").get().email())
+ .isEqualTo("test@email.com");
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void caseInsensitivityShouldWorkAfterMigration()
+ throws IOException, ConfigInvalidException {
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+
+ objectUnderTest.migrate();
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+ assertThat(
+ getExternalIdWithCaseInsensitive(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent())
+ .isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isTrue();
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldThrowExceptionWhenDuplicateKeys() throws IOException, ConfigInvalidException {
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "jondoe", Account.id(67), isUserNameCaseInsensitive);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isTrue();
+
+ assertThrows(DuplicateExternalIdKeyException.class, () -> objectUnderTest.migrate());
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "false")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "true")
+ public void shouldSkipMigrationWhenUserNameCaseInsensitiveIsSetToFalse()
+ throws RepositoryNotFoundException, IOException, ConfigInvalidException {
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+
+ objectUnderTest.migrate();
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "JonDoe").isPresent()).isTrue();
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+ }
+ }
+
+ @Test
+ @GerritConfig(name = "auth.userNameCaseInsensitive", value = "true")
+ @GerritConfig(name = "auth.userNameCaseInsensitiveMigrationMode", value = "false")
+ public void shouldSkipMigrationWhenUserNameCaseInsensitiveMigrationModeIsSetToFalse()
+ throws RepositoryNotFoundException, IOException, ConfigInvalidException {
+
+ try (Repository allUsersRepo = repoManager.openRepository(allUsers);
+ MetaDataUpdate md = metaDataUpdateFactory.create(allUsers)) {
+ ExternalIdNotes extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+
+ createExternalId(
+ md, extIdNotes, SCHEME_USERNAME, "JonDoe", accountId, isUserNameCaseInsensitive);
+
+ objectUnderTest.migrate();
+
+ extIdNotes = externalIdNotesFactory.load(allUsersRepo);
+ assertThat(getExactExternalId(extIdNotes, SCHEME_USERNAME, "jondoe").isPresent()).isFalse();
+ }
+ }
+
+ protected Optional<ExternalId> getExternalIdWithCaseInsensitive(
+ ExternalIdNotes extIdNotes, String scheme, String id)
+ throws IOException, ConfigInvalidException {
+ return extIdNotes.get(externalIdKeyFactory.create(scheme, id, true));
+ }
+
+ protected Optional<ExternalId> getExactExternalId(
+ ExternalIdNotes extIdNotes, String scheme, String id)
+ throws IOException, ConfigInvalidException {
+ return extIdNotes.get(externalIdKeyFactory.create(scheme, id, false));
+ }
+
+ protected void createExternalId(
+ MetaDataUpdate md,
+ ExternalIdNotes extIdNotes,
+ String scheme,
+ String id,
+ Account.Id accountId,
+ boolean isUserNameCaseInsensitive)
+ throws IOException {
+ ExternalId extId =
+ externalIdFactory.create(
+ externalIdKeyFactory.create(scheme, id, isUserNameCaseInsensitive), accountId);
+ extIdNotes.insert(extId);
+ extIdNotes.commit(md);
+ }
+}
diff --git a/javatests/com/google/gerrit/acceptance/ssh/BUILD b/javatests/com/google/gerrit/acceptance/ssh/BUILD
index ee1b221..6dec6af 100644
--- a/javatests/com/google/gerrit/acceptance/ssh/BUILD
+++ b/javatests/com/google/gerrit/acceptance/ssh/BUILD
@@ -11,7 +11,6 @@
acceptance_tests(
srcs = glob(
["*IT.java"],
- exclude = ["ElasticIndexIT.java"],
),
group = "ssh",
labels = ["ssh"],
@@ -23,20 +22,3 @@
"//lib/commons:compress",
],
)
-
-acceptance_tests(
- srcs = ["ElasticIndexIT.java"],
- group = "elastic",
- labels = [
- "docker",
- "elastic",
- "exclusive",
- "ssh",
- ],
- deps = [
- ":util",
- "//java/com/google/gerrit/elasticsearch",
- "//javatests/com/google/gerrit/elasticsearch:elasticsearch_test_utils",
- "//lib/commons:compress",
- ],
-)
diff --git a/javatests/com/google/gerrit/acceptance/ssh/DynamicOptionsBeanParseListenerIT.java b/javatests/com/google/gerrit/acceptance/ssh/DynamicOptionsBeanParseListenerIT.java
new file mode 100644
index 0000000..0afdbc6
--- /dev/null
+++ b/javatests/com/google/gerrit/acceptance/ssh/DynamicOptionsBeanParseListenerIT.java
@@ -0,0 +1,69 @@
+// Copyright (C) 2021 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.acceptance.ssh;
+
+import static com.google.common.truth.Truth.assertThat;
+
+import com.google.gerrit.acceptance.AbstractDaemonTest;
+import com.google.gerrit.acceptance.NoHttpd;
+import com.google.gerrit.acceptance.UseSsh;
+import com.google.gerrit.extensions.annotations.Exports;
+import com.google.gerrit.server.DynamicOptions;
+import com.google.gerrit.sshd.commands.ListProjectsCommand;
+import com.google.inject.AbstractModule;
+import java.util.Arrays;
+import java.util.List;
+import org.junit.Test;
+
+@NoHttpd
+@UseSsh
+public class DynamicOptionsBeanParseListenerIT extends AbstractDaemonTest {
+
+ @Test
+ public void testBeanParseListener() throws Exception {
+ createProjectOverAPI("project1", project, true, null);
+ createProjectOverAPI("project2", project, true, null);
+ try (AutoCloseable ignored = installPlugin("my-plugin", PluginModule.class)) {
+ String output = adminSshSession.exec("gerrit ls-projects");
+ adminSshSession.assertSuccess();
+ assertThat(getProjects(output)).hasSize(1);
+ }
+ }
+
+ protected List<String> getProjects(String sshOutput) {
+ return Arrays.asList(sshOutput.split("\n"));
+ }
+
+ protected static class ListProjectsCommandBeanListener
+ implements DynamicOptions.BeanParseListener {
+ @Override
+ public void onBeanParseStart(String plugin, Object bean) {
+ ListProjectsCommand command = (ListProjectsCommand) bean;
+ command.impl.setLimit(1);
+ }
+
+ @Override
+ public void onBeanParseEnd(String plugin, Object bean) {}
+ }
+
+ protected static class PluginModule extends AbstractModule {
+ @Override
+ public void configure() {
+ bind(DynamicOptions.DynamicBean.class)
+ .annotatedWith(Exports.named(ListProjectsCommand.class))
+ .to(ListProjectsCommandBeanListener.class);
+ }
+ }
+}
diff --git a/javatests/com/google/gerrit/acceptance/ssh/ElasticIndexIT.java b/javatests/com/google/gerrit/acceptance/ssh/ElasticIndexIT.java
deleted file mode 100644
index f35bcb7..0000000
--- a/javatests/com/google/gerrit/acceptance/ssh/ElasticIndexIT.java
+++ /dev/null
@@ -1,38 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.acceptance.ssh;
-
-import static com.google.gerrit.elasticsearch.ElasticTestUtils.createAllIndexes;
-import static com.google.gerrit.elasticsearch.ElasticTestUtils.getConfig;
-
-import com.google.gerrit.elasticsearch.ElasticVersion;
-import com.google.gerrit.index.IndexType;
-import com.google.gerrit.testing.ConfigSuite;
-import com.google.inject.Injector;
-import org.eclipse.jgit.lib.Config;
-
-/** Tests for every supported {@link IndexType#isElasticsearch()} most recent index version. */
-public class ElasticIndexIT extends AbstractIndexTests {
-
- @ConfigSuite.Default
- public static Config elasticsearchV7() {
- return getConfig(ElasticVersion.V7_8);
- }
-
- @Override
- public void configureIndex(Injector injector) {
- createAllIndexes(injector);
- }
-}
diff --git a/javatests/com/google/gerrit/acceptance/ssh/SshCommandsIT.java b/javatests/com/google/gerrit/acceptance/ssh/SshCommandsIT.java
index 2dde1a6..ac8a200 100644
--- a/javatests/com/google/gerrit/acceptance/ssh/SshCommandsIT.java
+++ b/javatests/com/google/gerrit/acceptance/ssh/SshCommandsIT.java
@@ -81,7 +81,8 @@
"set-reviewers",
"set-topic",
"stream-events",
- "test-submit");
+ "test-submit",
+ "migrate-externalids-to-insensitive");
private static final ImmutableList<String> EMPTY = ImmutableList.of();
private static final ImmutableMap<String, List<String>> MASTER_COMMANDS =
diff --git a/javatests/com/google/gerrit/acceptance/testsuite/index/DefaultIndexBindingIT.java b/javatests/com/google/gerrit/acceptance/testsuite/index/DefaultIndexBindingIT.java
index 8e0d4bb..f6e5fb3 100644
--- a/javatests/com/google/gerrit/acceptance/testsuite/index/DefaultIndexBindingIT.java
+++ b/javatests/com/google/gerrit/acceptance/testsuite/index/DefaultIndexBindingIT.java
@@ -15,7 +15,9 @@
package com.google.gerrit.acceptance.testsuite.index;
import static com.google.common.truth.Truth.assertThat;
+import static org.junit.Assume.assumeTrue;
+import com.google.common.base.Strings;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.index.IndexType;
import com.google.gerrit.index.testing.AbstractFakeIndex;
@@ -35,6 +37,10 @@
@Test
public void fakeIsBoundByDefault() throws Exception {
+ String gerritIndexTypeEnv = System.getenv("GERRIT_INDEX_TYPE");
+ assumeTrue(
+ Strings.isNullOrEmpty(gerritIndexTypeEnv) || gerritIndexTypeEnv.equalsIgnoreCase("fake"));
+
assertThat(System.getProperty(IndexType.SYS_PROP)).isEmpty();
assertThat(changeIndex.getSearchIndex()).isInstanceOf(AbstractFakeIndex.FakeChangeIndex.class);
}
diff --git a/javatests/com/google/gerrit/elasticsearch/BUILD b/javatests/com/google/gerrit/elasticsearch/BUILD
deleted file mode 100644
index 3036811..0000000
--- a/javatests/com/google/gerrit/elasticsearch/BUILD
+++ /dev/null
@@ -1,85 +0,0 @@
-load("@rules_java//java:defs.bzl", "java_library")
-load("//tools/bzl:junit.bzl", "junit_tests")
-
-java_library(
- name = "elasticsearch_test_utils",
- testonly = True,
- srcs = [
- "ElasticContainer.java",
- "ElasticTestUtils.java",
- ],
- visibility = ["//visibility:public"],
- deps = [
- "//java/com/google/gerrit/elasticsearch",
- "//java/com/google/gerrit/index",
- "//lib:guava",
- "//lib:jgit",
- "//lib:junit",
- "//lib/guice",
- "//lib/httpcomponents:httpcore",
- "//lib/jackson:jackson-annotations",
- "//lib/log:api",
- "//lib/testcontainers",
- "//lib/testcontainers:docker-java-api",
- "//lib/testcontainers:docker-java-transport",
- "//lib/testcontainers:testcontainers-elasticsearch",
- ],
-)
-
-ELASTICSEARCH_DEPS = [
- ":elasticsearch_test_utils",
- "//java/com/google/gerrit/elasticsearch",
- "//java/com/google/gerrit/testing:gerrit-test-util",
- "//lib/guice",
- "//lib:jgit",
-]
-
-HTTP_TEST_DEPS = [
- "//lib/httpcomponents:httpasyncclient",
- "//lib/httpcomponents:httpclient",
-]
-
-QUERY_TESTS_DEP = "//javatests/com/google/gerrit/server/query/%s:abstract_query_tests"
-
-TYPES = [
- "account",
- "change",
- "group",
- "project",
-]
-
-SUFFIX = "sTest.java"
-
-ELASTICSEARCH_TESTS_V7 = {i: "ElasticV7Query" + i.capitalize() + SUFFIX for i in TYPES}
-
-ELASTICSEARCH_TAGS = [
- "docker",
- "elastic",
-]
-
-[junit_tests(
- name = "elasticsearch_query_%ss_test_V7" % name,
- size = "large",
- srcs = [src],
- tags = ELASTICSEARCH_TAGS,
- deps = ELASTICSEARCH_DEPS + [QUERY_TESTS_DEP % name] + HTTP_TEST_DEPS,
-) for name, src in ELASTICSEARCH_TESTS_V7.items()]
-
-junit_tests(
- name = "elasticsearch_tests",
- size = "small",
- srcs = glob(
- ["*Test.java"],
- exclude = ["Elastic*Query*" + SUFFIX],
- ),
- tags = ["elastic"],
- deps = [
- "//java/com/google/gerrit/elasticsearch",
- "//java/com/google/gerrit/testing:gerrit-test-util",
- "//lib:guava",
- "//lib:jgit",
- "//lib/guice",
- "//lib/httpcomponents:httpcore",
- "//lib/truth",
- ],
-)
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticConfigurationTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticConfigurationTest.java
deleted file mode 100644
index 7e044c3..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticConfigurationTest.java
+++ /dev/null
@@ -1,129 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static com.google.common.truth.Truth.assertThat;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.DEFAULT_USERNAME;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.KEY_PASSWORD;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.KEY_PREFIX;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.KEY_SERVER;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.KEY_USERNAME;
-import static com.google.gerrit.elasticsearch.ElasticConfiguration.SECTION_ELASTICSEARCH;
-import static com.google.gerrit.testing.GerritJUnit.assertThrows;
-import static java.util.stream.Collectors.toList;
-
-import com.google.common.collect.ImmutableList;
-import com.google.inject.ProvisionException;
-import java.util.Arrays;
-import org.apache.http.HttpHost;
-import org.eclipse.jgit.lib.Config;
-import org.junit.Test;
-
-public class ElasticConfigurationTest {
- @Test
- public void singleServerNoOtherConfig() throws Exception {
- Config cfg = newConfig();
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertHosts(esCfg, "http://elastic:1234");
- assertThat(esCfg.username).isNull();
- assertThat(esCfg.password).isNull();
- assertThat(esCfg.prefix).isEmpty();
- }
-
- @Test
- public void serverWithoutPortSpecified() throws Exception {
- Config cfg = new Config();
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_SERVER, "http://elastic");
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertHosts(esCfg, "http://elastic:9200");
- }
-
- @Test
- public void prefix() throws Exception {
- Config cfg = newConfig();
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_PREFIX, "myprefix");
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertThat(esCfg.prefix).isEqualTo("myprefix");
- }
-
- @Test
- public void withAuthentication() throws Exception {
- Config cfg = newConfig();
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_USERNAME, "myself");
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_PASSWORD, "s3kr3t");
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertThat(esCfg.username).isEqualTo("myself");
- assertThat(esCfg.password).isEqualTo("s3kr3t");
- }
-
- @Test
- public void withAuthenticationPasswordOnlyUsesDefaultUsername() throws Exception {
- Config cfg = newConfig();
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_PASSWORD, "s3kr3t");
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertThat(esCfg.username).isEqualTo(DEFAULT_USERNAME);
- assertThat(esCfg.password).isEqualTo("s3kr3t");
- }
-
- @Test
- public void multipleServers() throws Exception {
- Config cfg = new Config();
- cfg.setStringList(
- SECTION_ELASTICSEARCH,
- null,
- KEY_SERVER,
- ImmutableList.of("http://elastic1:1234", "http://elastic2:1234"));
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertHosts(esCfg, "http://elastic1:1234", "http://elastic2:1234");
- }
-
- @Test
- public void noServers() throws Exception {
- assertProvisionException(new Config());
- }
-
- @Test
- public void singleServerInvalid() throws Exception {
- Config cfg = new Config();
- cfg.setString(SECTION_ELASTICSEARCH, null, KEY_SERVER, "foo");
- assertProvisionException(cfg);
- }
-
- @Test
- public void multipleServersIncludingInvalid() throws Exception {
- Config cfg = new Config();
- cfg.setStringList(
- SECTION_ELASTICSEARCH, null, KEY_SERVER, ImmutableList.of("http://elastic1:1234", "foo"));
- ElasticConfiguration esCfg = new ElasticConfiguration(cfg);
- assertHosts(esCfg, "http://elastic1:1234");
- }
-
- private static Config newConfig() {
- Config config = new Config();
- config.setString(SECTION_ELASTICSEARCH, null, KEY_SERVER, "http://elastic:1234");
- return config;
- }
-
- private void assertHosts(ElasticConfiguration cfg, Object... hostURIs) throws Exception {
- assertThat(Arrays.asList(cfg.getHosts()).stream().map(HttpHost::toURI).collect(toList()))
- .containsExactly(hostURIs);
- }
-
- private void assertProvisionException(Config cfg) {
- ProvisionException thrown =
- assertThrows(ProvisionException.class, () -> new ElasticConfiguration(cfg));
- assertThat(thrown).hasMessageThat().contains("No valid Elasticsearch servers configured");
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticContainer.java b/javatests/com/google/gerrit/elasticsearch/ElasticContainer.java
deleted file mode 100644
index c330961..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticContainer.java
+++ /dev/null
@@ -1,66 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import org.apache.http.HttpHost;
-import org.junit.AssumptionViolatedException;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.testcontainers.elasticsearch.ElasticsearchContainer;
-import org.testcontainers.utility.DockerImageName;
-
-/* Helper class for running ES integration tests in docker container */
-public class ElasticContainer extends ElasticsearchContainer {
- private static final int ELASTICSEARCH_DEFAULT_PORT = 9200;
-
- public static ElasticContainer createAndStart(ElasticVersion version) {
- // Assumption violation is not natively supported by Testcontainers.
- // See https://github.com/testcontainers/testcontainers-java/issues/343
- try {
- ElasticContainer container = new ElasticContainer(version);
- container.start();
- return container;
- } catch (Throwable t) {
- throw new AssumptionViolatedException("Unable to start container", t);
- }
- }
-
- private static String getImageName(ElasticVersion version) {
- switch (version) {
- case V7_6:
- return "blacktop/elasticsearch:7.6.2";
- case V7_7:
- return "blacktop/elasticsearch:7.7.1";
- case V7_8:
- return "blacktop/elasticsearch:7.8.1";
- }
- throw new IllegalStateException("No tests for version: " + version.name());
- }
-
- private ElasticContainer(ElasticVersion version) {
- super(
- DockerImageName.parse(getImageName(version))
- .asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch"));
- }
-
- @Override
- protected Logger logger() {
- return LoggerFactory.getLogger("org.testcontainers");
- }
-
- public HttpHost getHttpHost() {
- return new HttpHost(getContainerIpAddress(), getMappedPort(ELASTICSEARCH_DEFAULT_PORT));
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticTestUtils.java b/javatests/com/google/gerrit/elasticsearch/ElasticTestUtils.java
deleted file mode 100644
index dcc6880..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticTestUtils.java
+++ /dev/null
@@ -1,54 +0,0 @@
-// Copyright (C) 2016 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.index.IndexDefinition;
-import com.google.inject.Injector;
-import com.google.inject.Key;
-import com.google.inject.TypeLiteral;
-import java.util.Collection;
-import java.util.UUID;
-import org.eclipse.jgit.lib.Config;
-
-public final class ElasticTestUtils {
- public static void configure(Config config, ElasticContainer container, String prefix) {
- String hostname = container.getHttpHost().getHostName();
- int port = container.getHttpHost().getPort();
- config.setString("index", null, "type", "elasticsearch");
- config.setString("elasticsearch", null, "server", "http://" + hostname + ":" + port);
- config.setString("elasticsearch", null, "prefix", prefix);
- config.setInt("index", null, "maxLimit", 10000);
- }
-
- public static void createAllIndexes(Injector injector) {
- Collection<IndexDefinition<?, ?, ?>> indexDefs =
- injector.getInstance(Key.get(new TypeLiteral<Collection<IndexDefinition<?, ?, ?>>>() {}));
- for (IndexDefinition<?, ?, ?> indexDef : indexDefs) {
- indexDef.getIndexCollection().getSearchIndex().deleteAll();
- }
- }
-
- public static Config getConfig(ElasticVersion version) {
- ElasticContainer container = ElasticContainer.createAndStart(version);
- String indicesPrefix = UUID.randomUUID().toString();
- Config cfg = new Config();
- configure(cfg, container, indicesPrefix);
- return cfg;
- }
-
- private ElasticTestUtils() {
- // hide default constructor
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryAccountsTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryAccountsTest.java
deleted file mode 100644
index 4826490..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryAccountsTest.java
+++ /dev/null
@@ -1,64 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.server.query.account.AbstractQueryAccountsTest;
-import com.google.gerrit.testing.ConfigSuite;
-import com.google.gerrit.testing.InMemoryModule;
-import com.google.gerrit.testing.IndexConfig;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import org.eclipse.jgit.lib.Config;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
-
-public class ElasticV7QueryAccountsTest extends AbstractQueryAccountsTest {
- @ConfigSuite.Default
- public static Config defaultConfig() {
- return IndexConfig.createForElasticsearch();
- }
-
- private static ElasticContainer container;
-
- @BeforeClass
- public static void startIndexService() {
- if (container == null) {
- // Only start Elasticsearch once
- container = ElasticContainer.createAndStart(ElasticVersion.V7_8);
- }
- }
-
- @AfterClass
- public static void stopElasticsearchServer() {
- if (container != null) {
- container.stop();
- }
- }
-
- @Override
- protected void initAfterLifecycleStart() throws Exception {
- super.initAfterLifecycleStart();
- ElasticTestUtils.createAllIndexes(injector);
- }
-
- @Override
- protected Injector createInjector() {
- Config elasticsearchConfig = new Config(config);
- InMemoryModule.setDefaults(elasticsearchConfig);
- String indicesPrefix = testName.getSanitizedMethodName();
- ElasticTestUtils.configure(elasticsearchConfig, container, indicesPrefix);
- return Guice.createInjector(new InMemoryModule(elasticsearchConfig));
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryChangesTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryChangesTest.java
deleted file mode 100644
index d9a4d2e..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryChangesTest.java
+++ /dev/null
@@ -1,95 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static java.util.concurrent.TimeUnit.MINUTES;
-
-import com.google.gerrit.server.query.change.AbstractQueryChangesTest;
-import com.google.gerrit.testing.ConfigSuite;
-import com.google.gerrit.testing.GerritTestName;
-import com.google.gerrit.testing.InMemoryModule;
-import com.google.gerrit.testing.IndexConfig;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import org.apache.http.client.methods.HttpPost;
-import org.apache.http.client.protocol.HttpClientContext;
-import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
-import org.apache.http.impl.nio.client.HttpAsyncClients;
-import org.eclipse.jgit.lib.Config;
-import org.junit.After;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
-import org.junit.Rule;
-
-public class ElasticV7QueryChangesTest extends AbstractQueryChangesTest {
- @ConfigSuite.Default
- public static Config defaultConfig() {
- return IndexConfig.createForElasticsearch();
- }
-
- private static ElasticContainer container;
- private static CloseableHttpAsyncClient client;
-
- @BeforeClass
- public static void startIndexService() {
- if (container == null) {
- // Only start Elasticsearch once
- container = ElasticContainer.createAndStart(ElasticVersion.V7_8);
- client = HttpAsyncClients.createDefault();
- client.start();
- }
- }
-
- @AfterClass
- public static void stopElasticsearchServer() {
- if (container != null) {
- container.stop();
- }
- }
-
- @Rule public final GerritTestName testName = new GerritTestName();
-
- @After
- public void closeIndex() throws Exception {
- // Close the index after each test to prevent exceeding Elasticsearch's
- // shard limit (see Issue 10120).
- client
- .execute(
- new HttpPost(
- String.format(
- "http://%s:%d/%s*/_close",
- container.getHttpHost().getHostName(),
- container.getHttpHost().getPort(),
- testName.getSanitizedMethodName())),
- HttpClientContext.create(),
- null)
- .get(5, MINUTES);
- }
-
- @Override
- protected void initAfterLifecycleStart() throws Exception {
- super.initAfterLifecycleStart();
- ElasticTestUtils.createAllIndexes(injector);
- }
-
- @Override
- protected Injector createInjector() {
- Config elasticsearchConfig = new Config(config);
- InMemoryModule.setDefaults(elasticsearchConfig);
- String indicesPrefix = testName.getSanitizedMethodName();
- ElasticTestUtils.configure(elasticsearchConfig, container, indicesPrefix);
- return Guice.createInjector(new InMemoryModule(elasticsearchConfig));
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryGroupsTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryGroupsTest.java
deleted file mode 100644
index 0fc96f8..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryGroupsTest.java
+++ /dev/null
@@ -1,64 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.server.query.group.AbstractQueryGroupsTest;
-import com.google.gerrit.testing.ConfigSuite;
-import com.google.gerrit.testing.InMemoryModule;
-import com.google.gerrit.testing.IndexConfig;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import org.eclipse.jgit.lib.Config;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
-
-public class ElasticV7QueryGroupsTest extends AbstractQueryGroupsTest {
- @ConfigSuite.Default
- public static Config defaultConfig() {
- return IndexConfig.createForElasticsearch();
- }
-
- private static ElasticContainer container;
-
- @BeforeClass
- public static void startIndexService() {
- if (container == null) {
- // Only start Elasticsearch once
- container = ElasticContainer.createAndStart(ElasticVersion.V7_8);
- }
- }
-
- @AfterClass
- public static void stopElasticsearchServer() {
- if (container != null) {
- container.stop();
- }
- }
-
- @Override
- protected void initAfterLifecycleStart() throws Exception {
- super.initAfterLifecycleStart();
- ElasticTestUtils.createAllIndexes(injector);
- }
-
- @Override
- protected Injector createInjector() {
- Config elasticsearchConfig = new Config(config);
- InMemoryModule.setDefaults(elasticsearchConfig);
- String indicesPrefix = testName.getSanitizedMethodName();
- ElasticTestUtils.configure(elasticsearchConfig, container, indicesPrefix);
- return Guice.createInjector(new InMemoryModule(elasticsearchConfig));
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryProjectsTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryProjectsTest.java
deleted file mode 100644
index 1e56af9..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticV7QueryProjectsTest.java
+++ /dev/null
@@ -1,64 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import com.google.gerrit.server.query.project.AbstractQueryProjectsTest;
-import com.google.gerrit.testing.ConfigSuite;
-import com.google.gerrit.testing.InMemoryModule;
-import com.google.gerrit.testing.IndexConfig;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import org.eclipse.jgit.lib.Config;
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
-
-public class ElasticV7QueryProjectsTest extends AbstractQueryProjectsTest {
- @ConfigSuite.Default
- public static Config defaultConfig() {
- return IndexConfig.createForElasticsearch();
- }
-
- private static ElasticContainer container;
-
- @BeforeClass
- public static void startIndexService() {
- if (container == null) {
- // Only start Elasticsearch once
- container = ElasticContainer.createAndStart(ElasticVersion.V7_8);
- }
- }
-
- @AfterClass
- public static void stopElasticsearchServer() {
- if (container != null) {
- container.stop();
- }
- }
-
- @Override
- protected void initAfterLifecycleStart() throws Exception {
- super.initAfterLifecycleStart();
- ElasticTestUtils.createAllIndexes(injector);
- }
-
- @Override
- protected Injector createInjector() {
- Config elasticsearchConfig = new Config(config);
- InMemoryModule.setDefaults(elasticsearchConfig);
- String indicesPrefix = testName.getSanitizedMethodName();
- ElasticTestUtils.configure(elasticsearchConfig, container, indicesPrefix);
- return Guice.createInjector(new InMemoryModule(elasticsearchConfig));
- }
-}
diff --git a/javatests/com/google/gerrit/elasticsearch/ElasticVersionTest.java b/javatests/com/google/gerrit/elasticsearch/ElasticVersionTest.java
deleted file mode 100644
index 2ce3a2c..0000000
--- a/javatests/com/google/gerrit/elasticsearch/ElasticVersionTest.java
+++ /dev/null
@@ -1,46 +0,0 @@
-// Copyright (C) 2018 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.elasticsearch;
-
-import static com.google.common.truth.Truth.assertThat;
-import static com.google.gerrit.testing.GerritJUnit.assertThrows;
-
-import org.junit.Test;
-
-public class ElasticVersionTest {
- @Test
- public void supportedVersion() throws Exception {
- assertThat(ElasticVersion.forVersion("7.6.0")).isEqualTo(ElasticVersion.V7_6);
- assertThat(ElasticVersion.forVersion("7.6.1")).isEqualTo(ElasticVersion.V7_6);
-
- assertThat(ElasticVersion.forVersion("7.7.0")).isEqualTo(ElasticVersion.V7_7);
- assertThat(ElasticVersion.forVersion("7.7.1")).isEqualTo(ElasticVersion.V7_7);
-
- assertThat(ElasticVersion.forVersion("7.8.0")).isEqualTo(ElasticVersion.V7_8);
- assertThat(ElasticVersion.forVersion("7.8.1")).isEqualTo(ElasticVersion.V7_8);
- }
-
- @Test
- public void unsupportedVersion() throws Exception {
- ElasticVersion.UnsupportedVersion thrown =
- assertThrows(
- ElasticVersion.UnsupportedVersion.class, () -> ElasticVersion.forVersion("4.0.0"));
- assertThat(thrown)
- .hasMessageThat()
- .contains(
- "Unsupported version: [4.0.0]. Supported versions: "
- + ElasticVersion.supportedVersions());
- }
-}
diff --git a/javatests/com/google/gerrit/httpd/ProjectBasicAuthFilterTest.java b/javatests/com/google/gerrit/httpd/ProjectBasicAuthFilterTest.java
index da6092b..25334fd 100644
--- a/javatests/com/google/gerrit/httpd/ProjectBasicAuthFilterTest.java
+++ b/javatests/com/google/gerrit/httpd/ProjectBasicAuthFilterTest.java
@@ -102,9 +102,9 @@
res = new FakeHttpServletResponse();
extIdKeyFactory = new ExternalIdKeyFactory(new ExternalIdKeyFactory.ConfigImpl(authConfig));
- extIdFactory = new ExternalIdFactory(extIdKeyFactory);
+ extIdFactory = new ExternalIdFactory(extIdKeyFactory, authConfig);
authRequestFactory = new AuthRequest.Factory(extIdKeyFactory);
- pwdVerifier = new PasswordVerifier(extIdKeyFactory);
+ pwdVerifier = new PasswordVerifier(extIdKeyFactory, authConfig);
authSuccessful =
new AuthResult(AUTH_ACCOUNT_ID, extIdKeyFactory.create("username", AUTH_USER), false);
diff --git a/javatests/com/google/gerrit/integration/git/GitProtocolV2IT.java b/javatests/com/google/gerrit/integration/git/GitProtocolV2IT.java
index b114acc..acf9a50 100644
--- a/javatests/com/google/gerrit/integration/git/GitProtocolV2IT.java
+++ b/javatests/com/google/gerrit/integration/git/GitProtocolV2IT.java
@@ -333,7 +333,7 @@
}
private static void assertGitProtocolV2Refs(String commit, String out) {
- assertThat(out).contains("git< version 2");
+ assertThat(out).containsMatch("(git|ls-remote)< version 2");
assertThat(out).contains("refs/changes/01/1/1");
assertThat(out).contains("refs/changes/01/1/meta");
assertThat(out).contains(commit);
diff --git a/javatests/com/google/gerrit/server/BUILD b/javatests/com/google/gerrit/server/BUILD
index 689698e..c694a87 100644
--- a/javatests/com/google/gerrit/server/BUILD
+++ b/javatests/com/google/gerrit/server/BUILD
@@ -45,6 +45,7 @@
"//java/com/google/gerrit/git",
"//java/com/google/gerrit/index",
"//java/com/google/gerrit/index:query_exception",
+ "//java/com/google/gerrit/index/testing",
"//java/com/google/gerrit/jgit",
"//java/com/google/gerrit/json",
"//java/com/google/gerrit/lifecycle",
diff --git a/javatests/com/google/gerrit/server/account/externalids/AllExternalIdsTest.java b/javatests/com/google/gerrit/server/account/externalids/AllExternalIdsTest.java
index 7d9db0b..eb2133e 100644
--- a/javatests/com/google/gerrit/server/account/externalids/AllExternalIdsTest.java
+++ b/javatests/com/google/gerrit/server/account/externalids/AllExternalIdsTest.java
@@ -25,16 +25,20 @@
import com.google.gerrit.server.account.externalids.AllExternalIds.Serializer;
import com.google.gerrit.server.cache.proto.Cache.AllExternalIdsProto;
import com.google.gerrit.server.cache.proto.Cache.AllExternalIdsProto.ExternalIdProto;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.inject.TypeLiteral;
import java.lang.reflect.Type;
import java.util.Arrays;
import org.eclipse.jgit.lib.ObjectId;
import org.junit.Before;
import org.junit.Test;
+import org.mockito.Mock;
public class AllExternalIdsTest {
private ExternalIdFactory externalIdFactory;
+ @Mock AuthConfig authConfig;
+
@Before
public void setUp() throws Exception {
externalIdFactory =
@@ -45,7 +49,8 @@
public boolean isUserNameCaseInsensitive() {
return false;
}
- }));
+ }),
+ authConfig);
}
@Test
diff --git a/javatests/com/google/gerrit/server/account/externalids/ExternalIDCacheLoaderTest.java b/javatests/com/google/gerrit/server/account/externalids/ExternalIDCacheLoaderTest.java
index 4f8c559..1b5e951 100644
--- a/javatests/com/google/gerrit/server/account/externalids/ExternalIDCacheLoaderTest.java
+++ b/javatests/com/google/gerrit/server/account/externalids/ExternalIDCacheLoaderTest.java
@@ -28,6 +28,7 @@
import com.google.gerrit.server.account.externalids.testing.ExternalIdTestUtil;
import com.google.gerrit.server.config.AllUsersName;
import com.google.gerrit.server.config.AllUsersNameProvider;
+import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.meta.MetaDataUpdate;
@@ -44,6 +45,7 @@
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
+import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.MockitoJUnitRunner;
@@ -58,6 +60,7 @@
private ExternalIdReader externalIdReaderSpy;
private ExternalIdFactory externalIdFactory;
+ @Mock private AuthConfig authConfig;
@Before
public void setUp() throws Exception {
@@ -69,11 +72,13 @@
public boolean isUserNameCaseInsensitive() {
return false;
}
- }));
+ }),
+ authConfig);
externalIdCache = CacheBuilder.newBuilder().build();
repoManager.createRepository(ALL_USERS).close();
externalIdReader =
- new ExternalIdReader(repoManager, ALL_USERS, new DisabledMetricMaker(), externalIdFactory);
+ new ExternalIdReader(
+ repoManager, ALL_USERS, new DisabledMetricMaker(), externalIdFactory, authConfig);
externalIdReaderSpy = Mockito.spy(externalIdReader);
loader = createLoader(true);
}
@@ -277,7 +282,7 @@
try (Repository repo = repoManager.openRepository(ALL_USERS)) {
PersonIdent updater = new PersonIdent("Foo bar", "foo@bar.com");
ExternalIdNotes extIdNotes =
- ExternalIdNotes.loadNoCacheUpdate(ALL_USERS, repo, externalIdFactory);
+ ExternalIdNotes.loadNoCacheUpdate(ALL_USERS, repo, externalIdFactory, false);
update.accept(extIdNotes);
try (MetaDataUpdate metaDataUpdate =
new MetaDataUpdate(GitReferenceUpdated.DISABLED, null, repo)) {
diff --git a/javatests/com/google/gerrit/server/change/IncludedInResolverTest.java b/javatests/com/google/gerrit/server/change/IncludedInResolverTest.java
deleted file mode 100644
index b69a894..0000000
--- a/javatests/com/google/gerrit/server/change/IncludedInResolverTest.java
+++ /dev/null
@@ -1,175 +0,0 @@
-// Copyright (C) 2013 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package com.google.gerrit.server.change;
-
-import static com.google.common.truth.Truth.assertThat;
-import static com.google.gerrit.entities.RefNames.REFS_TAGS;
-
-import com.google.common.truth.Correspondence;
-import com.google.gerrit.truth.NullAwareCorrespondence;
-import org.eclipse.jgit.internal.storage.dfs.DfsRepositoryDescription;
-import org.eclipse.jgit.internal.storage.dfs.InMemoryRepository;
-import org.eclipse.jgit.junit.TestRepository;
-import org.eclipse.jgit.lib.Ref;
-import org.eclipse.jgit.lib.Repository;
-import org.eclipse.jgit.revwalk.RevCommit;
-import org.eclipse.jgit.revwalk.RevObject;
-import org.eclipse.jgit.revwalk.RevTag;
-import org.junit.Before;
-import org.junit.Test;
-
-public class IncludedInResolverTest {
- // Branch names
- private static final String BRANCH_MASTER = "master";
- private static final String BRANCH_1_0 = "rel-1.0";
- private static final String BRANCH_1_3 = "rel-1.3";
- private static final String BRANCH_2_0 = "rel-2.0";
- private static final String BRANCH_2_5 = "rel-2.5";
-
- // Tag names
- private static final String TAG_1_0 = "1.0";
- private static final String TAG_1_0_1 = "1.0.1";
- private static final String TAG_1_3 = "1.3";
- private static final String TAG_2_0_1 = "2.0.1";
- private static final String TAG_2_0 = "2.0";
- private static final String TAG_2_5 = "2.5";
- private static final String TAG_2_5_ANNOTATED = "2.5-annotated";
- private static final String TAG_2_5_ANNOTATED_TWICE = "2.5-annotated_twice";
-
- // Commits
- private RevCommit commit_initial;
- private RevCommit commit_v1_3;
- private RevCommit commit_v2_5;
-
- private TestRepository<?> tr;
-
- @Before
- public void setUp() throws Exception {
- tr = new TestRepository<>(new InMemoryRepository(new DfsRepositoryDescription("repo")));
-
- /*- The following graph will be created.
-
- o tag 2.5, 2.5_annotated, 2.5_annotated_twice
- |\
- | o tag 2.0.1
- | o tag 2.0
- o | tag 1.3
- |/
- o c3
-
- | o tag 1.0.1
- |/
- o tag 1.0
- o c2
- o c1
-
- */
-
- // Version 1.0
- commit_initial = tr.branch(BRANCH_MASTER).commit().message("c1").create();
- tr.branch(BRANCH_MASTER).commit().message("c2").create();
- RevCommit commit_v1_0 = tr.branch(BRANCH_MASTER).commit().message("version 1.0").create();
- tag(TAG_1_0, commit_v1_0);
- RevCommit c3 = tr.branch(BRANCH_MASTER).commit().message("c3").create();
-
- // Version 1.01
- tr.branch(BRANCH_1_0).update(commit_v1_0);
- RevCommit commit_v1_0_1 = tr.branch(BRANCH_1_0).commit().message("version 1.0.1").create();
- tag(TAG_1_0_1, commit_v1_0_1);
-
- // Version 1.3
- tr.branch(BRANCH_1_3).update(c3);
- commit_v1_3 = tr.branch(BRANCH_1_3).commit().message("version 1.3").create();
- tag(TAG_1_3, commit_v1_3);
-
- // Version 2.0
- tr.branch(BRANCH_2_0).update(c3);
- RevCommit commit_v2_0 = tr.branch(BRANCH_2_0).commit().message("version 2.0").create();
- tag(TAG_2_0, commit_v2_0);
- RevCommit commit_v2_0_1 = tr.branch(BRANCH_2_0).commit().message("version 2.0.1").create();
- tag(TAG_2_0_1, commit_v2_0_1);
-
- // Version 2.5
- tr.branch(BRANCH_2_5).update(commit_v1_3);
- tr.branch(BRANCH_2_5).commit().parent(commit_v2_0_1).create(); // Merge v2.0.1
- commit_v2_5 = tr.branch(BRANCH_2_5).commit().message("version 2.5").create();
- tr.update(REFS_TAGS + TAG_2_5, commit_v2_5);
- RevTag tag_2_5_annotated = tag(TAG_2_5_ANNOTATED, commit_v2_5);
- tag(TAG_2_5_ANNOTATED_TWICE, tag_2_5_annotated);
- }
-
- @Test
- public void resolveLatestCommit() throws Exception {
- // Check tip commit
- IncludedInResolver.Result detail = resolve(commit_v2_5);
-
- // Check that only tags and branches which refer the tip are returned
- assertThat(detail.tags())
- .comparingElementsUsing(hasShortName())
- .containsExactly(TAG_2_5, TAG_2_5_ANNOTATED, TAG_2_5_ANNOTATED_TWICE);
- assertThat(detail.branches())
- .comparingElementsUsing(hasShortName())
- .containsExactly(BRANCH_2_5);
- }
-
- @Test
- public void resolveFirstCommit() throws Exception {
- // Check first commit
- IncludedInResolver.Result detail = resolve(commit_initial);
-
- // Check whether all tags and branches are returned
- assertThat(detail.tags())
- .comparingElementsUsing(hasShortName())
- .containsExactly(
- TAG_1_0,
- TAG_1_0_1,
- TAG_1_3,
- TAG_2_0,
- TAG_2_0_1,
- TAG_2_5,
- TAG_2_5_ANNOTATED,
- TAG_2_5_ANNOTATED_TWICE);
- assertThat(detail.branches())
- .comparingElementsUsing(hasShortName())
- .containsExactly(BRANCH_MASTER, BRANCH_1_0, BRANCH_1_3, BRANCH_2_0, BRANCH_2_5);
- }
-
- @Test
- public void resolveBetwixtCommit() throws Exception {
- // Check a commit somewhere in the middle
- IncludedInResolver.Result detail = resolve(commit_v1_3);
-
- // Check whether all succeeding tags and branches are returned
- assertThat(detail.tags())
- .comparingElementsUsing(hasShortName())
- .containsExactly(TAG_1_3, TAG_2_5, TAG_2_5_ANNOTATED, TAG_2_5_ANNOTATED_TWICE);
- assertThat(detail.branches())
- .comparingElementsUsing(hasShortName())
- .containsExactly(BRANCH_1_3, BRANCH_2_5);
- }
-
- private IncludedInResolver.Result resolve(RevCommit commit) throws Exception {
- return IncludedInResolver.resolve(tr.getRepository(), tr.getRevWalk(), commit);
- }
-
- private RevTag tag(String name, RevObject dest) throws Exception {
- return tr.update(REFS_TAGS + name, tr.tag(name, dest));
- }
-
- private static Correspondence<Ref, String> hasShortName() {
- return NullAwareCorrespondence.transforming(
- ref -> Repository.shortenRefName(ref.getName()), "has short name");
- }
-}
diff --git a/javatests/com/google/gerrit/server/change/LabelNormalizerTest.java b/javatests/com/google/gerrit/server/change/LabelNormalizerTest.java
index 5e3be9a..8f341aa 100644
--- a/javatests/com/google/gerrit/server/change/LabelNormalizerTest.java
+++ b/javatests/com/google/gerrit/server/change/LabelNormalizerTest.java
@@ -205,7 +205,7 @@
private void save(ProjectConfig pc) throws Exception {
try (MetaDataUpdate md = metaDataUpdateFactory.create(pc.getProject().getNameKey(), user)) {
pc.commit(md);
- projectCache.evict(pc.getProject().getNameKey());
+ projectCache.evictAndReindex(pc.getProject().getNameKey());
}
}
diff --git a/javatests/com/google/gerrit/server/events/EventDeserializerTest.java b/javatests/com/google/gerrit/server/events/EventDeserializerTest.java
index e0223e4..97f6e4e 100644
--- a/javatests/com/google/gerrit/server/events/EventDeserializerTest.java
+++ b/javatests/com/google/gerrit/server/events/EventDeserializerTest.java
@@ -22,6 +22,8 @@
import com.google.gerrit.entities.BranchNameKey;
import com.google.gerrit.entities.Change;
import com.google.gerrit.entities.Project;
+import com.google.gerrit.server.config.AllProjectsName;
+import com.google.gerrit.server.config.AllUsersName;
import com.google.gerrit.server.data.AccountAttribute;
import com.google.gerrit.server.data.ChangeAttribute;
import com.google.gerrit.server.data.RefUpdateAttribute;
@@ -240,6 +242,31 @@
assertSameChangeEvent(e, orig);
}
+ @Test
+ public void shouldSerializeAllProjectsToString() {
+ String allProjectsString = "foobar";
+ AllProjectsName allProjectsNameKey = new AllProjectsName(allProjectsString);
+
+ assertThat(gson.toJson(allProjectsNameKey))
+ .isEqualTo(String.format("\"%s\"", allProjectsString));
+ }
+
+ @Test
+ public void shouldSerializeAllUsersToString() {
+ String allUsersString = "foobar";
+ AllUsersName allUsersNameKey = new AllUsersName(allUsersString);
+
+ assertThat(gson.toJson(allUsersNameKey)).isEqualTo(String.format("\"%s\"", allUsersString));
+ }
+
+ @Test
+ public void shouldSerializeProjectNameKeyToString() {
+ String projectString = "foobar";
+ Project.NameKey projectNameKey = Project.nameKey(projectString);
+
+ assertThat(gson.toJson(projectNameKey)).isEqualTo(String.format("\"%s\"", projectString));
+ }
+
private <T> Supplier<T> createSupplier(T value) {
return Suppliers.memoize(() -> value);
}
diff --git a/javatests/com/google/gerrit/server/index/change/ChangeFieldTest.java b/javatests/com/google/gerrit/server/index/change/ChangeFieldTest.java
index a7b25b8..6ad2060 100644
--- a/javatests/com/google/gerrit/server/index/change/ChangeFieldTest.java
+++ b/javatests/com/google/gerrit/server/index/change/ChangeFieldTest.java
@@ -25,15 +25,19 @@
import com.google.gerrit.entities.Account;
import com.google.gerrit.entities.Change;
import com.google.gerrit.entities.LegacySubmitRequirement;
+import com.google.gerrit.entities.Project;
import com.google.gerrit.entities.SubmitRecord;
+import com.google.gerrit.index.testing.FakeStoredValue;
import com.google.gerrit.server.ReviewerSet;
import com.google.gerrit.server.notedb.ReviewerStateInternal;
+import com.google.gerrit.server.query.change.ChangeData;
import com.google.gerrit.server.util.time.TimeUtil;
import com.google.gerrit.testing.TestTimeUtil;
import java.sql.Timestamp;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.TimeUnit;
+import org.eclipse.jgit.lib.ObjectId;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@@ -129,6 +133,20 @@
assertStoredRecordRoundTrip(r);
}
+ @Test
+ public void tolerateNullValuesForInsertion() {
+ Project.NameKey project = Project.nameKey("project");
+ ChangeData cd = ChangeData.createForTest(project, Change.id(1), 1, ObjectId.zeroId());
+ assertThat(ChangeField.ADDED.setIfPossible(cd, new FakeStoredValue(null))).isTrue();
+ }
+
+ @Test
+ public void tolerateNullValuesForDeletion() {
+ Project.NameKey project = Project.nameKey("project");
+ ChangeData cd = ChangeData.createForTest(project, Change.id(1), 1, ObjectId.zeroId());
+ assertThat(ChangeField.DELETED.setIfPossible(cd, new FakeStoredValue(null))).isTrue();
+ }
+
private static SubmitRecord record(SubmitRecord.Status status, SubmitRecord.Label... labels) {
SubmitRecord r = new SubmitRecord();
r.status = status;
diff --git a/javatests/com/google/gerrit/server/index/change/FakeChangeIndex.java b/javatests/com/google/gerrit/server/index/change/FakeChangeIndex.java
index a23ccab..fc56a3c 100644
--- a/javatests/com/google/gerrit/server/index/change/FakeChangeIndex.java
+++ b/javatests/com/google/gerrit/server/index/change/FakeChangeIndex.java
@@ -74,6 +74,11 @@
}
@Override
+ public void insert(ChangeData obj) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
public void replace(ChangeData cd) {
throw new UnsupportedOperationException();
}
diff --git a/javatests/com/google/gerrit/server/notedb/ChangeNotesParserTest.java b/javatests/com/google/gerrit/server/notedb/ChangeNotesParserTest.java
index 32aefbe..548764b 100644
--- a/javatests/com/google/gerrit/server/notedb/ChangeNotesParserTest.java
+++ b/javatests/com/google/gerrit/server/notedb/ChangeNotesParserTest.java
@@ -276,6 +276,17 @@
+ "Submitted-with: NOT_READY\n"
+ "Submitted-with: OK: Verified: Change Owner <1@gerrit>\n"
+ "Submitted-with: NEED: Alternative-Code-Review\n");
+ assertParseSucceeds(
+ "Update change\n"
+ + "\n"
+ + "Branch: refs/heads/master\n"
+ + "Change-id: I577fb248e474018276351785930358ec0450e9f7\n"
+ + "Patch-set: 1\n"
+ + "Subject: This is a test change\n"
+ + "Submitted-with: NOT_READY\n"
+ + "Submitted-with: Rule-Name: gerrit~PrologRule\n" // Rule-Name footer is ignored
+ + "Submitted-with: OK: Verified: Change Owner <1@gerrit>\n"
+ + "Submitted-with: NEED: Code-Review\n");
assertParseFails("Update change\n\nPatch-set: 1\nSubmitted-with: OOPS\n");
assertParseFails("Update change\n\nPatch-set: 1\nSubmitted-with: NEED: X+Y\n");
assertParseFails(
@@ -612,7 +623,9 @@
"Update patch set 1\n"
+ "\n"
+ "Patch-set: 1\n"
- + "Attention: {\"person_ident\":\"Gerrit User 1000000 \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added by Administrator using the hovercard menu\"}",
+ + "Attention: {\"person_ident\":\"Gerrit User 1000000"
+ + " \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added"
+ + " by Administrator using the hovercard menu\"}",
false);
ChangeNotesParser changeNotesParser = newParser(commit);
changeNotesParser.parseAll();
@@ -631,7 +644,9 @@
+ "\n"
+ "Patch-set: 1\n"
+ "Subject: Change subject\n"
- + "Attention: {\"person_ident\":\"Gerrit User 1000000 \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added by Administrator using the hovercard menu\"}",
+ + "Attention: {\"person_ident\":\"Gerrit User 1000000"
+ + " \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added"
+ + " by Administrator using the hovercard menu\"}",
false);
ChangeNotesParser changeNotesParser = newParser(commit);
changeNotesParser.parseAll();
@@ -661,7 +676,9 @@
"Update patch set 1\n"
+ "\n"
+ "Patch-set: 1\n"
- + "Attention: {\"person_ident\":\"Gerrit User 1000000 \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added by Administrator using the hovercard menu\"}",
+ + "Attention: {\"person_ident\":\"Gerrit User 1000000"
+ + " \\u003c1000000@adce0b11-8f2e-4ab6-ac69-e675f183d871\\u003e\",\"operation\":\"ADD\",\"reason\":\"Added"
+ + " by Administrator using the hovercard menu\"}",
false);
ChangeNotesParser changeNotesParser = newParser(commit);
changeNotesParser.parseAll();
diff --git a/javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java b/javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
index 4853376..ecdb066 100644
--- a/javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
+++ b/javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java
@@ -1206,7 +1206,7 @@
cfg.upsertLabelType(verified);
cfg.commit(md);
}
- projectCache.evict(project);
+ projectCache.evictAndReindex(project);
String heads = RefNames.REFS_HEADS + "*";
projectOperations
@@ -2335,7 +2335,7 @@
});
config.commit(md);
- projectCache.evict(config.getProject());
+ projectCache.evictAndReindex(config.getProject());
}
}
diff --git a/lib/LICENSE-elasticsearch b/lib/LICENSE-elasticsearch
deleted file mode 100644
index 23cae9e..0000000
--- a/lib/LICENSE-elasticsearch
+++ /dev/null
@@ -1,5 +0,0 @@
-Elasticsearch
-Copyright 2009-2015 Elasticsearch
-
-This product includes software developed by The Apache Software
-Foundation (http://www.apache.org/).
diff --git a/lib/LICENSE-testcontainers b/lib/LICENSE-testcontainers
deleted file mode 100644
index 5d60e93..0000000
--- a/lib/LICENSE-testcontainers
+++ /dev/null
@@ -1,22 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2015 Richard North
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
-
diff --git a/lib/elasticsearch-rest-client/BUILD b/lib/elasticsearch-rest-client/BUILD
deleted file mode 100644
index e323263..0000000
--- a/lib/elasticsearch-rest-client/BUILD
+++ /dev/null
@@ -1,9 +0,0 @@
-load("@rules_java//java:defs.bzl", "java_library")
-
-package(default_visibility = ["//visibility:public"])
-
-java_library(
- name = "elasticsearch-rest-client",
- data = ["//lib:LICENSE-elasticsearch"],
- exports = ["@elasticsearch-rest-client//jar"],
-)
diff --git a/lib/httpcomponents/BUILD b/lib/httpcomponents/BUILD
index 07d4bb9..25e09c9 100644
--- a/lib/httpcomponents/BUILD
+++ b/lib/httpcomponents/BUILD
@@ -25,20 +25,3 @@
data = ["//lib:LICENSE-Apache2.0"],
exports = ["@httpcore//jar"],
)
-
-java_library(
- name = "httpasyncclient",
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = [
- "//java/com/google/gerrit/elasticsearch:__pkg__",
- "//javatests/com/google/gerrit/elasticsearch:__pkg__",
- ],
- exports = ["@httpasyncclient//jar"],
-)
-
-java_library(
- name = "httpcore-nio",
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = ["//java/com/google/gerrit/elasticsearch:__pkg__"],
- exports = ["@httpcore-nio//jar"],
-)
diff --git a/lib/jackson/BUILD b/lib/jackson/BUILD
deleted file mode 100644
index f11b96d..0000000
--- a/lib/jackson/BUILD
+++ /dev/null
@@ -1,20 +0,0 @@
-load("@rules_java//java:defs.bzl", "java_library")
-
-java_library(
- name = "jackson-annotations",
- testonly = True,
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = ["//visibility:public"],
- exports = ["@jackson-annotations//jar"],
-)
-
-java_library(
- name = "jackson-core",
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = [
- "//java/com/google/gerrit/acceptance:__pkg__",
- "//java/com/google/gerrit/elasticsearch:__pkg__",
- "//plugins:__pkg__",
- ],
- exports = ["@jackson-core//jar"],
-)
diff --git a/lib/log/BUILD b/lib/log/BUILD
index 96e140b..6a85bd1 100644
--- a/lib/log/BUILD
+++ b/lib/log/BUILD
@@ -4,7 +4,6 @@
name = "api",
data = ["//lib:LICENSE-slf4j"],
visibility = [
- "//javatests/com/google/gerrit/elasticsearch:__pkg__",
"//lib:__pkg__",
"//plugins:__pkg__",
],
diff --git a/lib/nongoogle_test.sh b/lib/nongoogle_test.sh
index 90d38b0..88ea414 100755
--- a/lib/nongoogle_test.sh
+++ b/lib/nongoogle_test.sh
@@ -14,10 +14,7 @@
backward-codecs
cglib-3_2
commons-io
-docker-java-api
-docker-java-transport
dropwizard-core
-duct-tape
eddsa
flogger
flogger-log4j-backend
@@ -27,13 +24,8 @@
guice-assistedinject
guice-library
guice-servlet
-httpasyncclient
-httpcore-nio
j2objc
-jackson-annotations
-jackson-core
jimfs
-jna
jruby
lucene-analyzers-common
lucene-core
@@ -47,13 +39,11 @@
sshd-mina
sshd-osgi
sshd-sftp
-testcontainers
truth
truth-java8-extension
truth-liteproto-extension
truth-proto-extension
tukaani-xz
-visible-assertions
xerces
EOF
diff --git a/lib/testcontainers/BUILD b/lib/testcontainers/BUILD
deleted file mode 100644
index 693a386..0000000
--- a/lib/testcontainers/BUILD
+++ /dev/null
@@ -1,64 +0,0 @@
-load("@rules_java//java:defs.bzl", "java_library")
-
-java_library(
- name = "docker-java-api",
- testonly = True,
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = ["//visibility:public"],
- exports = ["@docker-java-api//jar"],
-)
-
-java_library(
- name = "docker-java-transport",
- testonly = True,
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = ["//visibility:public"],
- exports = ["@docker-java-transport//jar"],
-)
-
-java_library(
- name = "duct-tape",
- testonly = True,
- data = ["//lib:LICENSE-testcontainers"],
- visibility = ["//visibility:public"],
- exports = ["@duct-tape//jar"],
-)
-
-java_library(
- name = "visible-assertions",
- testonly = True,
- data = ["//lib:LICENSE-testcontainers"],
- visibility = ["//visibility:public"],
- exports = ["@visible-assertions//jar"],
-)
-
-java_library(
- name = "jna",
- testonly = True,
- data = ["//lib:LICENSE-Apache2.0"],
- visibility = ["//visibility:public"],
- exports = ["@jna//jar"],
-)
-
-java_library(
- name = "testcontainers",
- testonly = True,
- data = ["//lib:LICENSE-testcontainers"],
- visibility = ["//visibility:public"],
- exports = ["@testcontainers//jar"],
- runtime_deps = [
- ":duct-tape",
- ":jna",
- ":visible-assertions",
- "//lib/log:ext",
- ],
-)
-
-java_library(
- name = "testcontainers-elasticsearch",
- testonly = True,
- data = ["//lib:LICENSE-testcontainers"],
- visibility = ["//visibility:public"],
- exports = ["@testcontainers-elasticsearch//jar"],
- runtime_deps = [":testcontainers"],
-)
diff --git a/plugins/BUILD b/plugins/BUILD
index 4b5343c..1271f04 100644
--- a/plugins/BUILD
+++ b/plugins/BUILD
@@ -82,7 +82,6 @@
"//lib/guice:javax_inject",
"//lib/httpcomponents:httpclient",
"//lib/httpcomponents:httpcore",
- "//lib/jackson:jackson-core",
"//lib:jgit-servlet",
"//lib:jgit",
"//lib:jsr305",
diff --git a/plugins/delete-project b/plugins/delete-project
index fed529c..fac8815 160000
--- a/plugins/delete-project
+++ b/plugins/delete-project
@@ -1 +1 @@
-Subproject commit fed529c129169d21fef98d1209b68b3bc3d10246
+Subproject commit fac8815949114d58b65dceda355bf80f7ec2adee
diff --git a/plugins/webhooks b/plugins/webhooks
index 9e4cd70..3b1ca2e 160000
--- a/plugins/webhooks
+++ b/plugins/webhooks
@@ -1 +1 @@
-Subproject commit 9e4cd708b96e4fab79b4101eaf3f3e6a8b872dca
+Subproject commit 3b1ca2e743ac74c3f1f4181b9b708b8fdf76c0af
diff --git a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor.ts b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor.ts
index 2d3b5c3..55f7567 100644
--- a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor.ts
+++ b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor.ts
@@ -23,7 +23,6 @@
import {PolymerElement} from '@polymer/polymer/polymer-element';
import {htmlTemplate} from './gr-plugin-config-array-editor_html';
import {property, customElement} from '@polymer/decorators';
-import {PolymerDeepPropertyChange} from '@polymer/polymer/interfaces';
import {
PluginConfigOptionsChangedEventDetail,
ArrayPluginOption,
@@ -55,19 +54,8 @@
@property({type: Object})
pluginOption!: ArrayPluginOption;
- @property({type: Boolean, computed: '_computeDisabled(pluginOption.*)'})
- disabled!: boolean; // _computeDisabled never returns null
-
- _computeDisabled(
- record: PolymerDeepPropertyChange<ArrayPluginOption, ArrayPluginOption>
- ) {
- return !(
- record &&
- record.base &&
- record.base.info &&
- record.base.info.editable
- );
- }
+ @property({type: Boolean, reflectToAttribute: true})
+ disabled = false;
_handleAddTap(e: MouseEvent) {
e.preventDefault();
diff --git a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_html.ts b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_html.ts
index c96b86c..7709198 100644
--- a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_html.ts
+++ b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_html.ts
@@ -85,6 +85,7 @@
id="input"
on-keydown="_handleInputKeydown"
bind-value="{{_newValue}}"
+ disabled$="[[disabled]]"
/>
</iron-input>
<gr-button
diff --git a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_test.js b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_test.js
index dfc191f..326ff44 100644
--- a/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_test.js
+++ b/polygerrit-ui/app/elements/admin/gr-plugin-config-array-editor/gr-plugin-config-array-editor_test.js
@@ -42,16 +42,6 @@
assert.equal(element._computeShowInputRow(false), '');
});
- test('_computeDisabled', () => {
- assert.isTrue(element._computeDisabled({}));
- assert.isTrue(element._computeDisabled({base: {}}));
- assert.isTrue(element._computeDisabled({base: {info: {}}}));
- assert.isTrue(
- element._computeDisabled({base: {info: {editable: false}}}));
- assert.isFalse(
- element._computeDisabled({base: {info: {editable: true}}}));
- });
-
suite('adding', () => {
setup(() => {
dispatchStub = sinon.stub(element, '_dispatchChanged');
@@ -60,11 +50,13 @@
test('with enter', () => {
element._newValue = '';
MockInteractions.pressAndReleaseKeyOn(element.$.input, 13); // Enter
+ assert.isFalse(element.$.input.hasAttribute('disabled'));
flush();
assert.isFalse(dispatchStub.called);
element._newValue = 'test';
MockInteractions.pressAndReleaseKeyOn(element.$.input, 13); // Enter
+ assert.isFalse(element.$.input.hasAttribute('disabled'));
flush();
assert.isTrue(dispatchStub.called);
@@ -91,6 +83,7 @@
test('deleting', () => {
dispatchStub = sinon.stub(element, '_dispatchChanged');
element.pluginOption = {info: {values: ['test', 'test2']}};
+ element.disabled = true;
flush();
const rows = getAll('.existingItems .row');
@@ -101,7 +94,7 @@
flush();
assert.isFalse(dispatchStub.called);
- element.pluginOption.info.editable = true;
+ element.disabled = false;
element.notifyPath('pluginOption.info.editable');
flush();
diff --git a/polygerrit-ui/app/elements/admin/gr-repo-plugin-config/gr-repo-plugin-config.ts b/polygerrit-ui/app/elements/admin/gr-repo-plugin-config/gr-repo-plugin-config.ts
index 37c88f2..f9f760c 100644
--- a/polygerrit-ui/app/elements/admin/gr-repo-plugin-config/gr-repo-plugin-config.ts
+++ b/polygerrit-ui/app/elements/admin/gr-repo-plugin-config/gr-repo-plugin-config.ts
@@ -66,6 +66,9 @@
@property({type: Object})
pluginData?: PluginData;
+ @property({type: Boolean, reflect: true})
+ disabled = false;
+
static override get styles() {
return [
sharedStyles,
@@ -145,6 +148,7 @@
<gr-plugin-config-array-editor
@plugin-config-option-changed=${this._handleArrayChange}
.pluginOption="${option}"
+ ?disabled=${this.disabled || !option.info.editable}
></gr-plugin-config-array-editor>
`;
} else if (option.info.type === ConfigParameterInfoType.BOOLEAN) {
@@ -153,7 +157,7 @@
?checked=${this._computeChecked(option.info.value)}
@change=${this._handleBooleanChange}
data-option-key=${option._key}
- ?disabled=${!option.info.editable}
+ ?disabled=${this.disabled || !option.info.editable}
@click=${this._onTapPluginBoolean}
></paper-toggle-button>
`;
@@ -165,7 +169,7 @@
>
<select
data-option-key=${option._key}
- ?disabled=${!option.info.editable}
+ ?disabled=${this.disabled || !option.info.editable}
>
${(option.info.permitted_values || []).map(
value => html`<option value="${value}">${value}</option>`
@@ -188,7 +192,7 @@
.value="${option.info.value ?? ''}"
@input=${this._handleStringChange}
data-option-key="${option._key}"
- ?disabled=${!option.info.editable}
+ ?disabled=${this.disabled || !option.info.editable}
/>
</iron-input>
`;
diff --git a/polygerrit-ui/app/elements/admin/gr-repo/gr-repo.ts b/polygerrit-ui/app/elements/admin/gr-repo/gr-repo.ts
index 1076990..3292cd7 100644
--- a/polygerrit-ui/app/elements/admin/gr-repo/gr-repo.ts
+++ b/polygerrit-ui/app/elements/admin/gr-repo/gr-repo.ts
@@ -736,7 +736,10 @@
<h3 class="heading-3">Plugins</h3>
${pluginData.map(
item => html`
- <gr-repo-plugin-config .pluginData=${item}></gr-repo-plugin-config>
+ <gr-repo-plugin-config
+ .pluginData=${item}
+ ?disabled=${this.readOnly}
+ ></gr-repo-plugin-config>
`
)}
</div>`;
diff --git a/resources/com/google/gerrit/httpd/auth/container/LoginRedirect.html b/resources/com/google/gerrit/httpd/auth/container/LoginRedirect.html
index 0567468..54c3661 100644
--- a/resources/com/google/gerrit/httpd/auth/container/LoginRedirect.html
+++ b/resources/com/google/gerrit/httpd/auth/container/LoginRedirect.html
@@ -4,6 +4,12 @@
<title>Gerrit Code Review</title>
<script type="text/javascript">
var href = window.location.href;
+ var query = "";
+ var q = href.indexOf('?');
+ if (q >= 0) {
+ query = href.substring(q);
+ href = href.substring(0,q);
+ }
var p = href.indexOf('#');
var token;
if (p >= 0) {
@@ -12,7 +18,7 @@
} else {
token = '';
}
- window.location.replace(href + 'login/' + token);
+ window.location.replace(href + 'login/' + token + query);
</script>
</head>
<body>
diff --git a/tools/deps.bzl b/tools/deps.bzl
index f64efff..b32293e 100644
--- a/tools/deps.bzl
+++ b/tools/deps.bzl
@@ -1,5 +1,4 @@
load("//tools/bzl:maven_jar.bzl", "GERRIT", "maven_jar")
-load("//tools:nongoogle.bzl", "TESTCONTAINERS_VERSION")
CAFFEINE_VERS = "2.8.5"
ANTLR_VERS = "3.5.2"
@@ -11,7 +10,7 @@
MIME4J_VERS = "0.8.1"
OW2_VERS = "9.0"
AUTO_VALUE_VERSION = "1.7.4"
-AUTO_VALUE_GSON_VERSION = "1.3.0"
+AUTO_VALUE_GSON_VERSION = "1.3.1"
PROLOG_VERS = "1.4.4"
PROLOG_REPO = GERRIT
GITILES_VERS = "0.4-1"
@@ -467,19 +466,19 @@
maven_jar(
name = "auto-value-gson-runtime",
artifact = "com.ryanharter.auto.value:auto-value-gson-runtime:" + AUTO_VALUE_GSON_VERSION,
- sha1 = "a69a9db5868bb039bd80f60661a771b643eaba59",
+ sha1 = "addda2ae6cce9f855788274df5de55dde4de7b71",
)
maven_jar(
name = "auto-value-gson-extension",
artifact = "com.ryanharter.auto.value:auto-value-gson-extension:" + AUTO_VALUE_GSON_VERSION,
- sha1 = "6a61236d17b58b05e32b4c532bcb348280d2212b",
+ sha1 = "0c4c01a3e10e5b10df2e5f5697efa4bb3f453ac1",
)
maven_jar(
name = "auto-value-gson-factory",
artifact = "com.ryanharter.auto.value:auto-value-gson-factory:" + AUTO_VALUE_GSON_VERSION,
- sha1 = "b1f01918c0d6cb1f5482500e6b9e62589334dbb0",
+ sha1 = "9ed8d79144ee8d60cc94cc11f847b5ed8ee9f19c",
)
maven_jar(
@@ -603,13 +602,6 @@
sha1 = "fd369423346b2f1525c413e33f8cf95b09c92cbd",
)
- # Base the following org.apache.httpcomponents versions on what
- # elasticsearch-rest-client explicitly depends on, except for
- # commons-codec (non-http) which is not necessary yet. Note that
- # below httpcore version(s) differs from the HTTPCOMP_VERS range,
- # upstream: that specific dependency has no HTTPCOMP_VERS version
- # equivalent currently.
-
maven_jar(
name = "fluent-hc",
artifact = "org.apache.httpcomponents:fluent-hc:" + HTTPCOMP_VERS,
@@ -737,19 +729,3 @@
artifact = "org.objenesis:objenesis:3.0.1",
sha1 = "11cfac598df9dc48bb9ed9357ed04212694b7808",
)
-
- # When upgrading elasticsearch-rest-client, also upgrade httpcore-nio
- # and httpasyncclient as necessary in tools/nongoogle.bzl. Consider
- # also the other org.apache.httpcomponents dependencies in
- # WORKSPACE.
- maven_jar(
- name = "elasticsearch-rest-client",
- artifact = "org.elasticsearch.client:elasticsearch-rest-client:7.8.1",
- sha1 = "59feefe006a96a39f83b0dfb6780847e06c1d0a8",
- )
-
- maven_jar(
- name = "testcontainers-elasticsearch",
- artifact = "org.testcontainers:elasticsearch:" + TESTCONTAINERS_VERSION,
- sha1 = "595e3a50f59cd3c1d281ca6c1bc4037e277a1353",
- )
diff --git a/tools/eclipse/BUILD b/tools/eclipse/BUILD
index 61ea4fe..cd9f132 100644
--- a/tools/eclipse/BUILD
+++ b/tools/eclipse/BUILD
@@ -9,7 +9,6 @@
)
TEST_DEPS = [
- "//javatests/com/google/gerrit/elasticsearch:elasticsearch_test_utils",
"//javatests/com/google/gerrit/server:server_tests",
]
diff --git a/tools/maven/gerrit-acceptance-framework_pom.xml b/tools/maven/gerrit-acceptance-framework_pom.xml
index 3705407..7fc0b01 100644
--- a/tools/maven/gerrit-acceptance-framework_pom.xml
+++ b/tools/maven/gerrit-acceptance-framework_pom.xml
@@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.gerrit</groupId>
<artifactId>gerrit-acceptance-framework</artifactId>
- <version>3.5.0-SNAPSHOT</version>
+ <version>3.5.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Gerrit Code Review - Acceptance Test Framework</name>
<description>Framework for Gerrit's acceptance tests</description>
@@ -77,6 +77,9 @@
<name>Patrick Hiesel</name>
</developer>
<developer>
+ <name>Patrick Mulhall</name>
+ </developer>
+ <developer>
<name>Saša Živkov</name>
</developer>
<developer>
@@ -85,6 +88,9 @@
<developer>
<name>Tao Zhou</name>
</developer>
+ <developer>
+ <name>Thomas Dräbing</name>
+ </developer>
</developers>
<mailingLists>
diff --git a/tools/maven/gerrit-extension-api_pom.xml b/tools/maven/gerrit-extension-api_pom.xml
index ba35ca2..9db1d06 100644
--- a/tools/maven/gerrit-extension-api_pom.xml
+++ b/tools/maven/gerrit-extension-api_pom.xml
@@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.gerrit</groupId>
<artifactId>gerrit-extension-api</artifactId>
- <version>3.5.0-SNAPSHOT</version>
+ <version>3.5.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Gerrit Code Review - Extension API</name>
<description>API for Gerrit Extensions</description>
@@ -77,6 +77,9 @@
<name>Patrick Hiesel</name>
</developer>
<developer>
+ <name>Patrick Mulhall</name>
+ </developer>
+ <developer>
<name>Saša Živkov</name>
</developer>
<developer>
@@ -85,6 +88,9 @@
<developer>
<name>Tao Zhou</name>
</developer>
+ <developer>
+ <name>Thomas Dräbing</name>
+ </developer>
</developers>
<mailingLists>
diff --git a/tools/maven/gerrit-plugin-api_pom.xml b/tools/maven/gerrit-plugin-api_pom.xml
index b7954c7..833ed51 100644
--- a/tools/maven/gerrit-plugin-api_pom.xml
+++ b/tools/maven/gerrit-plugin-api_pom.xml
@@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.gerrit</groupId>
<artifactId>gerrit-plugin-api</artifactId>
- <version>3.5.0-SNAPSHOT</version>
+ <version>3.5.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Gerrit Code Review - Plugin API</name>
<description>API for Gerrit Plugins</description>
@@ -77,6 +77,9 @@
<name>Patrick Hiesel</name>
</developer>
<developer>
+ <name>Patrick Mulhall</name>
+ </developer>
+ <developer>
<name>Saša Živkov</name>
</developer>
<developer>
@@ -85,6 +88,9 @@
<developer>
<name>Tao Zhou</name>
</developer>
+ <developer>
+ <name>Thomas Dräbing</name>
+ </developer>
</developers>
<mailingLists>
diff --git a/tools/maven/gerrit-war_pom.xml b/tools/maven/gerrit-war_pom.xml
index 118cf39..93adc12 100644
--- a/tools/maven/gerrit-war_pom.xml
+++ b/tools/maven/gerrit-war_pom.xml
@@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.gerrit</groupId>
<artifactId>gerrit-war</artifactId>
- <version>3.5.0-SNAPSHOT</version>
+ <version>3.5.1-SNAPSHOT</version>
<packaging>war</packaging>
<name>Gerrit Code Review - WAR</name>
<description>Gerrit WAR</description>
@@ -77,6 +77,9 @@
<name>Patrick Hiesel</name>
</developer>
<developer>
+ <name>Patrick Mulhall</name>
+ </developer>
+ <developer>
<name>Saša Živkov</name>
</developer>
<developer>
@@ -85,6 +88,9 @@
<developer>
<name>Tao Zhou</name>
</developer>
+ <developer>
+ <name>Thomas Dräbing</name>
+ </developer>
</developers>
<mailingLists>
diff --git a/tools/nongoogle.bzl b/tools/nongoogle.bzl
index 80db2fa..0637479 100644
--- a/tools/nongoogle.bzl
+++ b/tools/nongoogle.bzl
@@ -8,8 +8,6 @@
GUAVA_DOC_URL = "https://google.github.io/guava/releases/" + GUAVA_VERSION + "/api/docs/"
-TESTCONTAINERS_VERSION = "1.15.3"
-
def declare_nongoogle_deps():
"""loads dependencies that are not used at Google.
@@ -69,20 +67,6 @@
sha1 = "22799941ec7bd5170ea890363cb968e400a69c41",
)
- # elasticsearch-rest-client explicitly depends on this version
- maven_jar(
- name = "httpasyncclient",
- artifact = "org.apache.httpcomponents:httpasyncclient:4.1.4",
- sha1 = "f3a3240681faae3fa46b573a4c7e50cec9db0d86",
- )
-
- # elasticsearch-rest-client explicitly depends on this version
- maven_jar(
- name = "httpcore-nio",
- artifact = "org.apache.httpcomponents:httpcore-nio:4.4.12",
- sha1 = "84cd29eca842f31db02987cfedea245af020198b",
- )
-
maven_jar(
name = "openid-consumer",
artifact = "org.openid4java:openid4java:1.0.0",
@@ -109,12 +93,6 @@
)
maven_jar(
- name = "jackson-core",
- artifact = "com.fasterxml.jackson.core:jackson-core:2.12.0",
- sha1 = "afe52c6947d9939170da7989612cef544115511a",
- )
-
- maven_jar(
name = "commons-io",
artifact = "commons-io:commons-io:2.4",
sha1 = "b1b6ea3b7e4aa4f492509a4952029cd8e48019ad",
@@ -195,52 +173,6 @@
sha1 = "dc13ae4faca6df981fc7aeb5a522d9db446d5d50",
)
- DOCKER_JAVA_VERS = "3.2.8"
-
- maven_jar(
- name = "docker-java-api",
- artifact = "com.github.docker-java:docker-java-api:" + DOCKER_JAVA_VERS,
- sha1 = "4ac22a72d546a9f3523cd4b5fabffa77c4a6ec7c",
- )
-
- maven_jar(
- name = "docker-java-transport",
- artifact = "com.github.docker-java:docker-java-transport:" + DOCKER_JAVA_VERS,
- sha1 = "c3b5598c67d0a5e2e780bf48f520da26b9915eab",
- )
-
- # https://github.com/docker-java/docker-java/blob/3.2.8/pom.xml#L61
- # <=> DOCKER_JAVA_VERS
- maven_jar(
- name = "jackson-annotations",
- artifact = "com.fasterxml.jackson.core:jackson-annotations:2.10.3",
- sha1 = "0f63b3b1da563767d04d2e4d3fc1ae0cdeffebe7",
- )
-
- maven_jar(
- name = "testcontainers",
- artifact = "org.testcontainers:testcontainers:" + TESTCONTAINERS_VERSION,
- sha1 = "95c6cfde71c2209f0c29cb14e432471e0b111880",
- )
-
- maven_jar(
- name = "duct-tape",
- artifact = "org.rnorth.duct-tape:duct-tape:1.0.8",
- sha1 = "92edc22a9ab2f3e17c9bf700aaee377d50e8b530",
- )
-
- maven_jar(
- name = "visible-assertions",
- artifact = "org.rnorth.visible-assertions:visible-assertions:2.1.2",
- sha1 = "20d31a578030ec8e941888537267d3123c2ad1c1",
- )
-
- maven_jar(
- name = "jna",
- artifact = "net.java.dev.jna:jna:5.5.0",
- sha1 = "0e0845217c4907822403912ad6828d8e0b256208",
- )
-
maven_jar(
name = "jimfs",
artifact = "com.google.jimfs:jimfs:1.2",
diff --git a/tools/release_noter/release_noter.py b/tools/release_noter/release_noter.py
index 73c1a05..167f68a 100644
--- a/tools/release_noter/release_noter.py
+++ b/tools/release_noter/release_noter.py
@@ -172,7 +172,6 @@
)
doc = Component("Documentation", {"document"})
jgit = Component("JGit", {"jgit"})
- elastic = Component("Elasticsearch", {"elastic"})
deps = Component("Other dependency", {"upgrade", "dependenc"})
otherwise = Component("Other core", {})
diff --git a/version.bzl b/version.bzl
index f2e0d0c..10de529 100644
--- a/version.bzl
+++ b/version.bzl
@@ -2,4 +2,4 @@
# Used by :api_install and :api_deploy targets
# when talking to the destination repository.
#
-GERRIT_VERSION = "3.5.0-SNAPSHOT"
+GERRIT_VERSION = "3.5.1-SNAPSHOT"
