Documentation/dev-bazel.txt

:linkattrs:
= Gerrit Code Review - Building with Bazel

[[summary]]
== TL;DR

If you have the prerequisites, running

```
  $ bazel build gerrit
```

should generate a .war file under `bazel-bin/gerrit.war`.

[[installation]]
== Prerequisites

To build Gerrit from source, you need:

* A Linux or macOS system (Windows is not supported)
* A JDK for Java 25
* Python 3
* link:https://github.com/nodesource/distributions/blob/master/README.md[Node.js (including npm),role=external,window=_blank]
* Yarn (`npm install -g yarn`)
* link:https://bazel.build/install[Bazel,role=external,window=_blank] -launched with
link:https://github.com/bazelbuild/bazelisk[Bazelisk,role=external,window=_blank]
* Maven
* zip, unzip
* curl
* gcc

[[bazel]]
=== Bazel

link:https://github.com/bazelbuild/bazelisk[Bazelisk,role=external,window=_blank] is a version
manager for link:https://bazel.build/[Bazel,role=external,window=_blank], similar to how `nvm`
manages `npm` versions. It takes care of downloading and installing Bazel itself, so you don't have
to worry about using the correct version of Bazel. One particular advantage to
using Bazelisk is that you can jump between different versions of Gerrit and not
worry about which version of Bazel you need.

Bazelisk can be installed in different ways:
link:https://bazel.build/install/bazelisk[Bazelisk Installation,role=external,window=_blank].
To execute the correct version of Bazel using Bazelisk you simply replace
the `bazel` command with `bazelisk`.

[[java]]
=== Java

Ensure that the link:install.html#prerequisites[required Java version]
is installed and that `JAVA_HOME` is set to it.

To check the installed version of Java, open a terminal window and run:

`java -version`

[[java-25]]
==== Java 25 support

To build Gerrit with Java 25 language level, run:

```
  $ bazelisk build :release
```

To execute the tests run:

```
  $ bazelisk test //...
```

=== Node.js and npm packages
See link:https://gerrit.googlesource.com/gerrit/+/master/polygerrit-ui/README.md#installing-node_js-and-npm-packages[Installing Node.js and npm packages,role=external,window=_blank].

[[build]]
== Building on the Command Line

=== Bazel module lockfile strictness

Gerrit uses Bazel modules with a checked-in lockfile (`MODULE.bazel.lock`).
To avoid accidental desynchronization between `MODULE.bazel` and the lockfile,
the build runs with strict lockfile checking:

----
--lockfile_mode=error
----

With this mode, Bazel will fail the build if the lockfile is out of date.

If you see an error like:

----
ERROR: Missing checksum for registry file ... not permitted with --lockfile_mode=error
----

it means that `MODULE.bazel.lock` needs to be updated.

To update the lockfile explicitly, run:

```
  $ bazelisk mod deps --lockfile_mode=update
```

and commit the resulting changes to `MODULE.bazel.lock`.

Note: Changes that do not modify `MODULE.bazel` should normally not modify
`MODULE.bazel.lock`. Any lockfile update should be intentional and reviewed.

=== Gerrit Development WAR File

To build the Gerrit web application:

----
  bazelisk build gerrit
----

The output executable WAR will be placed in:

----
  bazel-bin/gerrit.war
----

[[release]]
=== Gerrit Release WAR File

To build the Gerrit web application that includes the Gerrit UI,
core plugins and documentation:

----
  bazelisk build release
----

The output executable WAR will be placed in:

----
  bazel-bin/release.war
----

=== Headless Mode

To build Gerrit in headless mode, i.e. without the Gerrit UI:

----
  bazelisk build headless
----

The output executable WAR will be placed in:

----
  bazel-bin/headless.war
----

=== Extension and Plugin API JAR Files

To build the extension, plugin and acceptance-framework JAR files:

----
  bazelisk build api
----

The output archive that contains Java binaries, Java sources and
Java docs will be placed in:

----
  bazel-bin/api.zip
----

Install {extension,plugin,acceptance-framework}-api to the local
maven repository:

----
  tools/maven/api.sh install
----

Install gerrit.war to the local maven repository:

----
  tools/maven/api.sh war_install
----

=== Plugins

----
  bazelisk build plugins:core
----

The output JAR files for individual plugins will be placed in:

----
  bazel-bin/plugins/<name>/<name>.jar
----

The JAR files will also be packaged in:

----
  bazel-bin/plugins/core.zip
----

To build a specific plugin:

----
  bazelisk build plugins/<name>
----

The output JAR file will be placed in:

----
  bazel-bin/plugins/<name>/<name>.jar
----

Note that when building an individual plugin, the `core.zip` package
is not regenerated.

[[IDEs]]
== Using an IDE.

=== IntelliJ

The Gerrit build works with Bazel's link:https://ij.bazel.build[IntelliJ plugin,role=external,window=_blank].
Please follow the instructions on <<dev-intellij#,IntelliJ Setup>>.

=== Eclipse

==== Generating the Eclipse Project

Create the Eclipse project:

----
  tools/eclipse/project.py
----

and then follow the link:dev-eclipse.html#setup[setup instructions].

==== Refreshing the Classpath

If an updated classpath is needed, the Eclipse project can be
refreshed and missing dependency JARs can be downloaded by running
`project.py` again. For IntelliJ, you need to click the `Sync Project
with BUILD Files` button of link:https://ij.bazel.build[Bazel plugin,role=external,window=_blank].

[[documentation]]
== Documentation

To build only the documentation for testing or static hosting:

----
  bazelisk build Documentation:searchfree
----

The html files will be bundled into `searchfree.zip` in this location:

----
  bazel-bin/Documentation/searchfree.zip
----

To use local fonts with the searchfree target:

----
  bazel build Documentation:searchfree_safe
----

The html files will be bundled into `searchfree.zip` or `searchfree_safe.zip` in this location:

----
  bazel-bin/Documentation/searchfree.zip
  bazel-bin/Documentation/searchfree_safe.zip
----

To generate HTML files skipping the zip archiving:

----
  bazelisk build Documentation
----

And open `bazel-bin/Documentation/index.html`.

To build the Gerrit executable WAR with the documentation included:

----
  bazelisk build withdocs
----

The WAR file will be placed in:

----
  bazel-bin/withdocs.war
----

Alternatively, one can generate the documentation as flat files:

----
  bazelisk build Documentation:Documentation
----

The html, css, js files are placed in:

----
 `bazel-bin/Documentation/`
----

[[license]]
== License

To check that licenses are up to date, run:

----
  bazelisk test Documentation:check_licenses
----

You can then copy by doing:

----
  cp bazel-bin/Documentation/licenses.gen.txt Documentation/licenses.txt
  cp bazel-bin/Documentation/backend_licenses.gen.txt Documentation/backend_licenses.txt
  cp bazel-bin/Documentation/js_licenses.gen.txt Documentation/js_licenses.txt
----

Note: Follow link:#npm-binary[NPM Binaries] for adding npm package dependencies licenses.
      This is for js_licenses.txt.

[[release-war-metadata]]
== Release WAR metadata outputs

The `pkg_war` rule emits deterministic metadata files describing the
contents of a generated WAR. These files are produced without
materializing or unpacking the WAR archive and are intended for checks
and diagnostics.

=== <name>.war.jars.txt

This file contains the normalized, version-agnostic identifiers of
third-party JARs that would be packaged into the WAR.

It is used by the `//Documentation:check_release_war_jars` test as a
guardrail against unintended changes to the runtime dependency set
shipped in `release.war`.

The identifiers are normalized to remain stable across dependency
upgrades (for example by stripping version suffixes and excluding
Gerrit-internal artifacts).

=== <name>.war.entries.txt

This file lists the full entry paths of all JAR files as they would
appear inside the WAR (for example `WEB-INF/lib/foo.jar` or
`WEB-INF/pgm-lib/bar.jar`).

This output is intended as a lightweight diagnostic aid and for future
checks on WAR layout. It allows inspection of:
* which JARs are packaged
* where they are placed inside the WAR (lib vs pgm-lib)
* whether duplicate entries or unintended version skews (multiple
  versions of the same library) are introduced

without requiring the WAR to be built or unpacked.

At present, this file is not consumed by an automated test, but it is
kept to support debugging and potential future validation of WAR
structure.

[[release-war-jars]]
== Release WAR dependency allowlist

The set of third-party JARs packaged into `release.war` is checked by the
`//Documentation:check_release_war_jars` test. This test acts as a guardrail
to detect unintended changes to the runtime dependency set shipped in Gerrit
releases.

To run the check:

----
  bazelisk test //Documentation:check_release_war_jars
----

If the test fails with:

----
  WAR packaged third-party JAR set changed
----

the runtime dependency set of `release.war` has changed.

If the change is expected and has been reviewed, refresh the allowlist:

----
  bazelisk build //:release.war.jars.txt
  cp bazel-bin/release.war.jars.txt Documentation/release_war_jars.txt
----

Changes to the packaged dependency set may also require updating license
metadata. All runtime dependencies that are shipped in `release.war` must be
reachable from the Bazel dependency graph so that license checks can locate
them.

When adding or modifying dependencies:
* Ensure new runtime dependencies are wired explicitly (for example as
  `runtime_deps`) into the appropriate `//lib` wrapper targets.
* Update or add license metadata as needed.
* Run the license check:

----
  bazelisk test //Documentation:check_licenses
----

[[tests]]
== Running Unit Tests

Bazel BUILD files define test targets for Gerrit. You can run all declared
test targets with:

----
  bazelisk test --build_tests_only //...
----

[[testgroups]]
=== Running Test Groups

To run one or more specific labeled groups of tests:

----
  bazelisk test --test_tag_filters=api,git //...
----

The following label values are currently supported for the group name:

* annotation
* api
* edit
* git
* git-protocol-v2
* git-upload-archive
* notedb
* pgm
* rest
* server
* ssh

We can also select tests within a specific BUILD target group. For example
`javatests/com/google/gerrit/acceptance/rest/account/BUILD` declares a
rest_account test target group:

----
  bazelisk test //javatests/com/google/gerrit/acceptance/rest/account:rest_account
----

[[debugtests]]
=== Debugging Tests

To debug specific tests you will need to select the test target containing
that test then use `--test_filter` to select the specific test you want.
This `--test_filter` is a regex and can be used to select multiple tests
out of the target:

----
  bazelisk test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget
----

For example `javatests/com/google/gerrit/acceptance/api/change/BUILD`
defines a test target group for every `*IT.java` file in the directory.
We can execute the single `getAmbiguous()` test found in ChangeIT.java using
this `--test_filter` and target:

----
  bazelisk test --test_output=streamed \
    --test_filter=com.google.gerrit.acceptance.api.change.ChangeIT.getAmbiguous \
    //javatests/com/google/gerrit/acceptance/api/change:ChangeIT
----

[[additionaltestfiltering]]
=== Additional Test Filtering

To run only tests that do not use SSH:

----
  bazelisk test --test_env=GERRIT_USE_SSH=NO //...
----

To exclude tests that have been marked as flaky:

----
  bazelisk test --test_tag_filters=-flaky //...
----

To exclude tests that require very recent git client version:

----
  bazelisk test --test_tag_filters=-git-protocol-v2 //...
----

To run the tests against a specific index backend (LUCENE, FAKE):
----
  bazelisk test --test_env=GERRIT_INDEX_TYPE=LUCENE //...
----

Bazel itself supports a multitude of ways to
link:https://bazel.build/run/build#specifying-build-targets[specify targets,role=external,window=_blank]
for fine-grained test selection that can be combined with many of the examples
above.

[[testcaching]]
=== Test Caching

By default Bazel caches test results and will not reexecute tests unless they
or their dependencies have been modified. To ignore cached test results and
force the tests to rerun:

----
  bazelisk test --cache_test_results=NO //...
----

[[plugintests]]
=== Running Plugin Tests

Running tests for Gerrit plugins follows the process above. From within the
Gerrit project root with the desired plugins checked out into `plugins/` we
execute Bazel with the appropriate target:

----
  bazelisk test //plugins/replication/...
----

[[debugging-tests]]
== Debugging Unit Tests
In some cases it may be necessary to debug a test while running it in bazel. For example, when we
observe a different test result in Eclipse and bazel. Using the `--java_debug` option will start the
JVM in debug mode and await for a remote debugger to attach.

Example:
[source,bash]
----
  bazelisk test --java_debug --test_tag_filters=delete-project //...
  ...
  Listening for transport dt_socket at address: 5005
  ...
----

Now attach with a debugger to the port `5005`. For example use "Remote Java Application" launch
configuration in Eclipse and specify the port `5005`.

[[logging]]
=== Controlling logging level

Per default, logging level is set to `INFO` level for all tests. The `DEBUG`
log level can be enabled for the tests.

In IDE, set `-Dgerrit.logLevel=debug` as a VM argument. With `bazel`, pass
`GERRIT_LOG_LEVEL=debug` environment variable:

----
  bazelisk test --test_filter=com.google.gerrit.server.notedb.ChangeNotesTest \
    --test_env=GERRIT_LOG_LEVEL=debug \
    javatests/com/google/gerrit/server:server_tests
----

The log results can be found in:
`bazel-testlogs/javatests/com/google/gerrit/server/server_tests/test.log`.


== Dependencies

Dependency JARs are normally downloaded as needed, but you can
download everything upfront.  This is useful to enable
subsequent builds to run without network access:

----
  bazelisk fetch //...
----

When downloading from behind a proxy (which is common in some corporate
environments), it might be necessary to explicitly specify the proxy that
is then used by `curl`:

----
  export http_proxy=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
----

== Adding and/or updating Maven dependencies

Maven dependencies are maintained in two separate files: `tools/deps.toml` and
`tools/nongoogle.toml`. Depending on whether Google will use the dependency
or not, dependencies should be added to one or the other file. If unsure, the
dependency should be added to `tools/deps.toml`. A Googler will check the dependency
during review and provide feedback.

When adding a new maven dependency add a line like this:

[source,toml]
----
[libraries]
commons-codec = { module = "commons-codec:commons-codec", version = "1.15" }
----

To update a dependency, change the corresponding version in the above mentioned
toml-files.

In both cases, the dependencies have to be repinned. To do that, run:

[source,bash]
----
REPIN=1 bazel run @external_deps//:pin
----

After repinning, verify that the new or updated artifacts are available
on Gerrit's Maven mirror (see <<mirror-coverage-check>>), so the build
does not depend on Maven Central. A missing artifact has to be uploaded
to the mirror before the dependency can be relied on.

[[gerrit-maven-mirror]]
== Gerrit Maven mirror

Gerrit's `maven.install()` prefers Gerrit's Maven mirror before Maven
Central. Maven Central remains configured as a fallback while the mirror
is being populated.

The repository order is part of `external_deps.lock.json`. After
changing the repository order in `tools/java_deps.MODULE.bazel`, repin
the RJE lock; changing `MODULE.bazel` alone does not reorder generated
`http_file.urls`.

[[mirror-coverage-check]]
=== Checking mirror coverage

To verify that every artifact pinned in `external_deps.lock.json` is
present on the mirror, run the coverage checker through Bazel so it
behaves identically locally and in CI:

[source,bash]
----
bazel run //tools/bzl:gerrit-maven-mirror-check
----

`bazel run` sets `BUILD_WORKSPACE_DIRECTORY`, so the checker locates the
lock file at the workspace root regardless of the current directory. It
sends parallel `HEAD` requests and never downloads artifacts, so it puts
negligible load on the mirror; the checker only probes Gerrit's Maven
mirror and does not fetch artifacts from Maven Central. The command exits
with:

* `0` when all locked artifacts are mirrored;
* `1` when one or more artifacts are genuinely missing from the mirror (a
  `404`). Each non-mirrored artifact is printed as a tab-separated
  `status`, `coordinate` and `url`;
* `2` on an infrastructure or tool error: the lock file cannot be read or
  parsed, or a probe failed with a network or mirror error (timeout,
  connection refused, `5xx`, `429`). This is inconclusive rather than a
  coverage miss, so retry or investigate CI/network rather than treating
  it as missing.

A summary is printed to stderr, for example `checked=298 mirrored=298
missing=0 errors=0`. Use `--include-ok` to also print mirrored artifacts,
`--mirror` to check another repository, and `--lock-file` to check a
different RJE lock. The script can also be invoked directly from the
repository root as `./tools/bzl/gerrit_maven_mirror_check.py`.

To verify that a known mirrored artifact can be fetched without using
Maven Central, create a Bazel downloader config that blocks Central and
run a cold-cache fetch:

[source,bash]
----
cat >/tmp/download.cfg <<'EOF'
block repo1.maven.org
EOF

repo='@@rules_jvm_external++maven+'
repo="${repo}com_google_guava_guava_33_5_0_jre"
target="${repo}//file:file"

bazelisk --batch --output_user_root=/tmp/gerrit-mirror-fetch \
  fetch --repository_cache=/tmp/gerrit-mirror-cache \
  --downloader_config=/tmp/download.cfg \
  "$target"
----

== Building against unpublished Maven JARs

To build against unpublished Maven JARs, like PrologCafe, the custom JARs must
be installed in the local Maven repository (`mvn clean install`) and the local
repository has to be added to the `maven.install()`-call in `MODULE.bazel`:

[source,python]
----
  maven.install(
      artifacts = [],
      repositories = [
          "file://$HOME/.m2/repository",
          "https://gerrit-maven.storage.googleapis.com",
          "https://repo1.maven.org/maven2",
      ],
      ...
  )
----

== Building against artifacts from custom Maven repositories

The URL of custom Maven repository can be directly passed to the `maven.install`
function:

[source,python]
----
  GERRIT_FORGE = 'http://gerritforge.com/snapshot'

  maven.install(
      artifacts = [],
      repositories = [
          GERRIT_FORGE,
          "https://gerrit-maven.storage.googleapis.com",
          "https://repo1.maven.org/maven2",
      ],
      ...
  )
----

Note, that these repositories will apply to all artifacts being used.

== Building against SNAPSHOT Maven JARs

To build against SNAPSHOT Maven JARs, the complete SNAPSHOT version must be used:

[source,toml]
----
[libraries]
commons-codec = { module = "commons-codec:commons-codec", version = "1.15-SNAPSHOT-20190112.120241-16" }
----

[[bazel-local-caches]]

To accelerate builds, several caches are activated per default:

* ~/.gerritcodereview/bazel-cache/repository
* ~/.gerritcodereview/bazel-cache/cas

The cache locations can be adjusted with `bazelisk build` options
`--repository_cache` and `--disk_cache` respectively.

Currently none of these caches have a maximum size limit. See
link:https://github.com/bazelbuild/bazel/issues/5139[this bazel issue,role=external,window=_blank] for
details. Users should watch the cache sizes and clean them manually if
necessary.

[[npm-binary]]
== NPM Binaries

Parts of the Gerrit web app build require running NPM-based JavaScript programs
as "binaries". We don't attempt to resolve and download NPM dependencies at
build time, but instead use pre-built bundles of the NPM binary along with all
its dependencies. Some packages on
link:https://docs.npmjs.com/misc/registry[registry.npmjs.org,role=external,window=_blank] come with their
dependencies bundled, but this is the exception rather than the rule. More
commonly, to add a new binary to this list, you will need to bundle the binary
yourself.

[NOTE]
We can only use binaries that meet certain licensing requirements, and that do
not include any native code.

Start by checking that the license and file types of the bundle are acceptable:
[source,bash]
----
  gerrit_repo=/path/to/gerrit
  package=some-npm-package
  version=1.2.3

  # Note - yarn must be installed before running the following commands
  yarn global add license-checker && \
  rm -rf /tmp/$package-$version && mkdir -p /tmp/$package-$version && \
  cd /tmp/$package-$version && \
  yarn add $package@$version && \
  license-checker | grep licenses: | sort -u
----

This will output a list of the different licenses used by the package and all
its transitive dependencies. We can only legally distribute a bundle via our
storage bucket if the licenses allow us to do so. As long as all of the listed
license are allowed by
link:https://opensource.google.com/docs/thirdparty/licenses/[Google's
standards,role=external,window=_blank]. Any `by_exception_only`, commercial, prohibited, or unlisted
licenses are not allowed; otherwise, it is ok to distribute the source. If in
doubt, contact a maintainer who is a Googler.

Next, check the file types:
[source,bash]
----
  cd /tmp/$package-$version
  find . -type f | xargs file | grep -v 'ASCII\|UTF-8\|empty$'
----

If you see anything that looks like a native library or binary, then we can't
use the bundle.

If everything looks good, install the package with the following command:
[source, bash]
----
# Add to ui_npm. Other packages.json can be updated in the same way
cd $gerrit_repo/polygerrit-ui/app
bazelisk run @yarn//:yarn add $package
----

Update the `polygerrit-ui/app/node_modules_licenses/licenses.ts` file. You should add licenses
for the package itself and for all transitive dependencies. If you forgot to add a license, the
`Documentation:check_licenses` test will fail.

After the update, commit all changes to the repository (including `yarn.lock`).

[NOTE]
====
If a npm package has transitive dependencies (or just several files) with a not allowed
license and you can't avoid use it in release, then you can add this package.
For example some packages contain demo-code with a different license. Another example - optional
dependencies, which are not needed to build the Gerrit web app, but they are installed together
with the package anyway.

In this case you should exclude all files and/or transitive dependencies with a not allowed license.
Adding such package requires additional updates:

- Add dependencies (or files) to the license.ts with an appropriate license marked with
`allowed: false`.

- update package.json postinstall script to remove all non-allowed files (if you don't
update postinstall script, `Documentation:check_licenses` test will fail.)
====

=== Update NPM Binaries
To update a NPM binary the same actions as for a new one must be done (check licenses,
update `licenses.ts` file, etc...). The only difference is a command to install a package: instead
of `bazelisk run @yarn//:yarn add $package` you should run the `bazelisk run @yarn//:yarn upgrade ...`
command with correct arguments. You can find the list of arguments in the
link:https://classic.yarnpkg.com/en/docs/cli/upgrade/[yarn upgrade doc,role=external,window=_blank].


[[RBE]]
== Google Remote Build Support

The Bazel build can be used with Google's Remote Build Execution.


This needs the following setup steps:

```
gcloud auth application-default login
gcloud services enable remotebuildexecution.googleapis.com  --project=${PROJECT}
```

Create a worker pool. The instances should have at least 4 CPUs each
for adequate performance.

```
gcloud alpha remote-build-execution worker-pools create default \
    --project=${PROJECT} \
    --instance=default_instance \
    --worker-count=50 \
    --machine-type=e2-standard-4 \
    --disk-size=200
```

Note, that we are using Ubuntu2404 docker image from bazel project:


```
docker pull gcr.io/bazel-public/ubuntu2404@sha256:a5e969f2208ae4ab3cbe18cd85971f1f602345a650495674736a66b3ef9a55d1
```

Re-build rbe_autoconfig project, conduct a new release and switch to using it
in `MODULE.bazel` file. For more details see this
link:https://github.com/davido/rbe_autoconfig[repository,role=external,window=_blank]

Note, to authenticate to the gcr.io registry, the following command must be
used:

```
gcloud auth configure-docker
```

To see the documentation, developer must be added to this group:
https://groups.google.com/forum/#!forum/rbe-alpha-customers.

Documentation can be found at:
https://cloud.google.com/remote-build-execution/docs.

To use RBE, execute

```
bazelisk test --config=remote \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    javatests/...
```


== BuildBuddy Remote Build Support

To utilize the BuildBuddy Remote Build Execution service, please consult the
documentation available at the following link: https://www.buildbuddy.io[BuildBuddy].

To use RBE, execute

```
bazelisk test --config=remote_bb \
    --remote_instance_name=projects/${PROJECT}/instances/default_instance \
    --remote_header=x-buildbuddy-api-key=YOUR_API_KEY \
    javatests/...
```


GERRIT
------
Part of link:index.html[Gerrit Code Review]

SEARCHBOX
---------