| :linkattrs: | |
| = Gerrit Code Review - Building with Bazel | |
| [[summary]] | |
| == TL;DR | |
| If you have the prerequisites, running | |
| ``` | |
| $ bazel build gerrit | |
| ``` | |
| should generate a .war file under `bazel-bin/gerrit.war`. | |
| [[installation]] | |
| == Prerequisites | |
| To build Gerrit from source, you need: | |
| * A Linux or macOS system (Windows is not supported) | |
| * A JDK for Java 17 or Java 21 | |
| * Python 3 | |
| * link:https://github.com/nodesource/distributions/blob/master/README.md[Node.js (including npm),role=external,window=_blank] | |
| * Yarn (`npm install -g yarn`) | |
| * link:https://bazel.build/install[Bazel,role=external,window=_blank] -launched with | |
| link:https://github.com/bazelbuild/bazelisk[Bazelisk,role=external,window=_blank] | |
| * Maven | |
| * zip, unzip | |
| * curl | |
| * gcc | |
| [[bazel]] | |
| === Bazel | |
| link:https://github.com/bazelbuild/bazelisk[Bazelisk,role=external,window=_blank] is a version | |
| manager for link:https://bazel.build/[Bazel,role=external,window=_blank], similar to how `nvm` | |
| manages `npm` versions. It takes care of downloading and installing Bazel itself, so you don't have | |
| to worry about using the correct version of Bazel. One particular advantage to | |
| using Bazelisk is that you can jump between different versions of Gerrit and not | |
| worry about which version of Bazel you need. | |
| Bazelisk can be installed in different ways: | |
| link:https://bazel.build/install/bazelisk[Bazelisk Installation,role=external,window=_blank]. | |
| To execute the correct version of Bazel using Bazelisk you simply replace | |
| the `bazel` command with `bazelisk`. | |
| [[java]] | |
| === Java | |
| Ensure that the link:install.html#Requirements[required Java version] | |
| is installed and that `JAVA_HOME` is set to it. | |
| To check the installed version of Java, open a terminal window and run: | |
| `java -version` | |
| [[java-21]] | |
| ==== Java 21 support | |
| To build Gerrit with Java 21 language level, run: | |
| ``` | |
| $ bazelisk build --config=java21 :release | |
| ``` | |
| [[java-17]] | |
| ==== Java 17 support | |
| Java 17 is supported. To build Gerrit with Java 17, run: | |
| ``` | |
| $ bazelisk build :release | |
| ``` | |
| To run the tests with Java 17, run: | |
| ``` | |
| $ bazelisk test //... | |
| ``` | |
| === Node.js and npm packages | |
| See link:https://gerrit.googlesource.com/gerrit/+/master/polygerrit-ui/README.md#installing-node_js-and-npm-packages[Installing Node.js and npm packages,role=external,window=_blank]. | |
| [[build]] | |
| == Building on the Command Line | |
| === Gerrit Development WAR File | |
| To build the Gerrit web application: | |
| ---- | |
| bazelisk build gerrit | |
| ---- | |
| The output executable WAR will be placed in: | |
| ---- | |
| bazel-bin/gerrit.war | |
| ---- | |
| [[release]] | |
| === Gerrit Release WAR File | |
| To build the Gerrit web application that includes the Gerrit UI, | |
| core plugins and documentation: | |
| ---- | |
| bazelisk build release | |
| ---- | |
| The output executable WAR will be placed in: | |
| ---- | |
| bazel-bin/release.war | |
| ---- | |
| === Headless Mode | |
| To build Gerrit in headless mode, i.e. without the Gerrit UI: | |
| ---- | |
| bazelisk build headless | |
| ---- | |
| The output executable WAR will be placed in: | |
| ---- | |
| bazel-bin/headless.war | |
| ---- | |
| === Extension and Plugin API JAR Files | |
| To build the extension, plugin and acceptance-framework JAR files: | |
| ---- | |
| bazelisk build api | |
| ---- | |
| The output archive that contains Java binaries, Java sources and | |
| Java docs will be placed in: | |
| ---- | |
| bazel-bin/api.zip | |
| ---- | |
| Install {extension,plugin,acceptance-framework}-api to the local | |
| maven repository: | |
| ---- | |
| tools/maven/api.sh install | |
| ---- | |
| Install gerrit.war to the local maven repository: | |
| ---- | |
| tools/maven/api.sh war_install | |
| ---- | |
| === Plugins | |
| ---- | |
| bazelisk build plugins:core | |
| ---- | |
| The output JAR files for individual plugins will be placed in: | |
| ---- | |
| bazel-bin/plugins/<name>/<name>.jar | |
| ---- | |
| The JAR files will also be packaged in: | |
| ---- | |
| bazel-bin/plugins/core.zip | |
| ---- | |
| To build a specific plugin: | |
| ---- | |
| bazelisk build plugins/<name> | |
| ---- | |
| The output JAR file will be be placed in: | |
| ---- | |
| bazel-bin/plugins/<name>/<name>.jar | |
| ---- | |
| Note that when building an individual plugin, the `core.zip` package | |
| is not regenerated. | |
| [[IDEs]] | |
| == Using an IDE. | |
| === IntelliJ | |
| The Gerrit build works with Bazel's link:https://ij.bazel.build[IntelliJ plugin,role=external,window=_blank]. | |
| Please follow the instructions on <<dev-intellij#,IntelliJ Setup>>. | |
| === Eclipse | |
| ==== Generating the Eclipse Project | |
| Create the Eclipse project: | |
| ---- | |
| tools/eclipse/project.py | |
| ---- | |
| and then follow the link:dev-eclipse.html#setup[setup instructions]. | |
| ==== Refreshing the Classpath | |
| If an updated classpath is needed, the Eclipse project can be | |
| refreshed and missing dependency JARs can be downloaded by running | |
| `project.py` again. For IntelliJ, you need to click the `Sync Project | |
| with BUILD Files` button of link:https://ij.bazel.build[Bazel plugin,role=external,window=_blank]. | |
| [[documentation]] | |
| == Documentation | |
| To build only the documentation for testing or static hosting: | |
| ---- | |
| bazelisk build Documentation:searchfree | |
| ---- | |
| The html files will be bundled into `searchfree.zip` in this location: | |
| ---- | |
| bazel-bin/Documentation/searchfree.zip | |
| ---- | |
| To use local fonts with the searchfree target: | |
| ---- | |
| bazel build Documentation:searchfree_safe | |
| ---- | |
| The html files will be bundled into `searchfree.zip` or `searchfree_safe.zip` in this location: | |
| ---- | |
| bazel-bin/Documentation/searchfree.zip | |
| bazel-bin/Documentation/searchfree_safe.zip | |
| ---- | |
| To generate HTML files skipping the zip archiving: | |
| ---- | |
| bazelisk build Documentation | |
| ---- | |
| And open `bazel-bin/Documentation/index.html`. | |
| To build the Gerrit executable WAR with the documentation included: | |
| ---- | |
| bazelisk build withdocs | |
| ---- | |
| The WAR file will be placed in: | |
| ---- | |
| bazel-bin/withdocs.war | |
| ---- | |
| Alternatively, one can generate the documentation as flat files: | |
| ---- | |
| bazelisk build Documentation:Documentation | |
| ---- | |
| The html, css, js files are placed in: | |
| ---- | |
| `bazel-bin/Documentation/` | |
| ---- | |
| [[tests]] | |
| == Running Unit Tests | |
| Bazel BUILD files define test targets for Gerrit. You can run all declared | |
| test targets with: | |
| ---- | |
| bazelisk test --build_tests_only //... | |
| ---- | |
| [[testgroups]] | |
| === Running Test Groups | |
| To run one or more specific labeled groups of tests: | |
| ---- | |
| bazelisk test --test_tag_filters=api,git //... | |
| ---- | |
| The following label values are currently supported for the group name: | |
| * annotation | |
| * api | |
| * edit | |
| * git | |
| * git-protocol-v2 | |
| * git-upload-archive | |
| * notedb | |
| * pgm | |
| * rest | |
| * server | |
| * ssh | |
| We can also select tests within a specific BUILD target group. For example | |
| `javatests/com/google/gerrit/acceptance/rest/account/BUILD` declares a | |
| rest_account test target group: | |
| ---- | |
| bazelisk test //javatests/com/google/gerrit/acceptance/rest/account:rest_account | |
| ---- | |
| [[debugtests]] | |
| === Debugging Tests | |
| To debug specific tests you will need to select the test target containing | |
| that test then use `--test_filter` to select the specific test you want. | |
| This `--test_filter` is a regex and can be used to select multiple tests | |
| out of the target: | |
| ---- | |
| bazelisk test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget | |
| ---- | |
| For example `javatests/com/google/gerrit/acceptance/api/change/BUILD` | |
| defines a test target group for every `*IT.java` file in the directory. | |
| We can execute the single `getAmbiguous()` test found in ChangeIT.java using | |
| this `--test_filter` and target: | |
| ---- | |
| bazelisk test --test_output=streamed \ | |
| --test_filter=com.google.gerrit.acceptance.api.change.ChangeIT.getAmbiguous \ | |
| //javatests/com/google/gerrit/acceptance/api/change:ChangeIT | |
| ---- | |
| [[additionaltestfiltering]] | |
| === Additional Test Filtering | |
| To run only tests that do not use SSH: | |
| ---- | |
| bazelisk test --test_env=GERRIT_USE_SSH=NO //... | |
| ---- | |
| To exclude tests that have been marked as flaky: | |
| ---- | |
| bazelisk test --test_tag_filters=-flaky //... | |
| ---- | |
| To exclude tests that require very recent git client version: | |
| ---- | |
| bazelisk test --test_tag_filters=-git-protocol-v2 //... | |
| ---- | |
| To run the tests against a specific index backend (LUCENE, FAKE): | |
| ---- | |
| bazelisk test --test_env=GERRIT_INDEX_TYPE=LUCENE //... | |
| ---- | |
| Bazel itself supports a multitude of ways to | |
| link:https://bazel.build/run/build#specifying-build-targets[specify targets,role=external,window=_blank] | |
| for fine-grained test selection that can be combined with many of the examples | |
| above. | |
| [[testcaching]] | |
| === Test Caching | |
| By default Bazel caches test results and will not reexecute tests unless they | |
| or their dependencies have been modified. To ignore cached test results and | |
| force the tests to rerun: | |
| ---- | |
| bazelisk test --cache_test_results=NO //... | |
| ---- | |
| [[plugintests]] | |
| === Running Plugin Tests | |
| Running tests for Gerrit plugins follows the process above. From within the | |
| Gerrit project root with the desired plugins checked out into `plugins/` we | |
| execute Bazel with the appropriate target: | |
| ---- | |
| bazelisk test //plugins/replication/... | |
| ---- | |
| [[known-issues]] | |
| === Known Issues | |
| [[byte-buddy-not-initialized-or-unavailable]] | |
| ==== The Byte Buddy agent is not initialized or unavailable | |
| If running tests that make use of mocks fail with the exception below, set the | |
| `sandbox_tmpfs_path` flag for running tests in `.bazelrc` as described in this | |
| link:https://github.com/mockito/mockito/issues/1879#issuecomment-922459131[ | |
| issue], e.g. add this line: `test --sandbox_tmpfs_path=/tmp` | |
| .Exception: | |
| ---- | |
| ... | |
| Caused by: org.mockito.exceptions.base.MockitoInitializationException: | |
| Could not initialize inline Byte Buddy mock maker. | |
| It appears as if your JDK does not supply a working agent attachment mechanism. | |
| ... | |
| Caused by: java.lang.IllegalStateException: The Byte Buddy agent is not initialized or unavailable | |
| at net.bytebuddy.agent.ByteBuddyAgent.getInstrumentation(ByteBuddyAgent.java:230) | |
| at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:617) | |
| at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:568) | |
| at net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:545) | |
| at org.mockito.internal.creation.bytebuddy.InlineDelegateByteBuddyMockMaker.<clinit>(InlineDelegateByteBuddyMockMaker.java:115) | |
| ... 47 more | |
| ---- | |
| [[debugging-tests]] | |
| == Debugging Unit Tests | |
| In some cases it may be necessary to debug a test while running it in bazel. For example, when we | |
| observe a different test result in Eclipse and bazel. Using the `--java_debug` option will start the | |
| JVM in debug mode and await for a remote debugger to attach. | |
| Example: | |
| [source,bash] | |
| ---- | |
| bazelisk test --java_debug --test_tag_filters=delete-project //... | |
| ... | |
| Listening for transport dt_socket at address: 5005 | |
| ... | |
| ---- | |
| Now attach with a debugger to the port `5005`. For example use "Remote Java Application" launch | |
| configuration in Eclipse and specify the port `5005`. | |
| [[logging]] | |
| === Controlling logging level | |
| Per default, logging level is set to `INFO` level for all tests. The `DEBUG` | |
| log level can be enabled for the tests. | |
| In IDE, set `-Dgerrit.logLevel=debug` as a VM argument. With `bazel`, pass | |
| `GERRIT_LOG_LEVEL=debug` environment variable: | |
| ---- | |
| bazelisk test --test_filter=com.google.gerrit.server.notedb.ChangeNotesTest \ | |
| --test_env=GERRIT_LOG_LEVEL=debug \ | |
| javatests/com/google/gerrit/server:server_tests | |
| ---- | |
| The log results can be found in: | |
| `bazel-testlogs/javatests/com/google/gerrit/server/server_tests/test.log`. | |
| == Dependencies | |
| Dependency JARs are normally downloaded as needed, but you can | |
| download everything upfront. This is useful to enable | |
| subsequent builds to run without network access: | |
| ---- | |
| bazelisk fetch //... | |
| ---- | |
| When downloading from behind a proxy (which is common in some corporate | |
| environments), it might be necessary to explicitly specify the proxy that | |
| is then used by `curl`: | |
| ---- | |
| export http_proxy=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port> | |
| ---- | |
| Redirection to local mirrors of Maven Central and the Gerrit storage | |
| bucket is supported by defining specific properties in | |
| `local.properties`, a file that is not tracked by Git: | |
| ---- | |
| echo download.GERRIT = http://nexus.my-company.com/ >>local.properties | |
| echo download.MAVEN_CENTRAL = http://nexus.my-company.com/ >>local.properties | |
| ---- | |
| The `local.properties` file may be placed in the root of the gerrit repository | |
| being built, or in `~/.gerritcodereview/`. The file in the root of the gerrit | |
| repository has precedence. | |
| == Building against unpublished Maven JARs | |
| To build against unpublished Maven JARs, like PrologCafe, the custom JARs must | |
| be installed in the local Maven repository (`mvn clean install`) and | |
| `maven_jar()` must be updated to point to the `MAVEN_LOCAL` Maven repository for | |
| that artifact: | |
| [source,python] | |
| ---- | |
| maven_jar( | |
| name = 'prolog-runtime', | |
| artifact = 'com.googlecode.prolog-cafe:prolog-runtime:42', | |
| repository = MAVEN_LOCAL, | |
| ) | |
| ---- | |
| == Building against artifacts from custom Maven repositories | |
| To build against custom Maven repositories, two modes of operations are | |
| supported: with rewrite in local.properties and without. | |
| Without rewrite the URL of custom Maven repository can be directly passed | |
| to the maven_jar() function: | |
| [source,python] | |
| ---- | |
| GERRIT_FORGE = 'http://gerritforge.com/snapshot' | |
| maven_jar( | |
| name = 'gitblit', | |
| artifact = 'com.gitblit:gitblit:1.4.0', | |
| sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', | |
| repository = GERRIT_FORGE, | |
| ) | |
| ---- | |
| When the custom URL has to be rewritten, then the same logic as with Gerrit | |
| known Maven repository is used: Repo name must be defined that matches an entry | |
| in local.properties file: | |
| ---- | |
| download.GERRIT_FORGE = http://my.company.mirror/gerrit-forge | |
| ---- | |
| And corresponding WORKSPACE excerpt: | |
| [source,python] | |
| ---- | |
| GERRIT_FORGE = 'GERRIT_FORGE:' | |
| maven_jar( | |
| name = 'gitblit', | |
| artifact = 'com.gitblit:gitblit:1.4.0', | |
| sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', | |
| repository = GERRIT_FORGE, | |
| ) | |
| ---- | |
| == Building against SNAPSHOT Maven JARs | |
| To build against SNAPSHOT Maven JARs, the complete SNAPSHOT version must be used: | |
| [source,python] | |
| ---- | |
| maven_jar( | |
| name = "pac4j-core", | |
| artifact = "org.pac4j:pac4j-core:3.5.0-SNAPSHOT-20190112.120241-16", | |
| sha1 = "da2b1cb68a8f87bfd40813179abd368de9f3a746", | |
| ) | |
| ---- | |
| [[bazel-local-caches]] | |
| To accelerate builds, several caches are activated per default: | |
| * ~/.gerritcodereview/bazel-cache/downloaded-artifacts | |
| * ~/.gerritcodereview/bazel-cache/repository | |
| * ~/.gerritcodereview/bazel-cache/cas | |
| The `downloaded-artifacts` cache can be relocated by setting the | |
| `GERRIT_CACHE_HOME` environment variable. The other two can be adjusted with | |
| `bazelisk build` options `--repository_cache` and `--disk_cache` respectively. | |
| Currently none of these caches have a maximum size limit. See | |
| link:https://github.com/bazelbuild/bazel/issues/5139[this bazel issue,role=external,window=_blank] for | |
| details. Users should watch the cache sizes and clean them manually if | |
| necessary. | |
| [[npm-binary]] | |
| == NPM Binaries | |
| Parts of the Gerrit web app build require running NPM-based JavaScript programs | |
| as "binaries". We don't attempt to resolve and download NPM dependencies at | |
| build time, but instead use pre-built bundles of the NPM binary along with all | |
| its dependencies. Some packages on | |
| link:https://docs.npmjs.com/misc/registry[registry.npmjs.org,role=external,window=_blank] come with their | |
| dependencies bundled, but this is the exception rather than the rule. More | |
| commonly, to add a new binary to this list, you will need to bundle the binary | |
| yourself. | |
| [NOTE] | |
| We can only use binaries that meet certain licensing requirements, and that do | |
| not include any native code. | |
| Start by checking that the license and file types of the bundle are acceptable: | |
| [source,bash] | |
| ---- | |
| gerrit_repo=/path/to/gerrit | |
| package=some-npm-package | |
| version=1.2.3 | |
| # Note - yarn must be installed before running the following commands | |
| yarn global add license-checker && \ | |
| rm -rf /tmp/$package-$version && mkdir -p /tmp/$package-$version && \ | |
| cd /tmp/$package-$version && \ | |
| yarn add $package@$version && \ | |
| license-checker | grep licenses: | sort -u | |
| ---- | |
| This will output a list of the different licenses used by the package and all | |
| its transitive dependencies. We can only legally distribute a bundle via our | |
| storage bucket if the licenses allow us to do so. As long as all of the listed | |
| license are allowed by | |
| link:https://opensource.google.com/docs/thirdparty/licenses/[Google's | |
| standards,role=external,window=_blank]. Any `by_exception_only`, commercial, prohibited, or unlisted | |
| licenses are not allowed; otherwise, it is ok to distribute the source. If in | |
| doubt, contact a maintainer who is a Googler. | |
| Next, check the file types: | |
| [source,bash] | |
| ---- | |
| cd /tmp/$package-$version | |
| find . -type f | xargs file | grep -v 'ASCII\|UTF-8\|empty$' | |
| ---- | |
| If you see anything that looks like a native library or binary, then we can't | |
| use the bundle. | |
| If everything looks good, install the package with the following command: | |
| [source, bash] | |
| ---- | |
| # Add to ui_npm. Other packages.json can be updated in the same way | |
| cd $gerrit_repo/polygerrit-ui/app | |
| bazelisk run @nodejs//:yarn add $package | |
| ---- | |
| Update the `polygerrit-ui/app/node_modules_licenses/licenses.ts` file. You should add licenses | |
| for the package itself and for all transitive dependencies. If you forgot to add a license, the | |
| `Documentation:check_licenses` test will fail. | |
| After the update, commit all changes to the repository (including `yarn.lock`). | |
| [NOTE] | |
| ==== | |
| If a npm package has transitive dependencies (or just several files) with a not allowed | |
| license and you can't avoid use it in release, then you can add this package. | |
| For example some packages contain demo-code with a different license. Another example - optional | |
| dependencies, which are not needed to build the Gerrit web app, but they are installed together | |
| with the package anyway. | |
| In this case you should exclude all files and/or transitive dependencies with a not allowed license. | |
| Adding such package requires additional updates: | |
| - Add dependencies (or files) to the license.ts with an appropriate license marked with | |
| `allowed: false`. | |
| - update package.json postinstall script to remove all non-allowed files (if you don't | |
| update postinstall script, `Documentation:check_licenses` test will fail.) | |
| ==== | |
| === Update NPM Binaries | |
| To update a NPM binary the same actions as for a new one must be done (check licenses, | |
| update `licenses.ts` file, etc...). The only difference is a command to install a package: instead | |
| of `bazelisk run @nodejs//:yarn add $package` you should run the `bazelisk run @nodejs//:yarn upgrade ...` | |
| command with correct arguments. You can find the list of arguments in the | |
| link:https://classic.yarnpkg.com/en/docs/cli/upgrade/[yarn upgrade doc,role=external,window=_blank]. | |
| [[RBE]] | |
| == Google Remote Build Support | |
| The Bazel build can be used with Google's Remote Build Execution. | |
| This needs the following setup steps: | |
| ``` | |
| gcloud auth application-default login | |
| gcloud services enable remotebuildexecution.googleapis.com --project=${PROJECT} | |
| ``` | |
| Create a worker pool. The instances should have at least 4 CPUs each | |
| for adequate performance. | |
| ``` | |
| gcloud alpha remote-build-execution worker-pools create default \ | |
| --project=${PROJECT} \ | |
| --instance=default_instance \ | |
| --worker-count=50 \ | |
| --machine-type=e2-standard-4 \ | |
| --disk-size=200 | |
| ``` | |
| Note, that we are using Ubuntu2204 docker image from bazel project: | |
| ``` | |
| docker pull gcr.io/bazel-public/ubuntu2204-java17@sha256:ffe37746a34537d8e73cef5a20ccd3a4e3ec7af3e7410cba87387ba97c0e520f | |
| ``` | |
| Re-build rbe_autoconfig project, conduct a new release and switch to using it | |
| in `WORKSPACE` file. For more details see this | |
| link:https://github.com/davido/rbe_autoconfig[repository,role=external,window=_blank] | |
| Note, to authenticate to the gcr.io registry, the following command must be | |
| used: | |
| ``` | |
| gcloud auth configure-docker | |
| ``` | |
| To see the documentation, developer must be added to this group: | |
| https://groups.google.com/forum/#!forum/rbe-alpha-customers. | |
| Documentation can be found at: | |
| https://cloud.google.com/remote-build-execution/docs. | |
| To use RBE, execute | |
| ``` | |
| bazelisk test --config=remote \ | |
| --remote_instance_name=projects/${PROJECT}/instances/default_instance \ | |
| javatests/... | |
| ``` | |
| == BuildBuddy Remote Build Support | |
| To utilize the BuildBuddy Remote Build Execution service, please consult the | |
| documentation available at the following link: https://www.buildbuddy.io[BuildBuddy]. | |
| To use RBE, execute | |
| ``` | |
| bazelisk test --config=remote_bb \ | |
| --remote_instance_name=projects/${PROJECT}/instances/default_instance \ | |
| --remote_header=x-buildbuddy-api-key=YOUR_API_KEY \ | |
| javatests/... | |
| ``` | |
| GERRIT | |
| ------ | |
| Part of link:index.html[Gerrit Code Review] | |
| SEARCHBOX | |
| --------- |