| = Gerrit Code Review - Building with Bazel |
| |
| [[installation]] |
| == Prerequisites |
| |
| To build Gerrit from source, you need: |
| |
| * A Linux or macOS system (Windows is not supported at this time) |
| * A JDK for Java 8|9|10|11|... |
| * Python 2 or 3 |
| * link:https://github.com/nodesource/distributions/blob/master/README.md[Node.js (including npm)] |
| * Bower (`sudo npm install -g bower`) |
| * link:https://docs.bazel.build/versions/master/install.html[Bazel] |
| * Maven |
| * zip, unzip |
| * gcc |
| |
| [[java]] |
| === Java |
| |
| ==== MacOS |
| |
| On MacOS, ensure that "Java for MacOS X 10.5 Update 4" (or higher) is installed |
| and that `JAVA_HOME` is set to the |
| link:install.html#Requirements[required Java version]. |
| |
| Java installations can typically be found in |
| "/System/Library/Frameworks/JavaVM.framework/Versions". |
| |
| To check the installed version of Java, open a terminal window and run: |
| |
| `java -version` |
| |
| [[java-12]] |
| ==== Java 12 support |
| |
| Java 12 (and newer) is supported through vanilla java toolchain |
| link:https://docs.bazel.build/versions/master/toolchains.html[Bazel option]. |
| To build Gerrit with Java 12 and newer, specify vanilla java toolchain and |
| provide the path to JDK home: |
| |
| ``` |
| $ bazel build \ |
| --define=ABSOLUTE_JAVABASE=<path-to-java-12> \ |
| --javabase=@bazel_tools//tools/jdk:absolute_javabase \ |
| --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \ |
| --host_java_toolchain=//tools:toolchain_vanilla \ |
| --java_toolchain=//tools:toolchain_vanilla \ |
| :release |
| ``` |
| |
| To run the tests, `--javabase` option must be passed as well, because |
| bazel test runs the test using the target javabase: |
| |
| ``` |
| $ bazel test \ |
| --define=ABSOLUTE_JAVABASE=<path-to-java-12> \ |
| --javabase=@bazel_tools//tools/jdk:absolute_javabase \ |
| --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \ |
| --host_java_toolchain=//tools:toolchain_vanilla \ |
| --java_toolchain=//tools:toolchain_vanilla \ |
| //... |
| ``` |
| |
| To avoid passing all those options on every Bazel build invocation, |
| they could be added to ~/.bazelrc resource file: |
| |
| ``` |
| $ cat << EOF > ~/.bazelrc |
| > build --define=ABSOLUTE_JAVABASE=<path-to-java-12> |
| > build --javabase=@bazel_tools//tools/jdk:absolute_javabase |
| > build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase |
| > build --host_java_toolchain=//tools:toolchain_vanilla |
| > build --java_toolchain=//tools:toolchain_vanilla |
| > EOF |
| ``` |
| |
| Now, invoking Bazel with just `bazel build :release` would include |
| all those options. |
| |
| [[java-11]] |
| ==== Java 11 support |
| |
| Java 11 is supported through alternative java toolchain |
| link:https://docs.bazel.build/versions/master/toolchains.html[Bazel option]. |
| To build Gerrit with Java 11, specify JDK 11 java toolchain: |
| |
| ``` |
| $ bazel build \ |
| --host_javabase=@bazel_tools//tools/jdk:remote_jdk11 \ |
| --javabase=@bazel_tools//tools/jdk:remote_jdk11 \ |
| --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 \ |
| --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 \ |
| :release |
| ``` |
| |
| === Node.js and npm packages |
| See link:https://gerrit.googlesource.com/gerrit/+/master/polygerrit-ui/README.md#installing-node_js-and-npm-packages[Installing Node.js and npm packages]. |
| |
| [[build]] |
| == Building on the Command Line |
| |
| === Gerrit Development WAR File |
| |
| To build the Gerrit web application: |
| |
| ---- |
| bazel build gerrit |
| ---- |
| |
| The output executable WAR will be placed in: |
| |
| ---- |
| bazel-bin/gerrit.war |
| ---- |
| |
| [[release]] |
| === Gerrit Release WAR File |
| |
| To build the Gerrit web application that includes the PolyGerrit UI, |
| core plugins and documentation: |
| |
| ---- |
| bazel build release |
| ---- |
| |
| The output executable WAR will be placed in: |
| |
| ---- |
| bazel-bin/release.war |
| ---- |
| |
| === Headless Mode |
| |
| To build Gerrit in headless mode, i.e. without the PolyGerrit UI: |
| Web UI: |
| |
| ---- |
| bazel build headless |
| ---- |
| |
| The output executable WAR will be placed in: |
| |
| ---- |
| bazel-bin/headless.war |
| ---- |
| |
| === Extension and Plugin API JAR Files |
| |
| To build the extension, plugin and acceptance-framework JAR files: |
| |
| ---- |
| bazel build api |
| ---- |
| |
| The output archive that contains Java binaries, Java sources and |
| Java docs will be placed in: |
| |
| ---- |
| bazel-bin/api.zip |
| ---- |
| |
| Install {extension,plugin,acceptance-framework}-api to the local |
| maven repository: |
| |
| ---- |
| tools/maven/api.sh install |
| ---- |
| |
| Install gerrit.war to the local maven repository: |
| |
| ---- |
| tools/maven/api.sh war_install |
| ---- |
| |
| === Plugins |
| |
| ---- |
| bazel build plugins:core |
| ---- |
| |
| The output JAR files for individual plugins will be placed in: |
| |
| ---- |
| bazel-bin/plugins/<name>/<name>.jar |
| ---- |
| |
| The JAR files will also be packaged in: |
| |
| ---- |
| bazel-bin/plugins/core.zip |
| ---- |
| |
| To build a specific plugin: |
| |
| ---- |
| bazel build plugins/<name> |
| ---- |
| |
| The output JAR file will be be placed in: |
| |
| ---- |
| bazel-bin/plugins/<name>/<name>.jar |
| ---- |
| |
| Note that when building an individual plugin, the `core.zip` package |
| is not regenerated. |
| |
| [[IDEs]] |
| == Using an IDE. |
| |
| === IntelliJ |
| |
| The Gerrit build works with Bazel's link:https://ij.bazel.build[IntelliJ plugin]. |
| Please follow the instructions on <<dev-intellij#,IntelliJ Setup>>. |
| |
| === Eclipse |
| |
| ==== Generating the Eclipse Project |
| |
| Create the Eclipse project: |
| |
| ---- |
| tools/eclipse/project.py |
| ---- |
| |
| and then follow the link:dev-eclipse.html#setup[setup instructions]. |
| |
| ==== Refreshing the Classpath |
| |
| If an updated classpath is needed, the Eclipse project can be |
| refreshed and missing dependency JARs can be downloaded by running |
| `project.py` again. For IntelliJ, you need to click the `Sync Project |
| with BUILD Files` button of link:https://ij.bazel.build[Bazel plugin]. |
| |
| [[documentation]] |
| === Documentation |
| |
| To build only the documentation for testing or static hosting: |
| |
| ---- |
| bazel build Documentation:searchfree |
| ---- |
| |
| The html files will be bundled into `searchfree.zip` in this location: |
| |
| ---- |
| bazel-bin/Documentation/searchfree.zip |
| ---- |
| |
| To build the executable WAR with the documentation included: |
| |
| ---- |
| bazel build withdocs |
| ---- |
| |
| The WAR file will be placed in: |
| |
| ---- |
| bazel-bin/withdocs.war |
| ---- |
| |
| [[tests]] |
| == Running Unit Tests |
| |
| ---- |
| bazel test --build_tests_only //... |
| ---- |
| |
| Debugging tests: |
| |
| ---- |
| bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget |
| ---- |
| |
| Debug test example: |
| |
| ---- |
| bazel test --test_output=streamed --test_filter=com.google.gerrit.acceptance.api.change.ChangeIT.getAmbiguous //javatests/com/google/gerrit/acceptance/api/change:api_change |
| ---- |
| |
| To run a specific test group, e.g. the rest-account test group: |
| |
| ---- |
| bazel test //javatests/com/google/gerrit/acceptance/rest/account:rest_account |
| ---- |
| |
| To run only tests that do not use SSH: |
| |
| ---- |
| bazel test --test_env=GERRIT_USE_SSH=NO //... |
| ---- |
| |
| To exclude tests that have been marked as flaky: |
| |
| ---- |
| bazel test --test_tag_filters=-flaky //... |
| ---- |
| |
| To exclude tests that require a Docker host: |
| |
| ---- |
| bazel test --test_tag_filters=-docker //... |
| ---- |
| |
| To ignore cached test results: |
| |
| ---- |
| bazel test --cache_test_results=NO //... |
| ---- |
| |
| To run one or more specific groups of tests: |
| |
| ---- |
| bazel test --test_tag_filters=api,git //... |
| ---- |
| |
| The following values are currently supported for the group name: |
| |
| * annotation |
| * api |
| * docker |
| * edit |
| * elastic |
| * git |
| * notedb |
| * pgm |
| * rest |
| * server |
| * ssh |
| |
| [[elasticsearch]] |
| === Elasticsearch |
| |
| Successfully running the Elasticsearch tests requires Docker, and |
| may require setting the local |
| link:https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html[virtual memory]. |
| |
| If Docker is not available, the Elasticsearch tests will be skipped. |
| Note that Bazel currently does not show |
| link:https://github.com/bazelbuild/bazel/issues/3476[the skipped tests]. |
| |
| == Dependencies |
| |
| Dependency JARs are normally downloaded as needed, but you can |
| download everything upfront. This is useful to enable |
| subsequent builds to run without network access: |
| |
| ---- |
| bazel fetch //... |
| ---- |
| |
| When downloading from behind a proxy (which is common in some corporate |
| environments), it might be necessary to explicitly specify the proxy that |
| is then used by `curl`: |
| |
| ---- |
| export http_proxy=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port> |
| ---- |
| |
| Redirection to local mirrors of Maven Central and the Gerrit storage |
| bucket is supported by defining specific properties in |
| `local.properties`, a file that is not tracked by Git: |
| |
| ---- |
| echo download.GERRIT = http://nexus.my-company.com/ >>local.properties |
| echo download.MAVEN_CENTRAL = http://nexus.my-company.com/ >>local.properties |
| ---- |
| |
| The `local.properties` file may be placed in the root of the gerrit repository |
| being built, or in `~/.gerritcodereview/`. The file in the root of the gerrit |
| repository has precedence. |
| |
| == Building against unpublished Maven JARs |
| |
| To build against unpublished Maven JARs, like PrologCafe, the custom JARs must |
| be installed in the local Maven repository (`mvn clean install`) and |
| `maven_jar()` must be updated to point to the `MAVEN_LOCAL` Maven repository for |
| that artifact: |
| |
| [source,python] |
| ---- |
| maven_jar( |
| name = 'prolog-runtime', |
| artifact = 'com.googlecode.prolog-cafe:prolog-runtime:42', |
| repository = MAVEN_LOCAL, |
| ) |
| ---- |
| |
| == Building against artifacts from custom Maven repositories |
| |
| To build against custom Maven repositories, two modes of operations are |
| supported: with rewrite in local.properties and without. |
| |
| Without rewrite the URL of custom Maven repository can be directly passed |
| to the maven_jar() function: |
| |
| [source,python] |
| ---- |
| GERRIT_FORGE = 'http://gerritforge.com/snapshot' |
| |
| maven_jar( |
| name = 'gitblit', |
| artifact = 'com.gitblit:gitblit:1.4.0', |
| sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', |
| repository = GERRIT_FORGE, |
| ) |
| ---- |
| |
| When the custom URL has to be rewritten, then the same logic as with Gerrit |
| known Maven repository is used: Repo name must be defined that matches an entry |
| in local.properties file: |
| |
| ---- |
| download.GERRIT_FORGE = http://my.company.mirror/gerrit-forge |
| ---- |
| |
| And corresponding WORKSPACE excerpt: |
| |
| [source,python] |
| ---- |
| GERRIT_FORGE = 'GERRIT_FORGE:' |
| |
| maven_jar( |
| name = 'gitblit', |
| artifact = 'com.gitblit:gitblit:1.4.0', |
| sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', |
| repository = GERRIT_FORGE, |
| ) |
| ---- |
| |
| == Building against SNAPSHOT Maven JARs |
| |
| To build against SNAPSHOT Maven JARs, the complete SNAPSHOT version must be used: |
| |
| [source,python] |
| ---- |
| maven_jar( |
| name = "pac4j-core", |
| artifact = "org.pac4j:pac4j-core:3.5.0-SNAPSHOT-20190112.120241-16", |
| sha1 = "da2b1cb68a8f87bfd40813179abd368de9f3a746", |
| ) |
| ---- |
| |
| [[bazel-local-caches]] |
| |
| To accelerate builds, several caches are activated per default: |
| |
| * ~/.gerritcodereview/bazel-cache/downloaded-artifacts |
| * ~/.gerritcodereview/bazel-cache/repository |
| * ~/.gerritcodereview/bazel-cache/cas |
| |
| Currently none of these caches have a maximum size limit. See |
| link:https://github.com/bazelbuild/bazel/issues/5139[this bazel issue] for |
| details. Users should watch the cache sizes and clean them manually if |
| necessary. |
| |
| [[npm-binary]] |
| == NPM Binaries |
| |
| Parts of the PolyGerrit build require running NPM-based JavaScript programs as |
| "binaries". We don't attempt to resolve and download NPM dependencies at build |
| time, but instead use pre-built bundles of the NPM binary along with all its |
| dependencies. Some packages on |
| link:https://docs.npmjs.com/misc/registry[registry.npmjs.org] come with their |
| dependencies bundled, but this is the exception rather than the rule. More |
| commonly, to add a new binary to this list, you will need to bundle the binary |
| yourself. |
| |
| [NOTE] |
| We can only use binaries that meet certain licensing requirements, and that do |
| not include any native code. |
| |
| Start by checking that the license and file types of the bundle are acceptable: |
| [source,bash] |
| ---- |
| gerrit_repo=/path/to/gerrit |
| package=some-npm-package |
| version=1.2.3 |
| |
| npm install -g license-checker && \ |
| rm -rf /tmp/$package-$version && mkdir -p /tmp/$package-$version && \ |
| cd /tmp/$package-$version && \ |
| npm install $package@$version && \ |
| license-checker | grep licenses: | sort -u |
| ---- |
| |
| This will output a list of the different licenses used by the package and all |
| its transitive dependencies. We can only legally distribute a bundle via our |
| storage bucket if the licenses allow us to do so. As long as all of the listed |
| license are allowed by |
| link:https://opensource.google.com/docs/thirdparty/licenses/[Google's |
| standards]. Any `by_exception_only`, commercial, prohibited, or unlisted |
| licenses are not allowed; otherwise, it is ok to distribute the source. If in |
| doubt, contact a maintainer who is a Googler. |
| |
| Next, check the file types: |
| [source,bash] |
| ---- |
| cd /tmp/$package-$version |
| find . -type f | xargs file | grep -v 'ASCII\|UTF-8\|empty$' |
| ---- |
| |
| If you see anything that looks like a native library or binary, then we can't |
| use the bundle. |
| |
| If everything looks good, create the bundle, and note the SHA-1: |
| [source,bash] |
| ---- |
| $gerrit_repo/tools/js/npm_pack.py $package $version && \ |
| sha1sum $package-$version.tgz |
| ---- |
| |
| This creates a file named `$package-$version.tgz` in your working directory. |
| |
| Any project maintainer can upload this file to the |
| link:https://console.cloud.google.com/storage/browser/gerrit-maven/npm-packages[storage |
| bucket]. |
| |
| Finally, add the new binary to the build process: |
| ---- |
| # WORKSPACE |
| npm_binary( |
| name = "some-npm-package", |
| repository = GERRIT, |
| ) |
| |
| # lib/js/npm.bzl |
| NPM_VERSIONS = { |
| ... |
| "some-npm-package": "1.2.3", |
| } |
| |
| NPM_SHA1S = { |
| ... |
| "some-npm-package": "<sha1>", |
| } |
| ---- |
| |
| To use the binary from the Bazel build, you need to use the `run_npm_binary.py` |
| wrapper script. For an example, see the use of `crisper` in `tools/bzl/js.bzl`. |
| |
| |
| |
| [[RBE]] |
| == Google Remote Build Support |
| |
| The Bazel build can be used with Google's Remote Build Execution. |
| |
| |
| This needs the following setup steps: |
| |
| ``` |
| gcloud auth application-default login |
| gcloud services enable remotebuildexecution.googleapis.com --project=${PROJECT} |
| ``` |
| |
| Create a worker pool. The instances should have at least 4 CPUs each |
| for adequate performance. |
| |
| ``` |
| gcloud alpha remote-build-execution worker-pools create default \ |
| --project=${PROJECT} \ |
| --instance=default_instance \ |
| --worker-count=50 \ |
| --machine-type=n1-highcpu-4 \ |
| --disk-size=200 |
| ``` |
| |
| To use RBE, execute |
| |
| ``` |
| bazel test --config=remote \ |
| --remote_instance_name=projects/${PROJECT}/instances/default_instance \ |
| javatests/... |
| ``` |
| |
| |
| |
| |
| GERRIT |
| ------ |
| Part of link:index.html[Gerrit Code Review] |
| |
| SEARCHBOX |
| --------- |