tree ac5cc82f2c0fdb5bd67a837ed3010e6b62fc9986
parent 7a701f41769a55fe8288f4f8e55d2a025bb6642b
author David Ostrovsky <david@ostrovsky.org> 1590061439 +0200
committer Luca Milanesio <luca.milanesio@gmail.com> 1590144364 +0000

Revisit plugin test target detection

The way it is currently implemented to detect test targets is error
prone. In case of find-owners plugin there are two test targets:

$ grep -2 junit_tests BUILD | grep -o 'name = "[^"]*"' | cut -d '"' -f 2
findowners_junit_tests
findowners_IT_tests

But the code assumes that there are only one single target:

if [ "$TEST_TARGET" != "" ]
then
  bazelisk test --test_env DOCKER_HOST=$DOCKER_HOST plugins/{name}:$TEST_TARGET
fi

And this is why the second target findowners_IT_tests gets orphaned and
is not prefixed with "plugins/{name}" prefix and thus not found.

Moreover, this approach to grep for targets can also lead to false
negative results, because the test rules might be located in nested
plugin packages but not in the root package, like it is the case for
checks plugin. For such plugins no test are executed at all.

To rectify, avoid grep and just use Bazel's native wildcard "/..."
target feature, that should work in all three possible cases:

1. no test rule exist
2. one single test rule exsits in plugin's root package
3. many test rules exist in root and/or nested packages

This approach has problem with exit code of bazel in case that there are
no tests even thoufg test run was requested.

As documented in the user guide: [1], it is expected that Bazel
unconditionally returns exit code 4 in this situation. Check for that
exit code explicitly and warn CI users to contact plugin maintainers if
there were no tests found. Let's face it: the code is inherently broken
if there are no tests at all. Still we would like to avoid the
boilerplate to be needed and wrote this feature request: [2].

[1] https://docs.bazel.build/versions/master/guide.html#what-exit-code-will-i-get
[2] https://github.com/bazelbuild/bazel/issues/11465

Bug: Issue 12512
Change-Id: Id641cfd2300939d3eac5adb4cdc4a14b12d5f7c2