android-interop has been failing to build since 46485c8 because it
didn't have cmake installed and defined LDFLAGS/CXXFLAGS with pkg-config
before make_dependencies.sh had been run.
Android-interop didn't verify the codegen is up-to-date. Building the
codegen was just a relic from when android was its own separate gradle
build. Avoiding codegen means we don't have to compile absl/protobuf and
have a C++ toolchain.
To try to aid failure when building android-interop-testing
```
The Daemon will expire after the build after running out of JVM heap space.
The project memory settings are likely not configured or are configured to an insufficient value.
The daemon will restart for the next build, which may increase subsequent build times.
These settings can be adjusted by setting 'org.gradle.jvmargs' in 'gradle.properties'.
The currently configured max heap space is '512 MiB' and the configured max metaspace is '384 MiB'.
...
Exception in thread "Daemon client event forwarder" java.lang.OutOfMemoryError: Java heap space
...
> Task :grpc-android-interop-testing:mergeDexDebug FAILED
ERROR:D8: java.lang.OutOfMemoryError: Java heap space
com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives:
```
The Kokoro aarch64 build runs on x86 with an emulator, and has always
been flaky due to the slow execution speed. At present it is continually
failing due to deadline exceededs. GitHub Actions is running on aarch64
hardware, so is much faster (4 minutes vs 30 minutes, without including
the speedup from GitHub Action's caching).
This avoids the dependency on animalsniffer-annotations. grpc-api, and
particularly grpc-context, are used many low-level places and it is
beneficial for them to be very low dependency. This brings grpc-context
back to zero-dependency.
unix.sh is shared by multiple OSes and environments. Clear JAVA_HOME,
since we never want to use that as PATH is more reliable, better
supported, and more typical.
`envoyproxy/envoy`: Sync protos to the latest imported version
ab911ac2ff
(commit 2024-07-06, cl/651956889).
Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
Java 8 isn't installed, and was needed by the old Android SDK. With the
current SDK, it can work on Java 11 but it needs some dependencies
installed.
Python 2.7 isn't available any more, but instead of porting to Python 3,
it was just replaced with a curl command.
CentOS 7 became end-of-life on July 1st and is no longer working. We now
dynamically link against libstdc++, as RHEL 8 doesn't support static
linking: https://access.redhat.com/articles/rhel8-abi-compatibility
We now use objdump in check-artifact for all linux architectures. This
avoids using a mix of objdump and ldd. ldd shows transitive
dependencies, which is less convenient.
There are no longer any devices (virtual or otherwise) that support API
level 21, 22, or 23. Google Play services is still supporting API level
21 (although there is a pattern of notifying of dropped levels in July,
and dropping them in August).
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73.
Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh):
1. The list of tests in the each test suite
2. Per-test-suite flag customization
3. `run_test` methods
4. `build_docker_images_if_needed` methods
5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow.
Now all PSM Interop jobs share the same buildscripts by all test suites:
1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg)
2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`)
`psm-interop-test-{language}.sh`:
1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables.
2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo.
3. Sources `psm-interop-build-{language}.sh`
4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file).
`psm-interop-build-{language}.sh`:
1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`.
2. Invokes any repo-specific logic.
3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method.
References:
- b/288578634
- See the full list of the new features at grpc/psm-interop#73.
- Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
I'm trying to upgrade to a newer Windows Kokoro image, but the new one
has an old vswhere installed that breaks Gradle. Our old image doesn't
have vswhere at all. If vswhere isn't found, this rename prints some
errors, but the bat script continues executing. So this change is
compatible with both the older and newer image.
Splits the :grpc-android-interop-testing:assembleDebug and
:grpc-android-interop-testing:assembleDebugAndroidTest build
targets with hopes of avoiding OOMs.
They are a lot faster. Instead of 1-3 minutes of test execution, I now
see 2-22 seconds. There still may be 3 minutes of overhead for the
gcloud command to complete, but the reduction is noticable in the total
execution time. And it seems the tests are actually being run, as there
is some flakiness. The flakiness appears to be at a lower rate.
The script was slightly reorganized to make it easier to copy commands
to run locally.
Note that this uses Pixel2 and Pixel3. Also swap 26-27 from Nexus6P to
Pixel2. We tend to prefer the latest (virtual) device for each API
level.
The current models and their supported API levels are available via:
```
gcloud firebase test android models list --filter=form=virtual
```
Pixel2.arm supports 31-32, but is beta, so I didn't swap to it. It also
supports the preview 33.
Note that this changes the JDK used to compile releases to Java 11. That
should only impact the appearance of the Javadoc.
This adds the Android SDK to the build container, removing the
dependency on the Android SDK being available on the CI host. This
allows running on newer Kokoro images. 'Android' and 'Android interop'
CIs still depend on the Android SDK being available on the host, but
since they aren't used as part of the release process, they can more
easily migrate off Kokoro as part of future work.
This also causes Android components to now be built with -Werror, as we
use -PfailOnWarnings=true in unix.sh but were missing it from the
Android build invocations.
Gradle will auto-download the necessary version of build-tools. We don't
want to download it ourselves because the version we specify might not
even be used. Looking at logs, we were previously downloading a version
that was unused.
We now fork javac to avoid OOM. The build fails 2/3 times before the
forking, and 0/3 after.
Apparently our Kokoro image has this done already, and my laptop as
well. But the newer Kokoro image and other computers like my desktop
don't have it already.
We configured TestGrid to file bug separately for each
failed sub-target, if we still fail the main target,
TestGrid will fail duplicate bugs.
The same change in core:
https://github.com/grpc/grpc/pull/33222.
- Update to xDS Test Client and xDS test server Docker images to `eclipse-temurin:11-jre`.
- Perform software update so that we install patches for latest vulnerabilities.
MV Shiva