failOnVersionConflict has never been good for us. It is equivalent to
Maven dependencyConvergence which we discourage our users to use because
it is too tempermental and _creates_ version skew issues over time.
However, we had no real alternative for determining if our deps would be
misinterpeted by Maven.
failOnVersionConflict has been a constant drain and makes it really hard
to do seemingly-trivial upgrades. As evidenced by protobuf/build.gradle
in this change, it also caused _us_ to introduce a version downgrade.
This introduces our own custom requireUpperBoundDeps implementation so
that we can get back to simple dependency upgrades _and_ increase our
confidence in a consistent dependency tree.
Enables a codepath for zero-copy protobuf deserialization. Two new InputStream extension interfaces are added:
- HasByteBuffer: allows access to the underlying buffers containing inbound bytes directly without copying
- Detachable: allows customer marshaller to keep the buffers around until the application code is done with using the protobuf messages
Applications can implement a custom marshaller that takes over the ownership of ByteBuffers and wrap them into ByteStrings with protobuf's UnsafeByteOperations support. Then a RopeByteString, which is a in-place composite of ByteStrings can be created. This enables using the zero-copy codepath (requires immutable ByteBuffer indication) of CodedInputStream for deserialization.
Since static methods are pseudo-inherited by Builder implementations but
are trivially accidentally used, we re-define static methods in each
builder to make them behave more like the caller would expect. However,
not all the methods actually work; some just throw because the caller
was certainly not getting what they would expect.
Annotating with `@DoNotCall` can expose the problems at compile time
instead of runtime. While `@Deprecated` would also be an option, it is a
bit harder to figure out the ramifications and whether we want to go
that route.
This change was suggested by a lint tool for XdsServerBuilder and it
seems appropriate so I applied it to the other similar cases I could
find.
Currently each subchannel implicitly refreshes the name resolution when its state changes to IDLE or TRANSIENT_FAILURE. That is, this feature is built into subchannel's internal implementation. Although it eliminates the burden of having LB implementations refreshing the resolver when connections to backends are broken, this is gives LB policies no chance to disable or override this refresh (e.g., in some complex load balancing hierarchy like xDS, LB policies may embed a resolver inside for resolving backends so the refreshing resolution operation should be hooked to the resolver embedded in the LB policy instead of the one in Channel).
In order to make this transition smoothly, we add a check to SubchannelImpl that checks if the LoadBalancer has explicitly called Helper.refreshNameResolution for broken subchannels created by it. If not, it logs a warning and do the refresh.
A temporary LoadBalancer.Helper API ignoreRefreshNameResolution() is added to avoid false-positive warnings for xDS that intentionally does not want a refresh. Once the migration is done, this should be deleted.
ManagedChannelImpl should not override authority for createResolvingOobChannel(target, creds), because ManagedChannelImpl does not know what target and creds are.
Enhance `ManagedChannelBuilder.overrideAuthority()` to make it impossible to use a different authority to a backend by wrapping ClientTransportFactory.newClientTransport() and setting ClientTransportOptions’ authority. To avoid confusing the LB policy, it would need to keep the original addresses to return during `Subchannel.getAddresses()`
The class `OverrideAuthorityNameResolverFactory` is deleted and its logic is moved into `ManagedChannelImpl`.
Multiple users have tried things like
`mcb.enableRetry().maxRetryAttempts(3)` and been confused when no
retries were performed. Providing a reference to the gRFC and
`defaultServiceConfig()` should greatly increase the clarity of how to
use the method.
API change (See go/grpc-rls-callcreds-to-server):
- Add `ChannelCredentials.withoutBearerTokens()`
- Add `createResolvingOobChannelBuilder(String, ChannelCredentials)`, `getChannelCredentials()` and `getUnsafeChannelCredentials()` for `LoadBalancer.Helper`
This PR does not include the implementation of `createResolvingOobChannelBuilder(String, ChannelCredentials)`.
Interceptor-based config selector will be needed for fault injection.
Add `interceptor` field to `InternalConfigSelector.Result`. Keep `callOptions` and `committedCallback` fields for the moment, because it needs a refactoring to migrate the existing xds config selector implementation to the new API.
* fix channel builders ABI backward compatibility broken in v1.33.0
* fix server builders ABI backward compatibility broken in v1.33.0
* makes ForwardingServerBuilder package-private
It deprecates ExpectedException and Assert.assertThat(T, org.hamcrest.Matcher).
Without Java 8 we don't want to migrate away from ExpectedException at
this time. We tend to prefer Truth over Hamcrest, so I swapped the one
instance of Assert.assertThat() to use Truth. With this change we get a
warning-less build with JUnit 4.13. We don't yet upgrade because we
still need to support JUnit 4.12 for some use-cases, but will be able to
upgrade to 4.13 soon when they upgrade.
It appears getAttributes() javadoc was accidentally copied when
ConfigOrError was moved in commit 0244418d2. This restores the
documentation from before the move.
verifyZeroInteractions has the same behavior as verifyNoMoreInteractions. It
was deprecated in Mockito 3.0.1 and replaced with verifyNoInteractions, which
does not change behavior depending on previous verify() calls. All instances
were replaced with verifyNoInteractions, except those in
ApplicationThreadDeframerTest which were replaced with verifyNoMoreInteractions
since there is a verify() call in `@Before`.
We found that the interceptor approach for `ConfigSelector` would be adding a layer of indirection for no gain: The API Result selectConfig(LoadBalancer.PickSubchannelArgs args) consumes headers among other inputs, because route matching might need to match the headers; and the API produces ClientInterceptor among other outputs. But the headers is not available until clientCall.start(listner, headers), whereas the interceptor need be applied to the call before clientCall.start(). So the input is not available until the output is applied. That means we will need to delay calling the downstream newCall() (which will either be RealChannel or the interceptor) until start() is called.
So we want to change to the other approach similar to what c-core is taking: Have `Result(Object config, CallOptions, Runnable committedCallback)`, where CallOption is the selector modified CallOption, and committedCallback is used to monitor the call lifecycle.
It has been our intention for years to remove nameResolverFactory. We should
make it clear to users to avoid new code depending on it and so they can tell
us why they need it so we can provide replacements.
- Use gradle configuration `api` for dependencies that are part of grpc public api signatures.
- Replace deprecated gradle configurations `compile`, `testCompile`, `runtime` and `testRuntime`.
- With minimal change in dependencies: If we need dep X and Y to compile our code, and if X transitively depends on Y, then our build would still pass even if we only include X as `compile`/`implementation` dependency for our project. Ideally we should include both X and Y explicitly as `implementation` dependency for our project, but in this PR we don't add the missing Y if it is previously missing.
Define util function to exclude guava's transitive dependencies jsr305 and animal-sniffer-annotations, and always manually add them as runtimeOnly dependency. error_prone_annotations is an exception: It is also excluded but manually added not as runtimeOnly. It must always compile with guava, otherwise users will see warning spams if guava is in the compile classpath but error_prone_annotations is not.
Eliminate the hack of InternalNotifyOnBuild mechanism for letting ProtoReflectionService get access to the Sever instance, which makes ProtoReflectionService incompatible with server interceptors. This change put the Server instance into the Context and let the ProtoReflectionService RPC obtain it in its RPC Context. Also enhanced ProtoReflectionService so that one service instance can be used across multiple servers.
useMarshalledMessages works by duplicating a ServerServiceDefinition while replacing just the marshallers. It currently does not copy over the SchemaDescriptors, which breaks at least the ProtoReflectionService.
useInputStreamMessages ensures that the InputStream supports marking by wrapping the stream in a BufferedInputStream if markSupported() returns false. This change uses a new subclass of BufferedInputStream that also implements KnownLength, when the original stream also implements KnownLength.
The server does not _have_ to wait until half close in CLIENT_STREAMING, and
commonly wouldn't in error cases. {client,server}SendsOneMessage were way
over-specifying the behavior and included unnecessary and incorrect words like
"immediately." Those methods shouldn't be the defining the behavior in that
much precision anyway; that would be the job of the individual enum values, if
anything.
Eliminated the code path of resolving Grpclb balancer addresses in grpc-core and moved it into GrpclbNameResolver, which is a subclass of DnsNameResolver. Main changes:
- Slightly changed ResourceResolver and its JNDI implementation. ResourceResolver#resolveSrv(String) returns a list of SrvRecord so that it only parse SRV records and does nothing more. It's gRPC's name resolver's logic to use information parsed from SRV records.
- Created a GrpclbNameResolver class that extends DnsNameResolver. Logic of using information from SRV records to set balancer addresses as ResolutionResult attributes is implemented in GrpclbNameResolver only.
- Refactored DnsNameResolver, mainly the resolveAll(...) method. Logics for resolving backend addresses and service config are modularized into resolveAddresses() and resolveServiceConfig() methods respectively. They are shared implementation for subclasses (i.e., GrpclbNameResolver).
The dns scheme is only the default scheme with grpc-java. Other
libraries could add more NameResolvers and thus change the default. For
compatibility reasons, the schema should therefore be specified
explicitly.
First add a new a Metadata.BinaryStreamMarshaller interface which
serializes to/from instances of InputStream, and a corresponding
Key.of() factory method.
Values set with this type of key will be kept unserialized internally,
alongside a reference to the Marshaller. A new method
InternalMetadata.serializePartial(), returns values which are either
byte[] or InputStream, and allows transport-specific handling of
lazily-serialized values.
For the regular serialize() method, stream-marshalled values will be
converted to byte[] via an InputStreams.
The API review for #6279 came up with a more meaningful name that
better explains the intent. A setter for the old name was left in
ManagedChannelBuilder to ease migration to the new name by current
users.
Adds an Executor to NameResolver.Args, which is optionally set on ManagedChannelBuilder. This allows NameResolver implementations to avoid creating their own thread pools if the application already manages its own pools.
Addresses #3703.
If a `LoadBalancer` implementation does not override `handleResolvedAddressGroups()`, or overrides `handleResolvedAddressGroups()` but calls `super.handleResolvedAddressGroups()` at the beginning or the end, it will be trapped in an infinite loop.
This makes Deadline more test-friendly. Next step is to allow
ServerBuilder to take a custom Ticker and use it for creating
incoming Deadlines. With both changes in, application logic will be
able to verify the Deadlines they set.
No more methods on the `LoadBalancer` will be called after
`LoadBalancer#shutdown()` is called. This includes
`LoadBalancer#handleSubchannelState()` too. `SubchannelStateListener`
inherited this restriction. However, this special case makes
`onSubchannelState(SHUTDOWN)` an unreliable way of being notified
about `Subchannel` SHUTDOWN, and may confuse/complicate a
wrapping `LoadBalancer` that expects the full notification (e.g., #5875).
The javadoc isn't clear whether this restriction applies. I think
it's more useful to make it no apply.
The former is deprecated and replaced by the latter in Mockito 2.
However, there is a functional difference: ArgumentMatchers will reject
`null` and check the type if the matcher specified a type (e.g.
`any(Class)` or `anyInt()`). `any()` will remain to accept anything.
Previously PickResult's Subchannel must be the actual implementation
returned from the Channel's Helper, and Channel would cast it to the
implementation class in order to use it. This will be broken if
Subchannel is wrapped in the case of hierarchical LoadBalancers.
getInternalSubchannel() is the guaranteed path for the Channel to get
the InternalSubchannel implementation. It is friendly for wrapping.
Background: #5676
The pick_first policies in core and grpclb previously would call
Subchannel.requestConnection() from data-path. They now will schedule
that call in the sync-context to avoid the warning. They will only
call it for the first pick of each picker, to prevent storming the
sync-context.
This is a revised version of #5503 (62b03fd), which was rolled back in f8d0868. The newer version passes SubchannelStateListener to Subchannel.start() instead of SubchannelCreationArgs, which allows us to remove the Subchannel argument from the listener, which works as a solution for #5676.
LoadBalancers that call the old createSubchannel() will get start() implicitly called with a listener that passes updates to the deprecated LoadBalancer.handleSubchannelState(). Those who call the new createSubchannel() will have to call start() explicitly.
GRPCLB code is still using the old API, because it's a pain to migrate the SubchannelPool to the new API. Since CachedSubchannelHelper is on the way, it's easier to switch to it when it's ready. Keeping
GRPCLB with the old API would also confirm the backward compatibility.
Contrary to #5736, we will still keep the sync-context requirement of
requestConnection(), because it prevents API fragmentation.
PickFirstLoadBalancer is the only known violator. We will fix it on
master, but we don't want to make that change on 1.21.x because the
release is soon. We simply remove the warning in this release so that
users won't be annoyed.
This supersedes #5736
We will require Subchannel.requestConnection() to be called from
sync-context (#5722), but SubchannelPicker.requestConnection() is
currently calling it with the assumption of thread-safety. Actually
SubchannelPicker.requestConnection() is called already from
sync-context by ChannelImpl, it makes more sense to move this method
to LoadBalancer where all other methods are sync-context'ed, rather than
making SubchannelPicker.requestConnection() sync-context'ed and fragmenting
the SubchannelPicker API because pickSubchannel() is thread-safe.
C++ also has the requestConnection() equivalent on their LoadBalancer
interface.
I see more cases of wrapping Helper and Subchannel during the work of
XdsLoadBalancer, we will require that all methods that involve mutable
state to be called from the Synchronization Context. We will start
logging warnings first, and make them throw in a future release.
Helper.createSubchannel() is already doing so. This change adds
warnings to the other eligible methods.
https://github.com/grpc/grpc-java/issues/5015
This follows the sort of changes we've done in the past, where the '2'
implies "version 2." We can end up reclaiming the original name if we
wish in the future.
The main reason for this change is to avoid changing to Observer since
the rest of io.grpc consistently uses Listener.
Having two Helpers (the other being LoadBalancer.Helper) is proven to
be confusing, and the one in NameResolver is fundamentally different
from the one from LoadBalancer, as the latter mutates the states while
the former doesn't. Renaming it to Args make it less awkward to
expose it to LoadBalancer, which is needed for creating NameResolvers
for per-locality routing in the XdsLoadBalancer.
As we are now endorsing the wrapping of ClientStreamTracers by
providing ForwardingClientStreamTracer, there is a need for altering
StreamInfo, especially CallOptions before it's passed onto the
delegate. A Builder class and a toBuilder() provides a robust way
to copy the rest of the fields.
This is a breaking change for anybody who creates StreamInfo, which is
unlikely in non-test code, because StreamInfo was added as late as
1.20.0.
* api: fix bugs of missing out customOptions in CreateSubchannelArgs toBuider, hashCode, equals
* trash equals/hashCode for CreateSubchannelArgs as they are problematic
NameResolverRegistry takes on all the logic previously in
NameResolverProvider. But it also allows manual registration of
NameResolvers, which is useful when the providers have complex
construction or need objects injected into them.
This also avoids a circular dependency during class loading since
previously loading any Provider searched for all Providers via
ClassLoader since ClassLoader handling was static within the parent
class.
Fixes#5562
io.grpc has fewer dependencies than io.grpc.internal. Moving it to a
separate artifact lets users use the API without bringing in the deps.
If the library has an optional dependency on grpc, that can be quite
convenient.
We now version-pin both grpc-api and grpc-core, since both contain
internal APIs.
I had to change a few tests in grpc-api to avoid FakeClock. Moving
FakeClock to grpc-api was difficult because it uses
io.grpc.internal.TimeProvider, which can't be moved since it is a
production class. Having grpc-api's tests depend on grpc-core's test
classes would be weird and cause a circular dependincy. Having
grpc-api's tests depend on grpc-core is likely possible, but weird and
fairly unnecessary at this point. So instead I rewrote the tests to
avoid FakeClock.
Fixes#1447
Chengyuan Zhang