Package io.grpc

Class LoadBalancer

java.lang.Object
io.grpc.LoadBalancer
Direct Known Subclasses:
ForwardingLoadBalancer, MultiChildLoadBalancer, OutlierDetectionLoadBalancer
@ExperimentalApi("https://github.com/grpc/grpc-java/issues/1771") @NotThreadSafe public abstract class LoadBalancer extends Object
A pluggable component that receives resolved addresses from NameResolver and provides the channel a usable subchannel when asked.

Overview

A LoadBalancer typically implements three interfaces:

  1. LoadBalancer is the main interface. All methods on it are invoked sequentially in the same synchronization context (see next section) as returned by LoadBalancer.Helper.getSynchronizationContext(). It receives the results from the NameResolver, updates of subchannels' connectivity states, and the channel's request for the LoadBalancer to shutdown.
  2. SubchannelPicker does the actual load-balancing work. It selects a Subchannel for each new RPC.
  3. Factory creates a new LoadBalancer instance.

Helper is implemented by gRPC library and provided to Factory. It provides functionalities that a LoadBalancer implementation would typically need.

The Synchronization Context

All methods on the LoadBalancer interface are called from a Synchronization Context, meaning they are serialized, thus the balancer implementation doesn't need to worry about synchronization among them. LoadBalancer.Helper.getSynchronizationContext() allows implementations to schedule tasks to be run in the same Synchronization Context, with or without a delay, thus those tasks don't need to worry about synchronizing with the balancer methods.

However, the actual running thread may be the network thread, thus the following rules must be followed to prevent blocking or even dead-locking in a network:

  1. Never block in the Synchronization Context. The callback methods must return quickly. Examples or work that must be avoided: CPU-intensive calculation, waiting on synchronization primitives, blocking I/O, blocking RPCs, etc.
  2. Avoid calling into other components with lock held. The Synchronization Context may be under a lock, e.g., the transport lock of OkHttp. If your LoadBalancer holds a lock in a callback method (e.g., handleResolvedAddresses()) while calling into another method that also involves locks, be cautious of deadlock. Generally you wouldn't need any locking in the LoadBalancer if you follow the canonical implementation pattern below.

The canonical implementation pattern

A LoadBalancer keeps states like the latest addresses from NameResolver, the Subchannel(s) and their latest connectivity states. These states are mutated within the Synchronization Context,

A typical SubchannelPicker holds a snapshot of these states. It may have its own states, e.g., a picker from a round-robin load-balancer may keep a pointer to the next Subchannel, which are typically mutated by multiple threads. The picker should only mutate its own state, and should not mutate or re-acquire the states of the LoadBalancer. This way the picker only needs to synchronize its own states, which is typically trivial to implement.

When the LoadBalancer states changes, e.g., Subchannels has become or stopped being READY, and we want subsequent RPCs to use the latest list of READY Subchannels, LoadBalancer would create a new picker, which holds a snapshot of the latest Subchannel list. Refer to the javadoc of onSubchannelState() how to do this properly.

No synchronization should be necessary between LoadBalancer and its pickers if you follow the pattern above. It may be possible to implement in a different way, but that would usually result in more complicated threading.

Since:
1.2.0

  • Field Details

  • Constructor Details

    • LoadBalancer

      public LoadBalancer()

  • Method Details

    • handleResolvedAddresses

      public void handleResolvedAddresses(LoadBalancer.ResolvedAddresses resolvedAddresses)
      Handles newly resolved server groups and metadata attributes from name resolution system. servers contained in EquivalentAddressGroup should be considered equivalent but may be flattened into a single list if needed.

      Implementations should not modify the given servers.

      Parameters:
      resolvedAddresses - the resolved server addresses, attributes, and config.
      Since:
      1.21.0

    • acceptResolvedAddresses

      public Status acceptResolvedAddresses(LoadBalancer.ResolvedAddresses resolvedAddresses)
      Accepts newly resolved addresses from the name resolution system. The EquivalentAddressGroup addresses should be considered equivalent but may be flattened into a single list if needed.

      Implementations can choose to reject the given addresses by returning false.

      Implementations should not modify the given addresses.

      Parameters:
      resolvedAddresses - the resolved server addresses, attributes, and config.
      Returns:
      true if the resolved addresses were accepted. false if rejected.
      Since:
      1.49.0

    • handleNameResolutionError

      public abstract void handleNameResolutionError(Status error)
      Handles an error from the name resolution system.
      Parameters:
      error - a non-OK status
      Since:
      1.2.0

    • handleSubchannelState

      @Deprecated public void handleSubchannelState(LoadBalancer.Subchannel subchannel, ConnectivityStateInfo stateInfo)
      Deprecated.
      This method will be removed. Stop overriding it. Instead, pass LoadBalancer.SubchannelStateListener to LoadBalancer.Subchannel.start(io.grpc.LoadBalancer.SubchannelStateListener) to receive Subchannel state updates
      Handles a state change on a Subchannel.

      The initial state of a Subchannel is IDLE. You won't get a notification for the initial IDLE state.

      If the new state is not SHUTDOWN, this method should create a new picker and call Helper.updateBalancingState(). Failing to do so may result in unnecessary delays of RPCs. Please refer to PickResult.withSubchannel()'s javadoc for more information.

      SHUTDOWN can only happen in two cases. One is that LoadBalancer called LoadBalancer.Subchannel.shutdown() earlier, thus it should have already discarded this Subchannel. The other is that Channel is doing a forced shutdown or has already terminated, thus there won't be further requests to LoadBalancer. Therefore, the LoadBalancer usually don't need to react to a SHUTDOWN state.

      Parameters:
      subchannel - the involved Subchannel
      stateInfo - the new state
      Since:
      1.2.0

    • shutdown

      public abstract void shutdown()
      The channel asks the load-balancer to shutdown. No more methods on this class will be called after this method. The implementation should shutdown all Subchannels and OOB channels, and do any other cleanup as necessary.
      Since:
      1.2.0

    • canHandleEmptyAddressListFromNameResolution

      public boolean canHandleEmptyAddressListFromNameResolution()
      Whether this LoadBalancer can handle empty address group list to be passed to handleResolvedAddresses(ResolvedAddresses). The default implementation returns false, meaning that if the NameResolver returns an empty list, the Channel will turn that into an error and call handleNameResolutionError(io.grpc.Status). LoadBalancers that want to accept empty lists should override this method and return true.

      This method should always return a constant value. It's not specified when this will be called.

    • requestConnection

      public void requestConnection()
      The channel asks the LoadBalancer to establish connections now (if applicable) so that the upcoming RPC may then just pick a ready connection without waiting for connections. This is triggered by ManagedChannel.getState(true).

      If LoadBalancer doesn't override it, this is no-op. If it infeasible to create connections given the current state, e.g. no Subchannel has been created yet, LoadBalancer can ignore this request.

      Since:
      1.22.0