Circuit Breaker pattern implementation with Resilience4J

In a previous post we talked about the implementation of the Circuit Breaker pattern. As a reminder, the Circuit Breaker is a pattern that prevents cascading the failure of a single micro-service in the whole architecture, ensuring the system is resilient.

The pattern can be implemented by the code, with a library like Hystrix, or by the underlying infrastructure, e.g. using Istio. We had a look at the two implementations and focused on an example based on Hystrix.

At the time of this writing, another library is the new standard for fault tolerance in micro-services architectures, Hystrix being End-of-Life: Resilience4j.

Migrating From Hystrix to Resilience4J

Hystrix vs. Resilience4j in Brief

Hystrix is an Open Source library offered by Netflix that aims to improve the resiliency of a distributed system that makes HTTP requests to communicate among its distributed components.
It does so by implementing the Circuit Breaker pattern.

Resilience4J is a standalone library inspired by Hystrix but build on the principles of Functional Programming. The most prominent difference between the two is the fact that while Hystrix embraces an Object-Oriented design where calls to external systems have to be wrapped in a HystrixCommand offering multiple functionalities, Resilience4J relies on function composition to let you stack the specific decorators you need.

Those decorators include of course the Circuit Breaker, but also a Rate Limiter, Retry and Bulkhead. Such decorators can be executed synchronously or asynchronously, taking full advantage of lambdas, introduced in Java 8.

Other advantages of Resilience4J include more fine tuned configuration options (e.g. the number successful executions needed to close the Circuit Breaker pattern) and a lighter dependencies footprint.

Short Introduction to Functional Programming in Java

The idea behind function composition is that:

if function f is defined as Function<X, Y> - a function that takes a type X as input and returns a type Y
and if function g is defined as Function<Y, Z>
then a new function h can be defined as Function<X, Z>, that compose functions f and g

In mathematical parlance, this is noted g o f.

Java 8 brought some aspects of Functional Programming (FP) in its API. The above function composition could be translated as such in Java:

public class F implements Function<Integer, Integer> {

    @Override
    public Integer apply(Integer x) {
        return x + 1;
    }
}

public class G implements Function<Integer, Integer> {

    @Override
    public Integer apply(Integer y) {
        return 2 * y;
    }
}

public class H implements  Function<Integer, Integer> {

    @Override
    public Integer apply(Integer x) {
        var f = new F();
        var g = new G();
        Integer y = f.apply(x);
        return g.apply(y);
    }
}

This is pretty cumbersome to write, because Java was initially designed with Object-Oriented Programming (OOP) in mind.

Everything needs to belong to a class, even when that doesn’t make much sense. Hence, to bridge this gap between the OOP and the FP, and make FP code easier to write, Java 8 brings the notion of Functional Interface: a Functional Interface is an interface with a single abstract method, and is optionally annotated with @FunctionalInterface.

Any Functional Interface can be written in a simplified way, using the lambda notation. For example, Function<T, V> is a Functional Interface, as it has a single abstract method - apply(). Thus, the above code can be rewritten using lambdas:

Function<Integer, Integer> h = x -> {
    Function<Integer, Integer> f = y -> y + 1;
    Function<Integer, Integer> g = y -> y + 1;
    return g.apply(f.apply(x));
};

Another foundation of FP are higher-order functions. That just means that functions are types like any other, and can be passed as parameters in functions, and as well returned as results.

For example, Function interface defines the following method:

default <V> Function<T, V> andThen(Function<? super R, ? extends V> after) {
    Objects.requireNonNull(after);
    return (T t) -> after.apply(apply(t));
}

If it looks a lot like the function composition described above, that’s because it is. Using this method, we can rewrite the h function simply as:

var h = f.compose(g);

Resilience4J is entirely based on Functional Programming, and uses the notions exposed here a lot. Is important to keep that in mind migrating from Hystrix as that requires a change compared to the usual Java mindset.

Get Started With Resilience4J

An HTTP call could be thought as a function: it accepts an HTTP request as an input, and returns an HTTP response.

Likewise, a Circuit Breaker can be thought as a function, with as input the same HTTP request, and as return value either the HTTP response if the call succeeds or the default HTTP response if it fails.

Hence, using a Circuit Breaker is like composing the first “call” function with the second “circuit-breaker” function.

Here’s a sample to illustrate how it’s used:

public class Command {
    public Long run() {
        // Does the actual job of making the HTTP request
    }
}

var cb = CircuitBreaker.ofDefaults("circuit-breaker");
var result = cb.executeSupplier(new Command()::run);

Because every feature in Resilience4J is modeled as a function, combining those features requires just to apply the function composition principle described above.

This is the equivalent of the Decorator pattern in Object-Oriented Programming: the target is “wrapped” into a decorator object.

Here, we apply this design to compose three function calls. The first one calls the HTTP endpoint, the second one is the Circuit Breaker, and the third one retries if the call fails.

var cb = CircuitBreaker.ofDefaults("circuit-breaker");
var retry = Retry.ofDefaults("retry");
var cbSupplier = CircuitBreaker.decorateSupplier(cb, new Command()::run);
var retrySupplier = Retry.decorateSupplier(retry, cbSupplier);
var result = retrySupplier.get();

A Custom Cache Decorator

In the initial post about the Circuit Breaker pattern, we used Hystrix to cache prices: if the target HTTP endpoint was not available, the price of a product was returned from an in-memory cache.

While there’s a cache feature available in Resilience4J, it just returns the result if it’s available in the cache. Our requirement is different: it should return from the cache only if the decorated function fails.

However, it’s quite straightforward to design our own cache implementation function. The word “function” is important there, because as per Resilience4J design principle, state - the cache - should be external and passed to the function to keep it pure.

To keep the implementation simple, the cache will keep a single value that might get replaced when the decorated function returns successfully:

public class Cache<T> {

    private T value;

    static <T> Supplier<T> decorateSupplier(Cache<T> cache, Supplier<T> supplier) {
        return Try.ofSupplier(supplier)
                .fold(
                        throwable -> () -> cache.value,
                        value -> {
                            cache.value = value;
                            return () -> value;
                        });
    }
}

The Try class comes from the Vavr library, a Functional-Programming API for the Java language, and the only dependency of Resilience4J. It requires two lambdas:

The first accepts a Throwable and returns a function returning the result
The second accepts the value, and returns a result-returning function as well

Note that both are lazy: they don’t return the result directly, but instead a Supplier of the result.

With this custom cache, it’s now possible to decorate Circuit Breaker calls to return the cached value if the circuit is open:

var cb = CircuitBreaker.ofDefaults("circuit-breaker");
var cache = new Cache<Long>();
var cbDecorated = CircuitBreaker.decorateSupplier(cb, new Command()::run);
var cacheDecorated = Cache.decorateSupplier(cache, cbDecorated);
cacheDecorated.get();