Observability

See what the policies do through events, read their state through snapshots and handles, and build metrics, logs and health checks on top of both.

resilience4cata gives you two building blocks:

1. Events: a record for each thing a policy does other than letting a call through (a retry, a breaker transition, a rejection, a timeout, a fallback), delivered to listeners you register on the registry builder.
2. Snapshots and handles: the current state of each instance, read when you ask for it.

Tip For metrics, add the Micrometer module or the OpenTelemetry module: three lines of setup give you the metrics below, built on these two blocks. This page describes the blocks themselves, for logging, health checks, or another metrics library.

Events

Every event implements the sealed interface dev.resilience4cata.event.ResilienceEvent, with instance() (the instance id: the name, the derived pkg.Type#method(java.lang.String,int) id, or a pipeline id) and nanoTime() (a reading of the registry's clock: System.nanoTime(), or the manual clock in tests). Each policy has its own sealed interface of records:

A retry with enabled = false publishes nothing. A first failure that is not retryable publishes GaveUp with attempts = 1 and NOT_RETRYABLE.

Listening

Listeners are registered on ResilienceRegistry.Builder, and only there: they are fixed when the registry is built. Listen to one record type, a whole policy interface, or ResilienceEvent; on every instance, or on one instance id:

ResilienceRegistry registry = ResilienceRegistry.builder()
        // one record type, on every instance
        .listen(CircuitBreakerEvent.StateTransition.class,
                t -> log.add(t.instance() + ": " + t.from() + " -> " + t.to()))
        // every retry event (Retrying and GaveUp), on one instance only
        .listen("payments", RetryEvent.class, e -> log.add(e.toString()))
        // everything: keep a default branch, new policies add event types
        .listen(ResilienceEvent.class, e -> {
            switch (e) {
                case RetryEvent.GaveUp g -> log.add("gave up after " + g.attempts() + ": " + g.reason());
                default -> { }
            }
        })
        .build();

• Listeners run synchronously on the thread that caused the event: the caller's thread, or a library virtual thread for async timeouts and delayed async retries. Keep them fast; hand slow work (a network call) to your own executor.
• An exception thrown by a listener does not reach the call or the other listeners: it goes to the thread's uncaught exception handler.
• Order: events published from different threads can arrive out of order; each breaker transition arrives exactly once.
• EventListener<E> is a functional interface (void onEvent(E event)), so a lambda or a method reference works.

Important New versions add event types. The hierarchy is sealed and grows in minor versions (a new policy brings its own event interface, as HedgeEvent did in 0.4.0). Keep a default branch in a switch over ResilienceEvent or over one policy's interface, or listen to the record types you need (listen(RetryEvent.GaveUp.class, …)), which new types never affect. Code compiled against an older version that receives an unknown type throws MatchException in the listener; like any listener exception it goes to the uncaught exception handler and never breaks the call.

What events cost

Each policy resolves its listeners once, when the instance is created, into one field per event type. An event type nobody listens to costs one null check, and only on non-success paths. A successful call publishes nothing and checks nothing. When someone listens, the event record is allocated and the clock is read only at that point.

There are no per-call counters (no "successful calls" or latency totals) by design: a counter updated on every call would be a shared write on the hot path. If you need call counts or latency, measure them in your own code around the call, or with your HTTP client's or framework's metrics.

Snapshots

registry.snapshot() returns a ResilienceSnapshot with one list per policy, each sorted by instance id. It is built when you call it, never on the call path, and may allocate and walk the breaker windows: call it at scrape or health-check time, not per request.

ResilienceSnapshot snapshot = registry.snapshot(); // built when asked, never on the call path
for (CircuitBreakerSnapshot b : snapshot.circuitBreakers()) {
    lines.add(b.instance() + " " + b.state() + " failureRate=" + b.failureRate()
            + " buffered=" + b.bufferedCalls() + " failed=" + b.failedCalls() + " slow=" + b.slowCalls());
}
for (BulkheadSnapshot b : snapshot.bulkheads()) {
    lines.add(b.instance() + " in use " + b.concurrentCalls() + "/" + b.maxConcurrent()
            + ", waiting " + b.waitingCalls() + ", free " + b.availableSlots());
}
for (RateLimiterSnapshot r : snapshot.rateLimiters()) {
    lines.add(r.instance() + " permits " + r.availablePermits() + "/" + r.permits());
}

Handles

Handles read one instance live, and the breaker's can also change its state:

An instance exists only once its interface has been wrapped (Resilient.of, Resilient.create) or its Pipeline built; before that, or for an id that does not exist, the lookup throws IllegalArgumentException R4C-REG-009 with the closest id. Retry and timeout have no handle; their configuration is in the snapshot. There is no list method for bulkheads, rate limiters and adaptive limits: enumerate them through registry.snapshot().

Metrics

The Micrometer module (resilience4cata-micrometer) turns the events into counters and the handles into gauges. Its page has the setup, a real Prometheus scrape, Grafana queries and cardinality advice. The OpenTelemetry module (resilience4cata-opentelemetry) reports the same metrics, with the same names and tags, through the OpenTelemetry Metrics API, so dashboards work with either.

Metric names

Names are lowercase and dotted, as Micrometer recommends (Prometheus shows them as r4c_retry_retries_total and so on); every meter has an instance tag (the instance id: instance ids are fixed in your code, so the tag is bounded) and at most one or two more tags with a fixed set of values. Tag values are lowercase.

Another metrics library

A bridge to another library does what the Micrometer module does, with three pieces of the core API: a listener for the counters, the handles read at collection time for the gauges, and registry.onInstanceCreated(…) to learn about instances created after you bound the registry (instances are created when a service is wrapped or a pipeline is built, never during a call):

ResilienceRegistry registry = ResilienceRegistry.builder()
        // counters: from events, on the thread that caused them
        .listen(RetryEvent.Retrying.class, e -> retries.merge(e.instance(), 1L, Long::sum))
        .build();
// gauges: one per instance, read at collection time; new instances are announced once each
registry.onInstanceCreated(() -> {
    for (CircuitBreakerHandle breaker : registry.circuitBreakers()) {
        gauges.putIfAbsent(breaker.id(), () -> breaker.state() == CircuitBreakerState.OPEN ? 1 : 0);
    }
});

onInstanceCreated runs its action once per new instance, on the thread that creates it, after the instance is visible in snapshot() and the handle lookups; it returns a Registration whose close() stops it. Bulkheads and rate limiters have no list method: find them in registry.snapshot(). Cache your counters by instance and tag values, so an event does not register a meter each time.

Logging transitions

A listener that logs every breaker state change through System.Logger (which goes to your logging backend through its JDK bridge, for example SLF4J's slf4j-jdk-platform-logging):

import dev.resilience4cata.CircuitBreakerState;
import dev.resilience4cata.event.CircuitBreakerEvent;
import dev.resilience4cata.event.EventListener;
import java.lang.System.Logger.Level;

/** Logs every circuit breaker state change: a warning when a breaker opens, info otherwise. */
public final class BreakerTransitionLogger implements EventListener<CircuitBreakerEvent.StateTransition> {

    private final System.Logger logger;

    public BreakerTransitionLogger(System.Logger logger) {
        this.logger = logger;
    }

    @Override
    public void onEvent(CircuitBreakerEvent.StateTransition t) {
        boolean opened = t.to() == CircuitBreakerState.OPEN || t.to() == CircuitBreakerState.FORCED_OPEN;
        logger.log(opened ? Level.WARNING : Level.INFO, "Circuit breaker {0}: {1} -> {2}",
                t.instance(), t.from(), t.to());
    }
}

ResilienceRegistry registry = ResilienceRegistry.builder()
        .listen(CircuitBreakerEvent.StateTransition.class, new BreakerTransitionLogger(logger))
        .build();

Log transitions and give-ups, not every Retrying or rejection: those can arrive at the rate of your traffic during an incident. Count them instead.

Health checks

A check built on the snapshot, for a readiness or diagnostics endpoint:

import dev.resilience4cata.CircuitBreakerSnapshot;
import dev.resilience4cata.CircuitBreakerState;
import dev.resilience4cata.ResilienceRegistry;
import java.util.List;

/** A readiness-style check: degraded while any circuit breaker rejects calls. */
public record ResilienceHealth(boolean healthy, List<String> openBreakers) {

    public static ResilienceHealth of(ResilienceRegistry registry) {
        List<String> open = registry.snapshot().circuitBreakers().stream()
                .filter(b -> b.state() == CircuitBreakerState.OPEN || b.state() == CircuitBreakerState.FORCED_OPEN)
                .map(CircuitBreakerSnapshot::instance)
                .toList();
        return new ResilienceHealth(open.isEmpty(), open);
    }
}

ResilienceHealth health = ResilienceHealth.of(registry);
if (!health.healthy()) {
    // report "degraded" with health.openBreakers() on your readiness endpoint
}

Warning Don't wire an open breaker into a liveness probe: restarting the service does not fix the dependency the breaker protects, and every instance would restart at once. Report it as degraded, or on a readiness endpoint only if routing traffic away actually helps.