Quality of Service

Repository Label: label QoS

Ocelot currently supports a single Quality of Service (QoS) capability. It allows you to configure, on a per-route basis, the application of a circuit breaker when making requests to downstream services. This feature leverages a well-regarded .NET library known as Polly. For more details, visit the Polly library’s official repository.

Installation

To utilize the Administration API, begin by importing the appropriate NuGet package:

Install-Package Ocelot.Provider.Polly

Next, in your Program, incorporate Polly services by invoking the AddPolly() extension on the OcelotBuilder, as shown below [1]:

using Ocelot.Provider.Polly;

builder.Services
    .AddOcelot(builder.Configuration)
    .AddPolly();

Configuration

Then add the following section to a route configuration:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3,
  "DurationOfBreak": 1000, // ms
  "TimeoutValue": 5000 // ms
}

  • To implement this rule, you must set a value of 2 or higher for ExceptionsAllowedBeforeBreaking. [2]

  • DurationOfBreak indicates that the circuit breaker will remain open for 1 second after being triggered.

  • TimeoutValue specifies that if a request exceeds 5 seconds, it will automatically time out.

Circuit Breaker strategy

The options ExceptionsAllowedBeforeBreaking and DurationOfBreak can be configured independently from TimeoutValue:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3,
  "DurationOfBreak": 1000
}

Alternatively, you can omit DurationOfBreak, which will default to the implicit 5-second setting as specified in Polly’s documentation:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3
}

This setup activates only the Circuit breaker strategy.

Timeout strategy

The TimeoutValue can be configured independently from the ExceptionsAllowedBeforeBreaking and DurationOfBreak options:

"QoSOptions": {
  "TimeoutValue": 5000 // ms
}

This setup activates only the Timeout strategy.

Notes

  1. If a QoS section is not included, QoS will not be applied, and Ocelot will enforce a default timeout of 90 seconds for all downstream requests. To request additional configurability, consider opening an issue. [3]

  2. The Polly V7 syntax is no longer supported as of version 23.2. [4]

  3. Starting with Polly V8, the documentation outlines the following constraints on values:

    • The ExceptionsAllowedBeforeBreaking value must be 2 or higher.

    • The DurationOfBreak value must exceed 500 milliseconds, defaulting to 5000 milliseconds (5 seconds) if unspecified or if the value is 500 milliseconds or less.

    • The TimeoutValue must be over 10 milliseconds.

    Refer to the Resilience strategies documentation for a comprehensive explanation of each option.

Extensibility [4]

To use your ResiliencePipeline<T> provider, you can apply the following syntax:

builder.Services
    .AddOcelot(builder.Configuration)
    .AddPolly<MyProvider>();
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider

Additionally, if you want to utilize your own DelegatingHandler, the following syntax can be applied:

builder.Services
    .AddOcelot(builder.Configuration)
    .AddPolly<MyProvider>(MyQosDelegatingHandlerDelegate);
// MyQosDelegatingHandlerDelegate is a delegate use to get a DelegatingHandler. Refer to Ocelot's PollyResiliencePipelineDelegatingHandler

Finally, to define your own set of exceptions for mapping, you can apply the following syntax:

static Error CreateError(Exception e) => new RequestTimedOutError(e);
Dictionary<Type, Func<Exception, Error>> MyErrorMapping = new()
{
    {typeof(TaskCanceledException), CreateError},
    {typeof(TimeoutRejectedException), CreateError},
    {typeof(BrokenCircuitException), CreateError},
    {typeof(BrokenCircuitException<HttpResponseMessage>), CreateError},
};
builder.Services
    .AddOcelot(builder.Configuration)
    .AddPolly<MyProvider>(MyErrorMapping);
// Note: Default error mapping is defined in the DefaultErrorMapping field of the Ocelot.Provider.Polly.OcelotBuilderExtensions class