Quality of Service

Label: QoS

Ocelot supports one QoS capability at the current time. You can set on a per Route basis if you want to use a circuit breaker when making requests to a downstream service. This uses an awesome .NET library called Polly, check them out in official repository.

The first thing you need to do if you want to use the Administration API is bring in the relevant NuGet package:

Install-Package Ocelot.Provider.Polly

Then in your ConfigureServices method to add Polly services we must call the AddPolly() extension of the OcelotBuilder being returned by AddOcelot() [1] like below:

services.AddOcelot()
    .AddPolly();

Then add the following section to a Route configuration:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3,
  "DurationOfBreak": 1000,
  "TimeoutValue": 5000
}

  • You must set a number equal or greater than 2 against ExceptionsAllowedBeforeBreaking for this rule to be implemented. [2]

  • DurationOfBreak means the circuit breaker will stay open for 1 second after it is tripped.

  • TimeoutValue means if a request takes more than 5 seconds, it will automatically be timed out.

You can set the TimeoutValue in isolation of the ExceptionsAllowedBeforeBreaking and DurationOfBreak options:

"QoSOptions": {
  "TimeoutValue": 5000
}

There is no point setting the other two in isolation as they affect each other!

Defaults

If you do not add a QoS section, QoS will not be used, however Ocelot will default to a 90 seconds timeout on all downstream requests. If someone needs this to be configurable, open an issue. [2]

Polly v7 vs v8

Important changes in version 23.2: [3]

  • With Polly version 8+, the ExceptionsAllowedBeforeBreaking value must be equal to or greater than 2!

  • The AddPolly method has been migrated from v7 policy wrappers to v8 resilience pipelines. Consequently, it now exhibits different behavior based on v8 pipelines.

If you prefer not to modify your settings, you can continue using Polly v7 as follows:

services.AddOcelot()
    .AddPollyV7();

Note: Support for Polly v7 will be removed in a future version. We recommend avoiding this method (which is tagged as Obsolete) unless absolutely necessary.

Extensibility [3]

If you want to use your ResiliencePipeline<T> provider, you can use the following syntax:

 services.AddOcelot()
     .AddPolly<MyProvider>();
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider

If, in addition, you want to use your own DelegatingHandler, you can use the following syntax:

 services.AddOcelot()
     .AddPolly<MyProvider>(MyQosDelegatingHandlerDelegate);
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider
// MyQosDelegatingHandlerDelegate is a delegate use to get a DelegatingHandler

And finally, if you want to define your own set of exceptions to map, you can use the following syntax:

services.AddOcelot()
    .AddPolly<MyProvider>(MyErrorMapping);
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider

// MyErrorMapping is a Dictionary<Type, Func<Exception, Error>>, eg:
private static readonly Dictionary<Type, Func<Exception, Error>> MyErrorMapping = new()
{
    {typeof(TaskCanceledException), CreateError},
    {typeof(TimeoutRejectedException), CreateError},
    {typeof(BrokenCircuitException), CreateError},
    {typeof(BrokenCircuitException<HttpResponseMessage>), CreateError},
};
private static Error CreateError(Exception e) => new RequestTimedOutError(e);