Web Service Rate Limiter and Circuit Breaker

The rate limiter and circuit breaker determine how many calls can be made to a web service..

Rate Limiter

The rate limiter allows a maximum number of calls to a web service in a specified time interval. The rate limit is checked prior to every invocation of the web service. If the rate limit is reached, a ServiceUnavailableException is thrown.

The timeout behavior for the rate limiter is similar to the circuit breaker timeout behavior.

Circuit Breaker

The circuit breaker suspends calls to a web service if a certain number of calls fail within a specified time interval.

You can enable a circuit breaker for any service. The circuit breaker checks whether the limit of failed calls is reached prior to every invocation of the web service. If the limit is reached, B2C Commerce throws a ServiceUnavailableException rather than execute the service. Otherwise, B2C Commerce calls the web service as usual.

If the web service fails due to something detected as a misconfiguration or remote problem, then the circuit breaker is notified. These problems include:
  • Issues at initial connection:
    • Unknown host
    • Connection timeout
    • Connection refused
  • Protocol-specific errors, such as HTTP 500 - internal server error or HTTP 503 - service unavailable.
  • Exceptions explicitly thrown as part of the service call
Note: This list doesn't include HTTP 4xx errors. These are used to indicate the client is doing something wrong, but the remote server itself is fine. Assuming whatever underlying issue is causing problems is addressed, service calls will return to normal as the circuit breaker timeouts expire.

Circuit Breaker Timeout Behavior

The timeout caused when the circuit breaker reaches its limit is calculated dynamically. Each time a registered web service wrapper is called, the circuit breaker is checked. If the circuit breaker is triggered, then the call isn't made. In the example below, a web service is triggered every millisecond. In this example, the circuit breaker is configured with a maximum of three failed calls per 6 milliseconds.