Rate limits, canary releases and circuit breakers on every Route
Limit requests, rates and connections per client, Consumer or key, share the count across gateway nodes in Redis, split traffic between backend versions and stop sending requests to a failing backend. Set in a form, applied by the gateway, recorded in the audit log.
Limits that live in each service, or nowhere
When limits and release rules are written into each service, one busy client, one release or one failing backend reaches every other client.
One busy client slows down the others
A partner’s batch job takes the capacity of everyone else.
With G8Limit requests, request rates and concurrent connections per Consumer or key, with the window and the count you choose.
Releases reach every client at once
A new backend version goes live for everyone, and so does its first bug.
With G8Canary splits send part of the traffic to the new version by weight or match rule, and the rest stays where it was.
A failing backend takes clients with it
Requests queue up behind a backend that no longer answers.
With G8api-breaker opens the circuit after the statuses you define and answers clients with the response you set until the backend is healthy again.
Limits, splits and breakers in one form each
Each control is a Plugin of the G8 engine with its own form in the console. Attach it to a Route, a Service or a Consumer, or apply it across the gateway. Every change is checked and recorded.
Limit what each client can send
limit-count caps requests per time window, limit-req caps the request rate with a leaky bucket and limit-conn caps concurrent connections. Count by Consumer, by client address or by a key you read from the request.
- Limits from the requestSet the count and the window as numbers, or read them from each request, such as from a header.
- Per Route or across the gatewayAttach a limit to a Route, a Service or a Consumer, or apply it to every request.
Release backends gradually
traffic-split sends a share of the traffic to another Upstream by weight, or by match rules on headers, arguments or cookies. api-breaker opens the circuit after the unhealthy statuses you list and closes it after the healthy ones.
- Circuit breakerWhile the breaker is open, clients get the response you set and the upstream isn’t asked.
- Settings in the formBreak response code, Unhealthy and Healthy HTTP statuses and Break response body.
Answer repeated requests from the gateway
proxy-cache keeps upstream responses on disk or in memory, keyed on the parts of the request you choose, for the methods and statuses you list. Repeated requests never reach the backend.
- Keyed on the requestHost and URI by default, or the parts of the request you choose.
- What is cachedOnly the request methods and response statuses you list.
An online retailer before a campaign
A typical setup, drawn to show how the parts fit together. It doesn’t describe a specific customer.
A retailer expects ten times the usual traffic for a week. The catalog backend has a new version, the payments backend is shared with other channels, and three partners sell through the API.
Each partner is a Consumer with limit-count and limit-conn; the count is shared across the gateway nodes in Redis.
traffic-split sends ten percent of catalog requests to the new version; Analytics compares the response times of both.
api-breaker opens after three 5xx answers and returns a fixed response, so the checkout fails fast instead of queueing.