Distributed Systems
The Gatekeeper of Your Microservices Landscape
When a company splits its backend into dozens of microservices, it quickly ends up exposing a maze of internal services to the outside world. If your mobile app or frontend were to connect directly to all those individual services, you would face serious challenges: CORS issues, scattered authentication logic, duplicate SSL certificates, and the lack of centralized protection against overload. The architectural solution to this problem is the API Gateway.
The API Gateway acts as the single centralized entry point (Reverse Proxy) for all incoming external traffic. It handles SSL termination, validates JWT tokens before requests reach the microservices, performs load balancing, and implements one of the most critical security mechanisms for public APIs: Rate Limiting.
Why Rate Limiting Is Critical
Without rate limiting, excessive traffic can overwhelm an API and negatively impact databases and backend services. This may lead to performance degradation or service interruptions.
Rate limiting controls how many requests a user, IP address or API key can send during a specific period. For example, an API may allow up to 100 requests per minute. This helps maintain stability, availability and predictable performance.
Token Bucket vs. Leaky Bucket Algorithm
To calculate rate limits efficiently and at extremely high speed (often using an in-memory cache such as Redis), architects rely on specialized mathematical algorithms. The two most well-known are the Token Bucket and the Leaky Bucket.
1. Token Bucket Algorithm
Imagine a bucket with a fixed maximum capacity (for example, 10 tokens). The bucket is continuously refilled at a fixed rate (for example, 2 tokens per second).
Every time a client sends an API request, the gateway attempts to remove one token from the bucket.
- If the bucket is full, any additional tokens are discarded.
- If the bucket is empty, the request is rejected with an HTTP 429 Too Many Requests status code.
This algorithm is popular because it allows bursts. A user who has been inactive for a while can suddenly make ten requests in rapid succession, provided there are enough tokens available in the bucket.
2. Leaky Bucket Algorithm
This algorithm focuses on maintaining a constant outgoing flow of traffic.
Incoming requests are placed into a bucket (queue). The bucket then “leaks” requests to the backend services at a strictly constant and predictable rate.
- If requests arrive faster than they can be processed, they accumulate in the bucket.
- If the bucket overflows, excess requests are immediately dropped.
This approach protects sensitive backend databases from traffic spikes of any kind. However, from a user perspective, it may feel slower because bursts are smoothed out and processed over time.
Conclusion
A well-designed API Gateway combined with effective rate limiting protects your infrastructure and helps guarantee stable uptime and predictable performance.
Learn more about API strategies on Frankwatching:
Next: Service Mesh Architecture (Istio vs. Linkerd): Mutual TLS and Traffic Shifting
