nginx rate limit with limit_req_zone: burst tuning, per-key zones, and 429

I've watched three separate teams learn about nginx rate limiting the same way: a dashboard that reads fine for weeks, then a scraper points a botnet at /api/search, and database latency doubles in ninety seconds. The 503s you serve back cost the same CPU as the 200s they replaced, so nothing gets cheaper. Real users see slow pages, on-call sees a graph shaped like a cliff, and the scraper doesn't care. Nginx ships limit_req_zone as the answer, and the whole thing costs almost nothing at runtime: a small shared-memory block at the reverse-proxy tier, no application code to change. The catch, and this is the part that eats an afternoon, is that a first-pass config usually lands either too tight (real users get rejected on a page-load burst) or too loose (the scraper walks straight through). Below, I'll step through five commits on a small companion repo (nginx in front of FastAPI, a per-IP zone at 5 r/s, burst and nodelay tuning, a second zone keyed on X-API-Key so paying customers get a bigger budget, and a JSON 429 that ships Retry-After so clients back off cleanly instead of retry-looping). The stack runs on any laptop with docker-compose, no paid services, no cloud rate limiter, no application changes.

Step 1: Baseline nginx forwarding to FastAPI (commit 05c6764)

Try it: 05c6764

What does an unprotected endpoint actually look like when you point sixty requests per second at it? Two containers, twelve lines of nginx config, and any bash for loop can flood it. That's the starting shape we're going to fix. The baseline is a two-service docker-compose stack: FastAPI on port 8000 inside the network, nginx on port 80 (published as 8080 on the host).

http {
    upstream backend { server backend:8000; }

    server {
        listen 80;
        server_name _;

        location /api/ {
            proxy_pass         http://backend;
            proxy_set_header   Host              $host;
            proxy_set_header   X-Real-IP         $remote_addr;
            proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        }
    }
}

Bring the stack up with docker compose up -d --build, then curl http://127.0.0.1:8080/api/ping returns {"pong": true}. At this point, a for-loop in bash can hammer the endpoint at whatever rate your CPU will produce, and every single request lands on FastAPI. This is the state we're going to fix.

Two production-grade details are already baked in. First, X-Real-IP and X-Forwarded-For are set so the backend can tell who the real client is, which matters later when the backend logs the same info nginx will see when it applies the rate limit. Second, the upstream uses a name (backend:8000) rather than a container IP, which lets docker-compose swap the container without touching nginx.

Step 2: First limit_req_zone at 5 r/s (commit 787deda)

Try it: 787deda

Most people expect a rate-limit rule to live next to the endpoint it protects, but limit_req_zone refuses to sit inside server or location. It has to live one level up in the http block, and skipping that detail costs the next hour of debugging. A separate limit_req inside a location attaches the zone to specific requests.

http {
    limit_req_zone $binary_remote_addr zone=per_ip:10m rate=5r/s;

    server {
        location /api/ {
            limit_req zone=per_ip;
            proxy_pass http://backend;
        }
    }
}

Four parameters carry the important semantics. The key is $binary_remote_addr, four bytes per IPv4 client and sixteen bytes per IPv6 client, which is a lot cheaper than $remote_addr (its string form). The zone name is per_ip (arbitrary label). The size is 10m, ten megabytes of shared memory, which fits roughly a hundred sixty thousand unique IPs. The rate is 5r/s, expressed as a leaky-bucket refill of one slot every two hundred milliseconds.

With no burst argument on limit_req, the queue depth is zero. Any request that arrives faster than one every two hundred milliseconds is rejected immediately with the default 503. Fire the bundled scripts/load-test.sh and you'll see a spread like 2xx=6 429=54, because the tiny 5 r/s budget can't absorb a normal user opening a page with a dozen assets. This is the shape of a rate limit that's technically correct and operationally useless.

Step 3: burst=20 nodelay and switching to 429 (commit 3b94c12)

Try it: 3b94c12

A colleague of mine once shipped a 5 r/s limit on a Friday evening and spent the weekend explaining to customers why the checkout page kept returning 503s. Three small changes (the ones below) turn a technically-correct rate limit into one a real browser can survive. First, burst=20 gives the leaky bucket a queue of twenty extra slots, so a page-load burst can consume a whole page of tokens up front. Second, nodelay tells nginx to forward the queued requests immediately instead of pacing them at the 5 r/s refill rate. Third, limit_req_status 429 changes the reject code from 503 Service Unavailable (which suggests server failure) to 429 Too Many Requests (which suggests back-off).

http {
    limit_req_zone $binary_remote_addr zone=per_ip:10m rate=5r/s;
    limit_req_status 429;

    server {
        location /api/ {
            limit_req zone=per_ip burst=20 nodelay;
            proxy_pass http://backend;
        }
    }
}

Fire the load test again against the same 5 r/s zone and the numbers flip: 2xx=42 429=18 for a sixty-request hammer. The first twenty five requests (five per second plus twenty burst) pass in the first second. The next thirty five arrive while the queue is full and the bucket has only drained a few slots, so they get 429ed. The pattern lets a real page-load burst through but still catches a scraper that never slows down.

A word on nodelay versus no nodelay. Without nodelay, nginx smooths the burst: the twenty queued requests get released one every two hundred milliseconds, so the client experiences a slow response for its later assets but no rejections. That's often what you want on a public homepage. Adding nodelay prioritises fail-fast over slow-down, which is what most JSON APIs want because clients would rather retry than wait five seconds for a GET /api/search. Pick one per endpoint; don't blindly copy nodelay everywhere.

The 429 status also matters for downstream tooling. Many HTTP clients (browsers, requests, axios, undici) treat 503 as "server broke, log a scary error" and 429 as "server said slow down, apply the backoff hook you already wrote". The two-line change is one of the cheapest wins in this whole config.

Step 4: Per-API-key zone with a map fallback (commit 34e3062)

Try it: 34e3062

The per-IP zone is fine for anonymous traffic, but paying customers on a shared office NAT will all share one bucket and hit the limit constantly. The fix is a second zone keyed on the API key header, stacked on top of the per-IP zone. Both limit_req rules run; whichever bucket empties first wins.

http {
    map $http_x_api_key $api_key_bucket {
        default      $http_x_api_key;
        ""           "";
    }

    limit_req_zone $binary_remote_addr zone=per_ip:10m  rate=5r/s;
    limit_req_zone $api_key_bucket     zone=per_key:10m rate=50r/s;

    server {
        location /api/ {
            limit_req zone=per_ip  burst=20  nodelay;
            limit_req zone=per_key burst=100 nodelay;
            proxy_pass http://backend;
        }
    }
}

The map block might look redundant, but it does one important thing: when X-API-Key is missing or empty, the mapped value is the empty string, and nginx skips empty-key counters entirely. Without the map, all anonymous requests would collide into a single hot bucket labeled "" on the per_key zone, which is worse than useless (it defeats the purpose of both zones at once).

With the map in place, an authenticated caller with X-API-Key: acme_prod_123 gets a 50 r/s budget with a 100-slot burst, and the per-IP rule still runs as a safety net in case one key gets leaked. An anonymous caller falls through to per-IP alone. Testing with the bundled script:

scripts/load-test.sh http://127.0.0.1:8080/api/ping 60 acme_prod_123
# 2xx=60  429=0

scripts/load-test.sh http://127.0.0.1:8080/api/ping 60
# 2xx=42  429=18

The authenticated call sails through because the higher zone absorbs the whole test. The anonymous call gets the same profile as before because the per-IP zone hasn't changed. This is exactly the shape you want when rolling out a paid tier: raise the ceiling for people who paid, keep the guard for people who didn't.

Step 5: JSON 429 response with Retry-After (commit 2bd1798)

Try it: 2bd1798

The default nginx 429 page is HTML. If your API is JSON, a client that parses every response body will error out on the rejection with a SyntaxError from an unexpected < at position zero, which pollutes logs and confuses on-call. Fix it with an error_page rewrite to a named internal location.

server {
    listen 80;
    server_name _;

    error_page 429 = @rate_limited;

    location /api/ {
        limit_req zone=per_ip  burst=20  nodelay;
        limit_req zone=per_key burst=100 nodelay;
        proxy_pass http://backend;
    }

    location @rate_limited {
        internal;
        default_type application/json;
        add_header Retry-After 1 always;
        add_header Cache-Control "no-store" always;
        return 429 '{"error":"rate_limited","retry_after":1}';
    }
}

Three details are worth calling out. The internal; directive on @rate_limited prevents a client from hitting the location directly, which would otherwise let anyone GET a 429 on demand and inflate their own metrics. The always modifier on add_header is required because nginx by default only applies added headers on 2xx and 3xx responses; without always the header would silently drop on the 429. The body is a stable JSON envelope so a client can parse retry_after programmatically instead of scraping the Retry-After header, which some HTTP libraries strip.

Verifying end to end:

$ curl -sSi http://127.0.0.1:8080/api/ping | head -8
HTTP/1.1 429 Too Many Requests
Server: nginx/1.25.5
Content-Type: application/json
Content-Length: 40
Retry-After: 1
Cache-Control: no-store

{"error":"rate_limited","retry_after":1}

The response is now something an axios interceptor can act on: parse the JSON, read retry_after, schedule the retry. No HTML fallback, no header stripping, no ambiguity.

Comparing zone key strategies

Which variable you key on matters more than the rate you pick. The four common strategies:

Zone key	Directive	Best for	Failure mode
Per-IP	$binary_remote_addr	Anonymous public traffic	Office NAT shares one bucket
Per-API-key	$http_x_api_key	Paying customers, trusted callers	Missing header collapses to fallback
Per-user session	$http_authorization	Session-based apps	Rotates on every login, weak against burst
Per-country	$geo_country (via ngx_http_geo_module)	Geographic quota	Cheap to bypass with a VPN

Stack two of them the way this walkthrough does, and the trade-offs cancel: per-IP catches the office NAT bucket problem on the way out, while per-API-key raises the ceiling for authenticated callers who would otherwise be punished for sharing a network. The nginx limit_req module docs cover the full parameter set including delay=, dry_run, and multi-key composition, all worth reading once the basics are in place.

Where this configuration stops being enough

Nginx limit_req operates on request count per second per key. Two shapes of traffic slip past it. First, slow abuse: one request per second per IP for hours, adding up to millions of hits without ever crossing the per-second budget. That needs fail2ban reading the access log or an application-tier counter. Second, distributed abuse: a thousand IPs each doing five requests per second per key. The per-key zone catches that when the attacker uses one key, but not when they rotate. That needs application-level quota, or a WAF that can cluster by TLS fingerprint.

Nginx-level rate limiting also can't express business rules like one signup per email per hour, because nginx doesn't know what an email is. Anything key-shaped that lives in a database has to be enforced in the app. What nginx can do is stop the naive-loop scraper and the botnet crawler from ever reaching the app in the first place, and that's worth a lot for six lines of config.

Repository

Full source at https://github.com/vytharion/nginx-rate-limit-zone-burst.

• Lesson 1, commit 05c6764, baseline nginx plus FastAPI backend
• Lesson 2, commit 787deda, first limit_req_zone at 5 r/s with no burst
• Lesson 3, commit 3b94c12, burst=20 nodelay plus 429 status
• Lesson 4, commit 34e3062, per-API-key zone with map fallback
• Lesson 5, commit 2bd1798, JSON 429 body with Retry-After header

Clone the repo, run docker compose up -d --build, and step through the commits with git checkout <sha> to see each layer in isolation. The MDN reference for Retry-After is the canonical source on how compliant clients treat the header.

Wrap-up

The full protection stack is thirteen lines of nginx config on top of what you already run. Start with per-IP at 5 r/s to catch the naive attacker. Add burst and nodelay so real users survive their own asset bursts. Layer a per-API-key zone once you have paying tenants. Return 429 with a stable JSON body so client-side backoff works without special-case parsers. Ship it, watch the access log for two days, and adjust rates based on what real traffic looks like rather than what a synthetic benchmark says.

The commit list above is the entire progression. Every commit runs at every point; each one adds exactly one idea on top of the last.