Nginx - Rate Limiting

Introduction

Nginx is a widely used and versatile piece of open source software, best known for it’s high performance when handling many connections. It can be used as a webserver, reverse proxy, load balancer, SSL/TLS terminator and more. At its core Nginx is a collection of modules. To list currently installed modules use:

sudo nginx -V 2>&1 | tr -- - '\n' | grep  _module

This command does not list the modules already included in the source code like ngx_http_limit_req_module which can be found in the /src/http/modules directory.

Goal

Use ngx_http_limit_req_module in combination with fail2ban to add rate limiting to a webserver and temporarily ban users exceeding the limit.

Requirements

• Nginx
• fail2ban

Configuring Nginx

The official documentation for ngx_http_limit_req_module contains lots of useful information on configuring the module.

http {
    limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
    ...
    server {
        ...
        location /search/ {
            limit_req zone=one burst=5;
        }

This configuration allows not more than 1 request per second with bursts up to 5 requests and keeps track of the requests in zone “one” using up to 10MB of memory. For most websites those limits will not be sufficient and need to be tweaked. But for now those limits are fine since it will be easy to exceed them and test if fail2ban actually behaves as expected.

Configuring fail2ban

Fail2ban includes the nginx-limit-req jail which reads nginx logs and searches for any entries indicating that a client sent too many requests.

[nginx-limit-req]
port    = http,https
logpath = %(nginx_error_log)s

Limit testing

To test those limits you can manually trigger requests using your browser or use a tool like siege.

siege -d 1 -c 1 http://url

-d controls the delay in seconds, while -c sets the concurrency. Setting both to 1 will result in about 1 request per second. Once the limit is exceeded siege should start reporting errors like:

[error\] socket: unable to connect sock.c:282: Connection refused

To list/unban clients in fail2ban I wrote some short bash scripts.

#!/bin/bash
sudo fail2ban-client banned | tr ' " | jq

if ! [ -z $1 ]
then
        echo ""
        sudo fail2ban-client status $1
        echo ""
        sudo fail2ban-client get $1 banip --with-time
fi

#!/bin/bash
sudo fail2ban-client set nginx-limit-req unbanip $1