Tweaking Nginx for high performance

Overview

Nginx is the most common web server today, and a very powerful and fast option. However, the default configuration performs quite poorly and advanced Linux knowledge is required to scale it. This guide will help you achieve that.

Base requirements

Most users can skip this section, but incase Nginx is not installed here is an example for a new Ubuntu server.

apt-get update
apt-get install nginx

mkdir -p /tmp/cache

rm /var/www/html/*
echo "Basic file" > /var/www/html/index.html

Tweaking sysctl.conf

/etc/sysctl.conf holds the various OS tweaks for your server. It allows you to tune the performance of your system in general, and particularly what Nginx can achieve. Add these lines to the bottom of your sysctl.conf file:

net.ipv4.tcp_syncookies=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.all.secure_redirects=1
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv6.conf.all.accept_source_route=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_mem=786432 1697152 1945728
net.ipv4.tcp_rmem=4096 4096 16777216
net.ipv4.tcp_wmem=4096 4096 16777216
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.netfilter.ip_conntrack_max=999999
net.netfilter.nf_conntrack_max=999999
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_max_orphans=262144
net.ipv4.ip_local_port_range=1000 65535
net.ipv4.tcp_fin_timeout=30
net.core.netdev_max_backlog=10000
net.core.somaxconn=60000
net.ipv4.tcp_synack_retries=3
fs.file-max=640000

Tweaking nginx.conf

Below is a sample high performance /etc/nginx/nginx.conf example. You can adapt this to your own configuration.

In specific notice these values:

• worker_processes: this allows us to run one nginx for each CPU in the system
• worker_rlimit_nofile: raise the number of open files a worker can have
• worker_connections: max out the limit of connections per worker
• sendfile, tcp_nopush, tcp_nodelay, use epoll, multi_accept: these are all performance tweaks

user www-data;
worker_processes 12;
worker_rlimit_nofile 2621440;
pid /run/nginx.pid;

events {
        worker_connections 65536;
        use epoll;
        multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    access_log off;
    error_log /var/log/nginx/error.log;

    gzip off;

    proxy_cache_path /tmp/cache keys_zone=cache:10m levels=1:2 inactive=600s max_size=100m;

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        root /var/www/html;
        server_name _;

        location / {
            try_files $uri $uri/ =404;
        }

        proxy_cache cache;
        proxy_cache_valid 200 30s;
        proxy_cache_use_stale updating;
        proxy_cache_lock on;
    }
}

Increasing system file limits

In order to allow nginx to have a large number of open network connections and files you must raise the system limits.

Add this to the bottom of /etc/security/limits.conf and replace 'www-data' with your nginx user:

root soft nofile 65536
root hard nofile 65536
www-data soft nofile 65536
www-data hard nofile 65536

Add this to bottom of /etc/pam.d/common-session:

session required pam_limits.so

Add this to bottom of /etc/systemd/system.conf:

DefaultLimitNOFILE=65536

Apply changes

Now reboot the system to apply your changes. This configuration should achieve 50,000+ requests per second.