High Performance Nginx

Overview

Nginx (pronounced "engine X") is the most common web server today, and a very powerful and fast option.

However, the default configuration performs quite poorly and advanced Linux knowledge is required to scale it. This guide will help you achieve the best performance from your Nginx web server by tuning the operating system and nginx.conf settings.

Note: This guide requires root access for system settings, and the ability to edit nginx.conf for webserver settings.

Base requirements

Most users can skip this section, but incase Nginx is not installed here is an example for a new Ubuntu server. This will install Nginx and create a basic html file.

apt-get update
apt-get install nginx

mkdir -p /tmp/cache

rm /var/www/html/*
echo "Basic file" > /var/www/html/index.html

Tweaking sysctl.conf

/etc/sysctl.conf holds the various OS tweaks for your server. It allows you to tune the performance of your system in general, and particularly what Nginx can achieve.

Add these lines to the bottom of your sysctl.conf file:

net.ipv4.tcp_syncookies=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.all.secure_redirects=1
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv6.conf.all.accept_source_route=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_mem=786432 1697152 1945728
net.ipv4.tcp_rmem=4096 4096 16777216
net.ipv4.tcp_wmem=4096 4096 16777216
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.netfilter.ip_conntrack_max=999999
net.netfilter.nf_conntrack_max=999999
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_max_orphans=262144
net.ipv4.ip_local_port_range=1000 65535
net.ipv4.tcp_fin_timeout=30
net.core.netdev_max_backlog=10000
net.core.somaxconn=60000
net.ipv4.tcp_synack_retries=3
fs.file-max=640000

The above are general network tweaks. They are designed to improve the network performance of Nginx and increase the resources available and are all safe to use as defaults.

Tweaking nginx.conf

nginx.conf is the configuration file for your Nginx webserver and is where the majority of our performance changes reside.

Below is a sample high performance /etc/nginx/nginx.conf example. You can adapt this to your own configuration. In specific, please take note of these values:

• worker_processes: this allows us to run one nginx for each CPU in the system
• worker_rlimit_nofile: raise the number of open files a worker can have
• worker_connections: max out the limit of connections per worker
• sendfile, tcp_nopush, tcp_nodelay, use epoll, multi_accept: these are all performance tweaks

user www-data;
worker_processes 12;
worker_rlimit_nofile 2621440;
pid /run/nginx.pid;

events {
        worker_connections 65536;
        use epoll;
        multi_accept on;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    open_file_cache max=200000 inactive=20s;
    open_file_cache_valid 30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    access_log off;
    error_log /var/log/nginx/error.log;

    gzip off;

    proxy_cache_path /tmp/cache keys_zone=cache:10m levels=1:2 inactive=600s max_size=100m;

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        root /var/www/html;
        server_name _;

        location / {
            try_files $uri $uri/ =404;
        }

        proxy_cache cache;
        proxy_cache_valid 200 30s;
        proxy_cache_use_stale updating;
        proxy_cache_lock on;
    }
}

Increasing system file limits

In order to allow nginx to have a large number of open network connections and files you must raise the system limits. You will often see errors like "nginx: too many open files" in your error.log if you hit these limits.

Add this to the bottom of /etc/security/limits.conf and replace 'www-data' with your nginx user:

root soft nofile 65536
root hard nofile 65536
www-data soft nofile 65536
www-data hard nofile 65536

Add this to bottom of /etc/pam.d/common-session:

session required pam_limits.so

Add this to bottom of /etc/systemd/system.conf:

DefaultLimitNOFILE=65536

Apply changes

Now reboot the system to apply your changes. This configuration should achieve 50,000+ requests per second.

Testing

With LoadForge you can test the response time of all your website pages, and how they perform under stress by simulating many virtual users. Sign up today to test your systems.