How to get the highest RPS out of Nginx through tweaks, sysctls, and config options.
Nginx is the most common web server today, and a very powerful and fast option. However, the default configuration performs quite poorly and advanced Linux knowledge is required to scale it. This guide will help you achieve that.
Most users can skip this section, but incase Nginx is not installed here is an example for a new Ubuntu server.
apt-get update
apt-get install nginx
mkdir -p /tmp/cache
rm /var/www/html/*
echo "Basic file" > /var/www/html/index.html
/etc/sysctl.conf
holds the various OS tweaks for your server. It allows you to tune the
performance of your system in general, and particularly what Nginx can achieve. Add these lines
to the bottom of your sysctl.conf file:
net.ipv4.tcp_syncookies=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.all.secure_redirects=1
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv6.conf.all.accept_source_route=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_mem=786432 1697152 1945728
net.ipv4.tcp_rmem=4096 4096 16777216
net.ipv4.tcp_wmem=4096 4096 16777216
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.netfilter.ip_conntrack_max=999999
net.netfilter.nf_conntrack_max=999999
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_max_orphans=262144
net.ipv4.ip_local_port_range=1000 65535
net.ipv4.tcp_fin_timeout=30
net.core.netdev_max_backlog=10000
net.core.somaxconn=60000
net.ipv4.tcp_synack_retries=3
fs.file-max=640000
Below is a sample high performance /etc/nginx/nginx.conf
example. You can adapt this to your
own configuration.
In specific notice these values:
user www-data;
worker_processes 12;
worker_rlimit_nofile 2621440;
pid /run/nginx.pid;
events {
worker_connections 65536;
use epoll;
multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log off;
error_log /var/log/nginx/error.log;
gzip off;
proxy_cache_path /tmp/cache keys_zone=cache:10m levels=1:2 inactive=600s max_size=100m;
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
proxy_cache cache;
proxy_cache_valid 200 30s;
proxy_cache_use_stale updating;
proxy_cache_lock on;
}
}
In order to allow nginx to have a large number of open network connections and files you must raise the system limits.
Add this to the bottom of /etc/security/limits.conf
and replace 'www-data' with your nginx user:
root soft nofile 65536
root hard nofile 65536
www-data soft nofile 65536
www-data hard nofile 65536
Add this to bottom of /etc/pam.d/common-session
:
session required pam_limits.so
Add this to bottom of /etc/systemd/system.conf
:
DefaultLimitNOFILE=65536
Now reboot the system to apply your changes. This configuration should achieve 50,000+ requests per second.