Nginx at EGPHP

The front door, built properly.

Nginx decides what reaches your site: which server and which PHP pool answer, what comes from cache, and what is turned away at the door. This is how our team sets it up, with the configuration to show for it.

Runs on and built with
  • NGINX
  • PHP
  • Redis

One request, from the door to your code

Most requests never need PHP at all. Each stop below can answer on its own, and only what is left travels on to the next.

  1. The connection

    Encrypted with TLS 1.3, and one connection carries every file of the page over HTTP/2 or HTTP/3.

  2. Limits

    Too many requests from one address, too many open connections, or a body larger than the site accepts is answered here.

    429 or 413: stops here
  3. Forbidden paths

    Hidden files, backups and any script inside an upload folder are refused before PHP hears of them.

    403 or 404: stops here
  4. Static files

    Images, styles and scripts leave straight from disk, with long cache lifetimes.

    Served: stops here
  5. Page cache

    A visitor who is not logged in gets a finished page from memory.

    Cache hit: stops here
  6. The balancer

    Picks the server and the PHP pool with room for the request.

  7. The account's own pool

    PHP runs as the account's own user, inside its own folders, under its own php.ini.

Load balancing between servers

When one server is not enough, Nginx stands in front of several and shares the traffic out. The method decides who takes the next request.

An example: addresses and values are illustrative

The default. Each server takes the next request in turn: right when the servers are alike and requests cost about the same.

The next request goes to the server with the fewest open connections: right when some requests are long, such as reports, exports and uploads.

The visitor's address picks the server, so the same visitor keeps landing on the same one. For applications that keep sessions on local disk; better still, we move the sessions to Redis and drop the need.

conf.d/app.conf
upstream app_servers {
    # round robin: the default, no line needed    least_conn;    ip_hash;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.13:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.14:8080 backup;    # no backup server with ip_hash
    keepalive 32;
}

server {
    server_name example.com;
    location / {
        proxy_pass http://app_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_next_upstream error timeout http_502 http_503;
    }
}
max_fails · fail_timeout
Health checks with real traffic: after three failed attempts within thirty seconds a server is left out for thirty seconds, then tried again.
backup
A standby server that receives traffic only when every other server is unavailable.
proxy_next_upstream
A request that meets an error or a timeout is tried on the next server. A request that changes data is not sent twice, so an order is never placed twice.
keepalive
Connections to the servers stay open between requests instead of being opened again for each one.
weight
A larger server can be given a larger share of the requests.
down
One server is taken out for maintenance with a reload, while the others carry the traffic.

Load balancing between PHP pools

The same idea inside a single server. A site can have more than one PHP-FPM pool behind it, so one pool can restart while the other answers, and slow work never takes the workers the storefront needs.

Nginx ~ \.php$^~ /admin/
shop-a.sock Storefront, pool A Answering Answering, and taking B's requests
shop-b.sock Storefront, pool B Answering Restarting with a new setting: no visitor waits for it
shop-admin.sock Admin and reports Its own workers, apart from the shop
sites/example.com.confAn example
upstream php_shop {
    server unix:/run/php/shop-a.sock max_fails=2 fail_timeout=10s;
    server unix:/run/php/shop-b.sock max_fails=2 fail_timeout=10s;
    keepalive 8;
}

# server {}
location ~ \.php$ {
    try_files $uri =404;
    include fastcgi.conf;
    fastcgi_pass php_shop;
    fastcgi_keep_conn on;
    fastcgi_next_upstream error timeout;
}

location ^~ /admin/ {
    location ~ \.php$ {
        try_files $uri =404;
        include fastcgi.conf;
        fastcgi_pass unix:/run/php/shop-admin.sock;
    }
}
  • Two pools, one site

    Both pools run the same code as the same account. A new PHP setting reaches one pool, then the other, and visitors keep being answered throughout.

  • Failover

    fastcgi_next_upstream moves a request to the other pool when its socket refuses or times out. A request that changes data is not sent twice.

  • A lane for slow work

    The admin area, exports and reports get a pool of their own, so an hour-long report cannot take the workers a customer needs to check out.

The rules at the door

Protection starts before PHP runs a single line. These rules go into every server block we write, adjusted to the site.

sites/example.com.confAn example: the values are set per site
# http {}
limit_req_zone  $binary_remote_addr zone=perip:10m rate=10r/s;
limit_req_zone  $binary_remote_addr zone=login:10m rate=5r/m;
limit_conn_zone $binary_remote_addr zone=conns:10m;server {
    listen 443 ssl;
    http2 on;
    server_name example.com;
    limit_req  zone=perip burst=40 nodelay;
    limit_conn conns 30;
    limit_req_status 429;
    location = /wp-login.php {
        limit_req zone=login burst=5 nodelay;
        include fastcgi.conf;
        fastcgi_pass php_shop;
    }    client_max_body_size 64m;
    client_header_timeout 15s;
    client_body_timeout 15s;    location ~ /\.(?!well-known/) { deny all; }
    location ~* \.(?:bak|old|orig|sql|swp)$ { deny all; }    location ~* ^/(?:wp-content/uploads|uploads|files)/.*\.php$ {
        deny all;
    }    location ~ \.php$ {
        try_files $uri =404;
        include fastcgi.conf;
        fastcgi_pass php_shop;
    }    ssl_protocols TLSv1.2 TLSv1.3;
    server_tokens off;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;}
  1. Rate and connection limits

    Each address gets a steady rate with room for a short burst; past it, Nginx answers 429 and PHP is never woken. The login page gets a much tighter limit of its own, which takes the air out of password guessing.

  2. Body size and timeouts

    An upload larger than the site accepts is refused at once, and a client that sends its request very slowly is disconnected instead of holding a connection open.

  3. Hidden files and leftovers

    Anything whose name starts with a dot, such as .git or .env, and backup copies of files are never served. The one exception is the folder that certificate checks use.

  4. Uploads never run PHP

    A folder that accepts files from visitors serves them as files. A script placed there is refused, whatever its name.

  5. Only a PHP file that exists

    try_files $uri =404 sends a request to PHP only if that exact file is on disk. This closes the classic trap drawn below.

  6. Headers and TLS

    HSTS keeps browsers on HTTPS, the other headers stop a page being sniffed or framed by another site, and the server does not announce its version. TLS 1.2 and 1.3 only.

    How we secure the server itself

The trap one missing line opens

/uploads/avatar.jpgA real file: an image with PHP code hidden inside /shell.phpDoes not exist
Without the rule

On an old or careless setup, PHP walks back along the path, finds the image and runs the code inside it.

With it

A 404, and nothing runs. cgi.fix_pathinfo=0 in PHP and security.limit_extensions in the pool are the second and third locks on the same door.

Every account is a walled room

On a server shared by several customers, one site must never be able to read another. We build that wall in layers, so no single setting holds it alone.

shopAccount
Linux user
shop
PHP pool
/run/php/shop.sock
Allowed folders
/home/shop
Settings
pool.d/shop.conf
clinicAccount
Linux user
clinic
PHP pool
/run/php/clinic.sock
Allowed folders
/home/clinic
Settings
pool.d/clinic.conf
blogAccount
Linux user
blog
PHP pool
/run/php/blog.sock
Allowed folders
/home/blog
Settings
pool.d/blog.conf
The shop's code reaching for the clinic's files file_get_contents('/home/clinic/public_html/config.php') Permission deniedopen_basedir restriction in effect
  1. Its own Linux user

    Each site runs as its own user, and every home folder is closed to other accounts. This is the wall that holds even if PHP is left out of the picture.

  2. Its own PHP-FPM pool

    One pool per account, reachable only through its own socket. A busy or broken site uses up its own workers, not its neighbours'.

  3. open_basedir and include_path

    PHP may open files only inside the account's own folders, and looks for included files only there. A script that asks for a path outside them is refused before the disk is touched.

  4. Its own temporary files and sessions

    Uploads in progress and session files stay inside the account, not in a shared /tmp where any site could read another site's logged-in sessions.

  5. No functions the site does not need

    open_basedir is not a sandbox: it does not stop a program started from PHP. So the functions that run shell commands are switched off per account, unless the site truly needs one.

php/8.3/fpm/pool.d/shop.confAn example
[shop]
user  = shop
group = shop
listen = /run/php/shop.sock
listen.owner = www-data
listen.group = www-data
listen.mode  = 0660

pm = ondemand
pm.max_children = 12
pm.process_idle_timeout = 10s

php_admin_value[open_basedir] = /home/shop/public_html:/home/shop/lib:/home/shop/tmp
php_admin_value[include_path] = .:/home/shop/lib
php_admin_value[upload_tmp_dir] = /home/shop/tmp
php_admin_value[sys_temp_dir] = /home/shop/tmp
php_admin_value[session.save_path] = /home/shop/tmp/sessions
php_admin_value[disable_functions] = exec,passthru,shell_exec,system,proc_open,popen
php_admin_flag[allow_url_include] = off

php.ini, per account

Every account has its own settings, written into its own pool. Some are fixed by us, some the site may change, and the difference is one word.

php_admin_value: fixed

Set for the account. Neither ini_set() nor a .user.ini file can change it.

memory_limit = 256M
max_execution_time = 60
open_basedir = /home/shop/…
disable_functions = exec,…

php_value: a starting value

Our default for the account. The site may raise or lower it where PHP allows.

upload_max_filesize = 32M
post_max_size = 32M
max_input_vars = 3000

.user.ini: the site's own

A file in the site's folder for the settings it is allowed to change. PHP rereads it every five minutes by default.

upload_max_filesize = 64M
post_max_size = 64M
display_errors = Off

An example: the values are set per site

Limits that have to agree

Upload size
upload_max_filesizePHP≤ post_max_sizePHP≤ client_max_body_sizeNginx

If Nginx allows less, the upload is refused with a 413 before PHP sees it; if PHP allows less, the file arrives and is thrown away.

Time
max_execution_timePHP< request_terminate_timeoutPHP-FPM≤ fastcgi_read_timeoutNginx

PHP should stop itself first, with an error it can log; the pool stops what PHP could not; and Nginx waits long enough not to show a 504 while PHP is still working.

Cache, in layers

Each layer keeps a different thing, and a request goes only as deep as it must. Switch between the two visitors to see where each one stops.

  1. BrowserStyles, scripts and images with a version in their name, kept for a year.
    Already has the filesAlready has the files
  2. Nginx page cacheFinished pages, kept from a few seconds to a few minutes.
    Hit: the page is answered hereSkipped: a login or cart cookie means the page is personal
  3. OPcachePHP compiled once and kept in memory.
    Not reachedThe code runs without being read or compiled again
  4. RedisRepeated database answers, and sessions.
    Not reachedMost answers come from memory
  5. DatabaseThe source of truth.
    Not reachedAsked only for what Redis did not have
conf.d/cache.confAn example: the values are set per site
# http {}
fastcgi_cache_path /var/cache/nginx/shop levels=1:2
                   keys_zone=shop:50m inactive=60m max_size=2g;

map $http_cookie $skip_cache {
    default                     0;
    ~wordpress_logged_in_       1;
    ~woocommerce_items_in_cart  1;
}

# location ~ \.php$ {}
fastcgi_cache shop;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_valid 200 301 10s;
fastcgi_cache_bypass $skip_cache;
fastcgi_no_cache     $skip_cache;
fastcgi_cache_use_stale updating error timeout http_500 http_503;
fastcgi_cache_background_update on;
fastcgi_cache_lock on;
The key
Scheme, method, host and the full address, so two different pages never share an entry.
The bypass
A login or a cart skips the cache both ways: the page is neither served from it nor stored in it. Forms sent with POST are never cached.
Stale while updating
When an entry expires, one request refreshes it in the background while everyone else still gets the previous copy. If PHP fails, the last good page is served.
One rule people miss
An add_header inside a location replaces every header set above it, so we keep the security headers in one snippet and include it wherever headers are added.

Speed, and the rest of a good configuration

The settings that make every site on the server faster, and the habits that keep a change safe.

A change, applied with nothing dropped
$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
$ sudo systemctl reload nginx

The new configuration is tested first; if the test fails, nothing changes. On reload, new workers start with it while the old ones finish the requests they hold, so no visitor is cut off.

log_format timed '$remote_addr "$request" $status $body_bytes_sent '
                 'rt=$request_time urt=$upstream_response_time '
                 'cache=$upstream_cache_status';

Every log line records the total time, the time PHP took and what the cache did, so a slow page is traced to Nginx, PHP or the network instead of guessed at.

http2 on; listen 443 quic;
One connection carries the whole page; HTTP/3 is added where the build and the network allow it.
gzip · brotli
Text is compressed before it leaves, with Brotli where the module is built in. Images and video are not compressed twice.
sendfile · tcp_nopush
The kernel sends files from disk to the network itself, in full packets.
open_file_cache
Nginx remembers which files exist and where they are, instead of asking the disk on every request.
worker_processes auto
One worker per processor core, each holding many connections in a single event loop.
Cache-Control: immutable
Files with a version in their name are cached for a year and never checked again.

What sits behind the front door

Grouped by the job each part does, all of it decided in the Nginx configuration.

Connections

  • Encrypted connectionsTLS 1.3 encrypts the connection, and HTTP/2 carries every file of a page over one connection.
  • Rate and size limitsToo many requests, too many open connections or an oversized body are answered at the door.

Delivery

  • Static filesImages, styles and scripts leave straight from disk with long cache lifetimes.
  • Page cacheA visitor who is not logged in gets a finished page from memory.
  • Load balancingRequests go to the server and the PHP pool that have room for them.

Protection

  • Forbidden pathsHidden files, backups and scripts inside upload folders are refused before PHP hears of them.
  • Security headersKept in one snippet and included wherever headers are added.
  • Walled accountsEach account runs PHP as its own user, inside its own folders.

Changing the configuration without dropping a visitor

Nginx applies a new configuration in place. This is how its own documentation describes the order of events.

  1. Test the file

    The nginx -t command reads the whole configuration and stops at the first mistake, before anything is running on it.

  2. Send the reload signal

    The master process gets the reload signal and checks the new configuration. If the check fails, it rolls back and the old configuration keeps serving.

  3. New workers start

    If the check passes, the master starts new worker processes that use the new configuration.

  4. Old workers finish

    Old workers stop taking new connections and finish the requests they already hold.

  5. Old workers exit

    When the last visitor is served they shut down, and only workers on the new configuration remain.

Nginx documentation: controlling nginx

Bring us the site. We will build its front door.

Tell us where it runs today and what it needs. We read the current setup first, then propose the Nginx and PHP configuration, and quote the work after that conversation.

MickeyAnswers in seconds