nginx as a Reverse Proxy for a Node.js App

A developer handed me a Node app that listened on port 3000 and asked me to "just open the port". I'd rather not. Node doesn't do TLS or static files as well as a real web server, and an unprivileged user can't bind port 80 anyway. nginx in front fixes all that and lets one box host several apps on different hostnames.

Here's how I run a Node app as a systemd service on Ubuntu 16.04 and proxy to it, WebSockets and real client IPs included. It assumes nginx is already serving a basic site. I use Node 6 LTS from NodeSource:

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs
node -v

A dedicated user for the app

I never run an app as root or as my own login. Make a system account and put the code where it can read it:

sudo useradd --system --home /srv/myapp --shell /usr/sbin/nologin myapp
sudo mkdir -p /srv/myapp
sudo git clone https://git.example.com/myapp.git /srv/myapp/app
cd /srv/myapp/app && sudo npm install --production
sudo chown -R myapp:myapp /srv/myapp

Make the app read its port from an environment variable (process.env.PORT || 3000) and bind to 127.0.0.1, not 0.0.0.0, so it's only reachable through nginx.

The systemd unit

pm2 and forever are popular, but I stick with systemd because it's already there and handles restarts, logging and boot ordering. Create /etc/systemd/system/myapp.service:

[Unit]
Description=MyApp Node.js service
After=network.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/srv/myapp/app
Environment=NODE_ENV=production
Environment=PORT=3000
ExecStart=/usr/bin/node /srv/myapp/app/server.js
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal
SyslogIdentifier=myapp

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable myapp
sudo systemctl start myapp
sudo systemctl status myapp
curl -I http://127.0.0.1:3000/

Logs go to the journal: journalctl -u myapp -f.

nginx upstream and proxy

Create /etc/nginx/conf.d/myapp.conf (or sites-available/myapp plus a symlink in sites-enabled on the Ubuntu package layout):

upstream myapp_backend {
    server 127.0.0.1:3000;
    keepalive 32;
}

server {
    listen 80;
    server_name app.example.com;

    access_log /var/log/nginx/myapp.access.log;
    error_log  /var/log/nginx/myapp.error.log;

    location /static/ {
        alias /srv/myapp/app/public/;
        expires 7d;
    }

    location / {
        proxy_pass http://myapp_backend;
        proxy_http_version 1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Connection        "";
        proxy_read_timeout 60s;
    }
}

keepalive in the upstream plus proxy_http_version 1.1 lets nginx reuse connections to Node. The X-Forwarded-* headers tell the app the client's real IP and whether the request was HTTPS; in Express, app.set('trust proxy', 'loopback') makes req.ip use them.

WebSockets, if you need them

WebSockets need the Upgrade and Connection headers passed through. Add a location for the socket path (/socket.io/ for socket.io):

    location /socket.io/ {
        proxy_pass http://myapp_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade    $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host       $host;
        proxy_read_timeout 3600s;
    }

The long proxy_read_timeout stops nginx from closing idle sockets after the default 60 seconds. That one produced "the chat drops every minute" tickets before I understood it.

sudo nginx -t && sudo systemctl reload nginx

Trying it

curl -H "Host: app.example.com" -I http://127.0.0.1/
sudo systemctl stop myapp; curl -I http://app.example.com/; sudo systemctl start myapp

The first should return whatever your app sends for /. With the app stopped you want a quick 502 Bad Gateway, not a hang; after starting it the next request should work. Also look in journalctl -u myapp and make sure the app logs the real client IP, not 127.0.0.1.

What usually breaks

• 502 Bad Gateway with connect() failed (111: Connection refused) while connecting to upstream: the app isn't running or is on a different port. ss -tlnp | grep 3000.
• 502 with (13: Permission denied) on CentOS 7: SELinux blocks outbound connections from nginx. setsebool -P httpd_can_network_connect 1.
• Every client looks like 127.0.0.1: trust proxy isn't set in Express, or X-Forwarded-For is missing from the location block.
• systemctl start myapp fails with code=exited, status=203/EXEC: wrong path in ExecStart. On NodeSource it's /usr/bin/node; check which node.
• Uploads over 1 MB fail with 413 Request Entity Too Large: set client_max_body_size 20m; in the location block.

A second app is just another unit file and another conf.d file with its own hostname and port.