DB-유

Redis Installation and Basic Tuning

· linux, redis, cache, performance

Redis is in almost every stack I look after: sessions, a cache in front of the database, a job queue. It's fast and easy to run, but the defaults are meant for a developer's laptop. It ships listening without a password, with no memory limit, and with kernel warnings in the log that everyone ignores until the box runs out of memory. I've been that person.

Here's how I install Redis 5 on CentOS 7 from Remi (EPEL still has 3.2) and Redis 4.0 on Ubuntu 18.04, plus the handful of settings that matter on a real server. Decide up front how much memory you're willing to give it.

Installing

CentOS 7, from Remi:

sudo yum -y install epel-release
sudo yum -y install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
sudo yum -y --enablerepo=remi install redis
redis-server --version

Ubuntu 18.04:

sudo apt-get install -y redis-server
redis-server --version

Don't start it yet if you can avoid it; fix the config first. It's /etc/redis.conf on CentOS and /etc/redis/redis.conf on Ubuntu.

Network first

Redis has no encryption and, by default, no authentication. Only the application servers should ever reach it.

bind 127.0.0.1
protected-mode yes
port 6379
requirepass a-long-random-password
rename-command FLUSHALL ""
rename-command CONFIG "CONFIG_a8f3e"

If application servers are on another host, bind to the private interface address as well (bind 127.0.0.1 192.168.10.20) and restrict the port with the firewall to those hosts:

sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=192.168.10.30 port port=6379 protocol=tcp accept'
sudo firewall-cmd --reload

Generate the password with openssl rand -base64 32. Redis can check tens of thousands of guesses a second, so a short one is useless. Renaming or disabling dangerous commands protects against a compromised app wiping the dataset.

Memory limit and eviction

Without maxmemory, Redis grows until the kernel kills it. Decide how much of the machine is Redis's and what happens when it fills up:

maxmemory 2gb
maxmemory-policy allkeys-lru

Policies worth knowing: allkeys-lru (evict least recently used, good for a pure cache), volatile-lru (only evict keys that have a TTL, good when some keys must never disappear), and noeviction (return errors when full, correct for a queue where losing data is worse than failing). Leave headroom for the fork during snapshots; maxmemory should be well under total RAM.

Persistence

Default RDB snapshotting (save 900 1, save 300 10, save 60 10000) writes the whole dataset to dump.rdb periodically. For a pure cache I turn it off so a restart comes up empty and fast:

save ""

For sessions or queues where losing the last minute of writes matters, enable the append-only file instead:

appendonly yes
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

dir /var/lib/redis is the default on both distributions; make sure that partition has at least twice maxmemory free for rewrites.

Kernel warnings

Start Redis and read its log. On an untuned host you'll see three warnings:

sudo systemctl enable --now redis          # redis-server on Ubuntu
sudo tail -n 20 /var/log/redis/redis.log   # /var/log/redis/redis-server.log on Ubuntu

Memory overcommit. Redis forks to save snapshots; without overcommit the fork can fail on a large dataset. Create /etc/sysctl.d/99-redis.conf:

vm.overcommit_memory = 1
net.core.somaxconn = 1024

sudo sysctl -p /etc/sysctl.d/99-redis.conf

Transparent huge pages. THP causes latency spikes during forks. Disable it at boot with a small unit, /etc/systemd/system/disable-thp.service:

[Unit]
Description=Disable Transparent Huge Pages
Before=redis.service

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now disable-thp

somaxconn. tcp-backlog 511 in redis.conf is capped by the kernel's net.core.somaxconn (128 by default on CentOS 7). The sysctl file above handles it.

Restart Redis and confirm the warnings are gone.

Poking at it

redis-cli -a a-long-random-password ping
redis-cli -a a-long-random-password INFO memory | grep -E 'used_memory_human|maxmemory_human|maxmemory_policy'
redis-cli -a a-long-random-password INFO persistence | grep -E 'aof_enabled|rdb_last_bgsave_status'
redis-cli --latency -a a-long-random-password

From another host, redis-cli -h 192.168.10.20 ping should time out or be refused unless that host is an allowed application server. redis-benchmark -q -n 100000 -a a-long-random-password is a quick sanity check that throughput is in the right range for the hardware.

Error messages

  • (error) NOAUTH Authentication required.: the client didn't send the password. Apps need it in the connection string; with redis-cli use -a or AUTH.
  • MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk: dir isn't writable or the disk is full. Redis refuses writes until fixed; check ls -ld /var/lib/redis and df -h. config set stop-writes-on-bgsave-error no is a temporary bypass, nothing more.
  • Can't save in background: fork: Cannot allocate memory: overcommit is still 0, or the dataset is close to RAM size. Apply the sysctl and lower maxmemory.
  • WARNING: The TCP backlog setting of 511 cannot be enforced after the sysctl change: Redis reads it at startup, so restart the service.
  • OOM command not allowed when used memory > 'maxmemory': policy is noeviction and the limit is hit. Either that's intended (a full queue) or you wanted allkeys-lru.

Afterwards I graph used_memory and evicted_keys from INFO, so I see an undersized limit before users do.