DB-유

InfluxDB and Grafana for Synology NAS Metrics

· synology, grafana, monitoring

I wanted to know whether my disks got warmer after I moved the NAS into a cupboard, and Resource Monitor couldn't tell me. It shows now and a few days back, not how volume usage trended over a year or what a nightly backup does to the network. For that you want a time series database and a dashboard. Telegraf, InfluxDB and Grafana run happily as three containers on the very NAS they monitor.

Below: SNMP on DSM 6.2, the three containers via docker-compose, and a first dashboard from the Synology MIB. Use a model with at least 2 GB of RAM (the trio takes about 400 MB), and be comfortable with compose files and volumes.

SNMP first

Telegraf reads the NAS metrics over SNMP, which has the advantage of not requiring anything privileged inside the containers. Control Panel → Terminal & SNMP → SNMP:

Enable SNMP service:       on
SNMPv1, SNMPv2c service:   on, community: nasmetrics (not "public")
SNMPv3 service:            off (v2c is enough on a LAN)
Enable SNMP privacy:       n/a

Then in the DSM firewall allow UDP 161 from the Docker bridge subnet 172.16.0.0/12, or simply from the LAN if the firewall already allows the LAN. Test from the NAS itself after installing the container in the next step, or from a Linux box with snmpwalk -v2c -c nasmetrics 10.20.30.5 1.3.6.1.4.1.6574.1.

Download the Synology MIB files from the Synology website (the "Synology MIB Guide" download) and unzip them to /volume1/docker/monitoring/mibs/.

The stack

sudo mkdir -p /volume1/docker/monitoring/{influxdb,grafana,telegraf,mibs}
sudo chown 472:472 /volume1/docker/monitoring/grafana

/volume1/docker/monitoring/docker-compose.yml:

version: '2'
services:
  influxdb:
    image: influxdb:1.8
    container_name: influxdb
    restart: unless-stopped
    ports:
      - "8086:8086"
    environment:
      INFLUXDB_DB: telegraf
      INFLUXDB_HTTP_AUTH_ENABLED: "true"
      INFLUXDB_ADMIN_USER: admin
      INFLUXDB_ADMIN_PASSWORD: change-me
      INFLUXDB_USER: telegraf
      INFLUXDB_USER_PASSWORD: telegraf-pass
    volumes:
      - /volume1/docker/monitoring/influxdb:/var/lib/influxdb

  telegraf:
    image: telegraf:1.15
    container_name: telegraf
    restart: unless-stopped
    depends_on:
      - influxdb
    volumes:
      - /volume1/docker/monitoring/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
      - /volume1/docker/monitoring/mibs:/usr/share/snmp/mibs:ro

  grafana:
    image: grafana/grafana:7.1.5
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    depends_on:
      - influxdb
    environment:
      GF_SECURITY_ADMIN_PASSWORD: change-me-too
      GF_SERVER_ROOT_URL: https://grafana.example.com
    volumes:
      - /volume1/docker/monitoring/grafana:/var/lib/grafana

I stick to InfluxDB 1.8 rather than 2.0: Grafana's InfluxQL data source and the Telegraf output are simplest against 1.x, and I already know how its retention policies work. Grafana runs as UID 472, hence the chown.

Telegraf

/volume1/docker/monitoring/telegraf/telegraf.conf:

[agent]
  interval = "30s"
  flush_interval = "30s"

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  database = "telegraf"
  username = "telegraf"
  password = "telegraf-pass"

[[inputs.snmp]]
  agents = ["10.20.30.5:161"]
  version = 2
  community = "nasmetrics"
  name = "synology"

  [[inputs.snmp.field]]
    name = "hostname"
    oid = "RFC1213-MIB::sysName.0"
    is_tag = true
  [[inputs.snmp.field]]
    name = "uptime"
    oid = "DISMAN-EVENT-MIB::sysUpTimeInstance"
  [[inputs.snmp.field]]
    name = "system_status"
    oid = "SYNOLOGY-SYSTEM-MIB::systemStatus.0"
  [[inputs.snmp.field]]
    name = "temperature"
    oid = "SYNOLOGY-SYSTEM-MIB::temperature.0"
  [[inputs.snmp.field]]
    name = "cpu_idle"
    oid = "UCD-SNMP-MIB::ssCpuIdle.0"
  [[inputs.snmp.field]]
    name = "mem_total"
    oid = "UCD-SNMP-MIB::memTotalReal.0"
  [[inputs.snmp.field]]
    name = "mem_avail"
    oid = "UCD-SNMP-MIB::memAvailReal.0"

  [[inputs.snmp.table]]
    name = "synology_disk"
    oid = "SYNOLOGY-DISK-MIB::diskTable"
    [[inputs.snmp.table.field]]
      name = "disk_id"
      oid = "SYNOLOGY-DISK-MIB::diskID"
      is_tag = true

  [[inputs.snmp.table]]
    name = "synology_volume"
    oid = "HOST-RESOURCES-MIB::hrStorageTable"
    [[inputs.snmp.table.field]]
      name = "descr"
      oid = "HOST-RESOURCES-MIB::hrStorageDescr"
      is_tag = true

  [[inputs.snmp.table]]
    name = "synology_net"
    oid = "IF-MIB::ifXTable"
    [[inputs.snmp.table.field]]
      name = "ifName"
      oid = "IF-MIB::ifName"
      is_tag = true

The disk table includes temperature, status and model per drive; hrStorageTable gives used and total blocks per volume; ifXTable gives 64-bit byte counters per interface.

Start everything:

cd /volume1/docker/monitoring
sudo docker-compose up -d
sudo docker logs telegraf

Errors about unknown MIB names mean the MIB folder is empty or the file names inside differ from the module names; the Synology zip has SYNOLOGY-SYSTEM-MIB.txt and friends, which is what snmptranslate expects.

Grafana data source and dashboard

Open http://nas:3000, log in as admin, and go to Configuration → Data Sources → Add → InfluxDB:

URL:        http://influxdb:8086
Database:   telegraf
User:       telegraf
Password:   telegraf-pass

Save & Test should report the data source is working. Then create a dashboard with a few panels; the query editor's visual mode makes this straightforward:

  • Volume usage: measurement synology_volume, field hrStorageUsed, math * hrStorageAllocationUnits, group by tag descr, filtered to descr =~ /volume/.
  • Disk temperature: measurement synology_disk, field diskTemperature, group by disk_id.
  • CPU: measurement synology, field cpu_idle, transform 100 - value.
  • Network: measurement synology_net, field ifHCInOctets with derivative(1s) * 8 for bits per second, group by ifName.

Grafana's dashboard import also accepts community Synology SNMP dashboards by ID; they expect the measurement names above in most cases.

Retention

Thirty-second samples forever is overkill. Cap it in InfluxDB:

CREATE RETENTION POLICY "one_year" ON "telegraf" DURATION 365d REPLICATION 1 DEFAULT

run via sudo docker exec -it influxdb influx -username admin -password change-me. A year of NAS metrics at this rate is a few gigabytes.

Is data arriving?

sudo docker exec influxdb influx -username admin -password change-me -database telegraf -execute 'SELECT last("temperature") FROM "synology"'

should return the current chassis temperature. In Grafana the panels fill within a couple of minutes. Compare the volume usage panel against Storage Manager to confirm the unit math is right.

What went wrong on my first try

  • Telegraf logs snmp: Timeout, no response from 10.20.30.5. The DSM firewall is blocking 161 from the Docker bridge, or the community string differs. Test from the NAS shell with snmpget -v2c -c nasmetrics 127.0.0.1 sysUpTime.0.
  • E! [inputs.snmp] Error in plugin: initializing table synology_disk: translating: MIB search path: ... Cannot find module (SYNOLOGY-DISK-MIB). The MIB volume is not mounted or the files are missing. ls /volume1/docker/monitoring/mibs on the NAS and docker exec telegraf ls /usr/share/snmp/mibs should match.
  • Grafana shows "InfluxDB returned error: authorization failed". The user or password in the data source does not match INFLUXDB_USER, or auth was enabled after the database was created. Recreate the user with CREATE USER in the influx shell.
  • Grafana container exits with GF_PATHS_DATA='/var/lib/grafana' is not writable. The chown 472:472 step was skipped.

It pays off slowly. The first time you can point at a graph and say the volume fills up in four months, the afternoon of Telegraf fiddling is repaid.