Automated Backups With Restic

Preface

Restic is a backup program written in Go, with many features which include encryption and compression. It can create backups locally or on a remote host using a variety of methods.

This post will demonstrate how I backup the data on my server to a Hetzner Storage Box. I will assume readers who wish to do this will set up SSH access on their Hetzner Storage Box without guidance from me. Read the documentation before attempting to implement any of this.

Step 1: Preparation

As with all good things, preparation is essential.

Install Restic and Autorestic

Begin by installing Restic and Autorestic.

SSH

Next ensure the SSH configuration is set up on the host to be backed up.

Host hetzner
    User username
    Port 23
    Hostname username.your-storagebox.de
    IdentityFile ~/.ssh/id_hetzner

Restic password

Create a file called .autorestic.env with the following content:

AUTORESTIC_HETZNER_INFRA_RESTIC_PASSWORD=

To the right of the = symbol, write a long password. This will be used to encrypt the backups.

List of files not to backup

Create a list of directories to exclude from backups.

/boot
/dev
/home
/media
/lost+found
/mnt
/opt
/proc
/run
/srv
/sys
/tmp
/var/cache
/var/lib/postgresql/14/main
/var/lock
/var/log
/var/run
/var/tmp

Autorestic

Next prepare the .autorestic.yml file. Review the official documentation to understand the configuration below. Most of it is self explanatory however I will point out a few things.

backends.hetzner_infra is my named backend.
backends.hetzner_infra.type sets the protocol to use to interact with the remote host.
backends.hetzner_infra.path sets the path to ~/restic on the host hetzner as defined in the SSH configuration.
backends.hetzner_infra.key exactly as you see it will load the corresponding entry from .autorestic.env and use that as the encryption key.

backends:
  hetzner_infra:
    type: sftp
    path: hetzner:restic
    key: "{AUTORESTIC_HETZNER_INFRA_RESTIC_PASSWORD}"
    requirekey: false
    env: {}
    rest:
      user: ""
      password: ""
    options: {}
global:
  forget:
    keep-last: 5 # always keep at least 5 snapshots
    keep-hourly: 24 # keep 24 last hourly snapshots
    keep-daily: 7 # keep 7 last daily snapshots
    keep-weekly: 52 # keep 52 last weekly snapshots
    keep-monthly: 12 # keep 12 last monthly snapshots
    keep-yearly: 7 # keep 7 last yearly snapshots
    keep-within: '14d' # keep snapshots from the last 14 days
locations:
  home:
    from:
    - /home/
    type: ""
    to:
    - hetzner_infra
    hooks:
      dir: ""
      prevalidate: []
      before: []
      after: []
      success: []
      failure: []
    cron: "0 * * * * " # Hourly
    forget: prune
    options:
      backup:
        exclude:
        - /home/daniel/scratch/
        exclude-larger-than:
        - 10G
        one-file-system:
        - true
    # forgetoption: ""
    copyoption: {}
  system:
    from:
    - /
    type: ""
    to:
    - hetzner_infra
    hooks:
      dir: ""
      prevalidate: []
      before: []
      after: []
      success: []
      failure: []
    cron: "0 * * * * " # Hourly
    forget: prune
    options:
      backup:
        exclude-file:
        - /path/to/the/block-list
        exclude-larger-than:
        - 10G
        one-file-system:
        - true
    # forgetoption: ""
    copyoption: {}
version: 2

Step 2: Initialise

Initialize the restic repository on the remote.

restic -r sftp://hetzner/restic init

Now test it:

restic -r sftp://hetzner/restic snapshots

On my machine both .autorestic.yml and .autorestic.env reside under /root so the commands were prepended with sudo.

Step 3: Backup

Create a backup:

autorestic -av backup

Once again, remember to invoke the command as sudo if required.

Step 4: Cron

Now that the components are in place, the automation begins! It's as simple as running a cron job/ timer every few minutes. Autorestric will determine whether or not to create a backup based on the contents of the configuration file and to prune those backups which are deemed obsolete.

SHELL=/bin/bash
# m h  dom mon dow   command
*/5 * * * * autorestic -v -c /root/.autorestic.yml --ci cron

Why a cron job not a systemd timer? In this case it's of little consequence whether we chose one over the other. The simplicity of crontab syntax is one good reason to use it. Strictly speaking it might be wiser to set up a systemd unit/ timer with the dependencies configured e.g. networking.

Email Reports

I have set up email notifications by having cron instead call an intermediate script:

SHELL=/bin/bash
# m h  dom mon dow   command
*/5 * * * * /usr/local/bin/restic-automation

#!/bin/bash

set -o pipefail

Output="$(
/usr/bin/time -p \
        autorestic -v -c /root/.autorestic.yml --ci cron 2>&1
)"
Status="$?"

if (( Status == 0 ))
then
    Subject='autorestic::Ok'
else
    Subject="autorestic::Err::${Status}"
fi

neomutt \
    -s "${Subject}" \
    -- root \
    <<<"${Output}"

exit

With this, I wake up to a lot of emails (12 per hour, 144 over 12 hours) reporting what actions occured during the night. I can run a filter over the emails to selectively delete those emails which effectively say "Nothing happened", leaving the few which either resulted in an error (yet to be seen) or a new backup/ prune.

This is very handy although it makes good sense to migrate to a proper logging/ notification tool in the future. For now, local email remains convenient.

Storage Size

Monitoring the storage used is trivial:

ssh hetzner 'du -cshx restic/*'

9.5K	restic/config
8.6G	restic/data
 20M	restic/index
 65K	restic/keys
 56K	restic/locks
6.7M	restic/snapshots
8.6G	total

All of my backup data consumes 8.6GB!

Roundup

That's about it! Simple really. I wake up, check the backups when it's convenient to do so and I'm done. My backups are minted hourly which might seem like overkill however this system has been in place since three days before the new year and they currently occupy only 8.6GB of storage since very little data changes. I currently have 342 backups of / and 342 backups of /home. Glorious.

Go forth and backup your data!