VPS snapshots and off-site backups

Snapshots let you roll an Offshore VPS back quickly. Backups let you recover when the server, or your access to it, is gone. You want both. This guide explains the difference, the snapshot limits per plan, how to take snapshots that databases survive, and how to set up encrypted off-site backups.

Snapshots vs backups

A snapshot is a point-in-time copy of your VPS's whole disk, kept on our platform together with the VPS. Restoring it puts the entire server back in exactly that state, which makes it the fastest undo after a failed upgrade or a configuration mistake.

A backup is a copy of your data stored somewhere else: another server, another location, your own hardware. It is independent of the VPS, can keep many versions, and lets you restore single files. The usual rule is 3-2-1: three copies of your data, on two kinds of storage, one of them off-site.

Snapshots per plan

When all slots are in use, say in your request which snapshot to replace.

Create, restore and delete snapshots

Create

On your server's page in the client area, under Server actions, choose Take a snapshot and give it a name that says why it exists, such as before-php-upgrade. Our team takes it and the request shows as done on the same page: wait for that before a risky change such as a distribution upgrade, a new kernel or a database migration. If the server runs a database, prepare it first as described in the next section.

Restore

Under Server actions, choose Restore a snapshot and name the snapshot. The VPS comes back in the state of the snapshot, and the request shows as done once it is back online. Check that your services started and that scheduled jobs and certificates still look right.

Delete

Ask for it with Delete a snapshot under Server actions, naming the snapshot. Deleting a snapshot is permanent. Keep at least one known-good snapshot, for example the one taken after the initial setup, and delete pre-change snapshots once the change has run without problems for a few days.

Take consistent snapshots

A snapshot of a running server is crash-consistent: the disk looks as if the power had been cut at that instant. Journaling filesystems handle that well. Databases usually recover too, but they may lose their most recent transactions, and a table caught in the middle of a write can end up damaged. Use one of these two methods before you request a snapshot of a server that runs a database.

Stop the database for a moment

The simplest option. Stop the service (it is called postgresql for PostgreSQL):

systemctl stop mariadb

Request the snapshot, and start the service again once the request shows as done (the database stays down until then):

systemctl start mariadb

Dump the databases first

The better option when the database must stay up: a dump taken just before you request the snapshot is consistent even if the live database files are not, and you can reuse it for off-site backups. Write nothing important between the dump and the moment the request shows as done. For MariaDB (on MySQL, the command is mysqldump):

mkdir -p /root/db-dumps
chmod 700 /root/db-dumps
mariadb-dump --all-databases --single-transaction --routines --events > /root/db-dumps/mariadb.sql

For PostgreSQL:

runuser -u postgres -- pg_dumpall > /root/db-dumps/postgresql.sql

Off-site backups with restic or Borg

restic and Borg both encrypt on your server before anything leaves it, deduplicate data and keep many versions cheaply. The destination only ever sees encrypted data, so it can be any machine you reach over SSH: a second VPS, a storage server in another of our locations, or a machine at home. Back up /etc, /home, /root, your web roots and application data, plus database dumps rather than live database files. Adjust the paths in the examples to your server.

restic

On the VPS, install restic, create an SSH key for root and copy it to the backup host. Then generate a random repository password, initialize the repository and run the first backup:

apt install -y restic
ssh-keygen -t ed25519 -N "" -f /root/.ssh/id_ed25519
ssh-copy-id backup@BACKUP_HOST
install -m 600 /dev/null /root/.restic-pass
openssl rand -base64 32 > /root/.restic-pass
export RESTIC_REPOSITORY=sftp:backup@BACKUP_HOST:/srv/restic/web1
export RESTIC_PASSWORD_FILE=/root/.restic-pass
restic init
restic backup /etc /home /root /var/www --exclude-caches
restic snapshots

Borg

Borg must also be installed on the backup host, because it runs there over SSH. Set up the SSH key as in the restic example. These commands are for Borg 1.2 and 1.4, the versions packaged in current Debian and Ubuntu releases:

apt install -y borgbackup
install -m 600 /dev/null /root/.borg-pass
openssl rand -base64 32 > /root/.borg-pass
export BORG_REPO=ssh://backup@BACKUP_HOST/./borg/web1
export BORG_PASSCOMMAND="cat /root/.borg-pass"
borg init --encryption=repokey-blake2
borg create --stats --compression zstd '::{hostname}-{now}' /etc /home /root /var/www
borg list
borg key export "$BORG_REPO" /root/borg-key.txt

Store borg-key.txt and the passphrase somewhere safe off the server, then delete the exported file from the VPS.

Automate, prune and test

Put the two export lines, your database dump commands and the backup command in a script such as /usr/local/sbin/offsite-backup, and make it readable by root only:

chmod 700 /usr/local/sbin/offsite-backup
crontab -e

In root's crontab, add a line to run it every night at 03:30:

30 3 * * * /usr/local/sbin/offsite-backup

End the script with a retention rule so old versions are removed. These keep a week of daily, a month of weekly and six months of monthly backups. For restic:

restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

For Borg:

borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6
borg compact

An untested backup may not restore when you need it. Once a month, check the repository and restore into a scratch directory. With restic:

restic check
restic restore latest --target /root/restore-test

With Borg, list the archives, then extract one into an empty directory:

borg check
mkdir -p /root/restore-test
cd /root/restore-test
borg extract '::ARCHIVE_NAME' etc

A server that pushes its own backups can also delete them if an attacker gains root. To prevent that, make the repository append-only on the backup host: restic's rest-server has an --append-only mode, and Borg can be limited with borg serve --append-only in the backup user's authorized_keys. The hardening guide helps keep attackers out in the first place.