From 25bee21253b56d57d1c3f9e5c86d480835505225 Mon Sep 17 00:00:00 2001 From: Marian Beermann Date: Fri, 9 Jun 2017 16:54:55 +0200 Subject: [PATCH] docs: deployment: Automated backups to a local hard drive --- docs/borg_theme/css/borg.css | 7 + docs/deployment.rst | 1 + docs/deployment/automated-local.rst | 231 +++++++++++++++++++++++ docs/deployment/hosting-repositories.rst | 4 +- 4 files changed, 241 insertions(+), 2 deletions(-) create mode 100644 docs/deployment/automated-local.rst diff --git a/docs/borg_theme/css/borg.css b/docs/borg_theme/css/borg.css index f97b59078..c5aa57202 100644 --- a/docs/borg_theme/css/borg.css +++ b/docs/borg_theme/css/borg.css @@ -51,4 +51,11 @@ dt code { /* fancy red-orange stripes */ border-image: repeating-linear-gradient( -45deg,red 0,red 10px,#ffa800 10px,#ffa800 20px,red 20px) 0 20 repeat; + +.topic { + margin: 0 1em; + padding: 0 1em; + /* #4e4a4a = background of the ToC sidebar */ + border-left: 2px solid #4e4a4a;; + border-right: 2px solid #4e4a4a;; } diff --git a/docs/deployment.rst b/docs/deployment.rst index e4fc728a5..7b1caf925 100644 --- a/docs/deployment.rst +++ b/docs/deployment.rst @@ -11,3 +11,4 @@ This chapter details deployment strategies for the following scenarios. deployment/central-backup-server deployment/hosting-repositories + deployment/automated-local diff --git a/docs/deployment/automated-local.rst b/docs/deployment/automated-local.rst new file mode 100644 index 000000000..a64cec235 --- /dev/null +++ b/docs/deployment/automated-local.rst @@ -0,0 +1,231 @@ +.. include:: ../global.rst.inc +.. highlight:: none + +Automated backups to a local hard drive +======================================= + +This guide shows how to automate backups to a hard drive directly connected +to your computer. If a backup hard drive is connected, backups are automatically +started, and the drive shut-down and disconnected when they are done. + +This guide is written for a Linux-based operating system and makes use of +systemd and udev. + +Overview +-------- + +An udev rule is created to trigger on the addition of block devices. The rule contains a tag +that triggers systemd to start a oneshot service. The oneshot service executes a script in +the standard systemd service environment, which automatically captures stdout/stderr and +logs it to the journal. + +The script mounts the added block device, if it is a registered backup drive, and creates +backups on it. When done, it optionally unmounts the file system and spins the drive down, +so that it may be physically disconnected. + +Configuring the system +---------------------- + +First, create the ``/etc/backups`` directory (as root). +All configuration goes into this directory. + +Then, create ``etc/backups/40-backup.rules`` with the following content (all on one line):: + + ACTION=="add", SUBSYSTEM=="bdi", DEVPATH=="/devices/virtual/bdi/*", + TAG+="systemd", ENV{SYSTEMD_WANTS}="automatic-backup.service" + +.. topic:: Finding a more precise udev rule + + If you always connect the drive(s) to the same physical hardware path, e.g. the same + eSATA port, then you can make a more precise udev rule. + + Execute ``udevadm monitor`` and connect a drive to the port you intend to use. + You should see a flurry of events, find those regarding the `block` subsystem. + Pick the event whose device path ends in something similar to a device file name, + typically`sdX/sdXY`. Use the event's device path and replace `sdX/sdXY` after the + `/block/` part in the path with a star (\*). For example: + `DEVPATH=="/devices/pci0000:00/0000:00:11.0/ata3/host2/target2:0:0/2:0:0:0/block/*"`. + + Reboot a few times to ensure that the hardware path does not change: on some motherboards + components of it can be random. In these cases you cannot use a more accurate rule, + or need to insert additional stars for matching the path. + +The "systemd" tag in conjunction with the SYSTEMD_WANTS environment variable has systemd +launch the "automatic-backup" service, which we will create next, as the +``/etc/backups/automatic-backup.service`` file: + +.. code-block:: ini + + [Service] + Type=oneshot + ExecStart=/etc/backups/run.sh + +Now, create the main backup script, ``/etc/backups/run.sh``. Below is a template, +modify it to suit your needs (e.g. more backup sets, dumping databases etc.). + +.. code-block:: bash + + #!/bin/bash -ue + + # The udev rule is not terribly accurate and may trigger our service before + # the kernel has finished probing partitions. Sleep for a bit to ensure + # the kernel is done. + # + # This can be avoided by using a more precise udev rule, e.g. matching + # a specific hardware path and partition. + sleep 5 + + # + # Script configuration + # + + # The backup partition is mounted there + MOUNTPOINT=/mnt/backup + + # This is the location of the Borg repository + TARGET=$MOUNTPOINT/borg-backups/backup.borg + + # Archive name schema + DATE=$(date --iso-8601)-$(hostname) + + # This is the file that will later contain UUIDs of registered backup drives + DISKS=/etc/backups/backup.disks + + # Find whether the connected block device is a backup drive + for uuid in $(lsblk --noheadings --list --output uuid) + do + if grep --quiet --fixed-strings $uuid $DISKS; then + break + fi + uuid= + done + + if [ ! $uuid ]; then + echo "No backup disk found, exiting" + exit 0 + fi + + echo "Disk $uuid is a backup disk" + partition_path=/dev/disk/by-uuid/$uuid + # Mount file system if not already done. This assumes that if something is already + # mounted at $MOUNTPOINT, it is the backup drive. It won't find the drive if + # it was mounted somewhere else. + (mount | grep $MOUNTPOINT) || mount $partition_path $MOUNTPOINT + drive=$(lsblk --inverse --noheadings --list --paths --output name $partition_path | head --lines 1) + echo "Drive path: $drive" + + # + # Create backups + # + + # Options for borg create + BORG_OPTS="--stats --one-file-system --compression lz4 --checkpoint-interval 86400" + + # Set BORG_PASSPHRASE or BORG_PASSCOMMAND somewhere around here, using export, + # if encryption is used. + + # No one can answer if Borg asks these questions, it is better to just fail quickly + # instead of hanging. + export BORG_RELOCATED_REPO_ACCESS_IS_OK=no + export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no + + # Log Borg version + borg --version + + echo "Starting backup for $DATE" + + # This is just an example, change it however you see fit + borg create $BORG_OPTS \ + --exclude /root/.cache \ + --exclude /var/cache \ + --exclude /var/lib/docker/devicemapper \ + $TARGET::$DATE-$$-system \ + / /boot + + # /home is often a separate partition / file system. + # Even if it isn't (add --exclude /home above), it probably makes sense + # to have /home in a separate archive. + borg create $BORG_OPTS \ + --exclude 'sh:/home/*/.cache' \ + $TARGET::$DATE-$$-home \ + /home/ + + echo "Completed backup for $DATE" + + # Just to be completely paranoid + sync + + if [ -f /etc/backups/autoeject ]; then + umount $MOUNTPOINT + hdparm -Y $drive + fi + + if [ -f /etc/backups/backup-suspend ]; then + systemctl suspend + fi + +Create the ``/etc/backups/autoeject`` file to have the script automatically eject the drive +after creating the backup. Rename the file to something else (e.g. ``/etc/backup/autoeject-no``) +when you want to do something with the drive after creating backups (e.g running check). + +Create the ``/etc/backups/backup-suspend`` file if the machine should suspend after completing +the backup. Don't forget to physically disconnect the device before resuming, +otherwise you'll enter a cycle. You can also add an option to power down instead. + +Create an empty ``/etc/backups/backup.disks`` file, you'll register your backup drives +there. + +The last part is to actually enable the udev rules and services: + +.. code-block:: bash + + ln -s /etc/backups/40-backup.rules /etc/udev/rules.d/40-backup.rules + ln -s /etc/backups/automatic-backup.service /etc/systemd/system/automatic-backup.service + systemctl daemon-reload + udevadm control --reload + +Adding backup hard drives +------------------------- + +Connect your backup hard drive. Format it, if not done already. +Find the UUID of the file system that backups should be stored on:: + + lsblk -o+uuid,label + +Note the UUID into the ``/etc/backup/backup.disks`` file. + +Mount the drive to /mnt/backup. + +Initialize a Borg repository at the location indicated by ``TARGET``:: + + borg init --encryption ... /mnt/backup/borg-backups/backup.borg + +Unmount and reconnect the drive, or manually start the ``automatic-backup`` service +to start the first backup:: + + systemctl start --no-block automatic-backup + +See backup logs using journalctl:: + + journalctl -fu automatic-backup [-n number-of-lines] + +Security considerations +----------------------- + +The script as shown above will mount any file system with an UUID listed in +``/etc/backup/backup.disks``. The UUID check is a safety / annoyance-reduction +mechanism to keep the script from blowing up whenever a random USB thumb drive is connected. +It is not meant as a security mechanism. Mounting file systems and reading repository +data exposes additional attack surfaces (kernel file system drivers, +possibly user space services and Borg itself). On the other hand, someone +standing right next to your computer can attempt a lot of attacks, most of which +are easier to do than e.g. exploiting file systems (installing a physical key logger, +DMA attacks, stealing the machine, ...). + +Borg ensures that backups are not created on random drives that "just happen" +to contain a Borg repository. If an unknown unencrypted repository is encountered, +then the script aborts (BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no). + +Backups are only created on hard drives that contain a Borg repository that is +either known (by ID) to your machine or you are using encryption and the +passphrase of the repository has to match the passphrase supplied to Borg. diff --git a/docs/deployment/hosting-repositories.rst b/docs/deployment/hosting-repositories.rst index e502d644f..ae9a4e1ee 100644 --- a/docs/deployment/hosting-repositories.rst +++ b/docs/deployment/hosting-repositories.rst @@ -55,7 +55,7 @@ multiple times to permit access to more than one repository. The repository may not exist yet; it can be initialized by the user, which allows for encryption. -Storage quotas can be enabled by adding the ``--storage-quota`` option +**Storage quotas** can be enabled by adding the ``--storage-quota`` option to the ``borg serve`` command line:: restrict,command="borg serve --storage-quota 20G ..." ... @@ -70,4 +70,4 @@ support storage quotas. Refer to :ref:`internals_storage_quota` for more details on storage quotas. Refer to the `sshd(8) `_ -for more details on securing SSH. +man page for more details on SSH options.