From f66f6caecd5f2d2a60137358aecc09596e0af4f8 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Mon, 26 Jun 2017 22:32:49 +0200 Subject: [PATCH 1/2] update CHANGES (1.0-maint) --- docs/changes.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/changes.rst b/docs/changes.rst index dd23f70f8..19c98a955 100644 --- a/docs/changes.rst +++ b/docs/changes.rst @@ -128,8 +128,8 @@ The best check that everything is ok is to run a dry-run extraction:: Changelog ========= -Version 1.0.11rc1 (not released yet) ------------------------------------- +Version 1.0.11rc1 (2017-06-27) +------------------------------ Bug fixes: @@ -229,7 +229,9 @@ Other changes: to break was fixed in cygwin at least since cygwin 2.8, maybe even since 2.7.0). - remove skipping the noatime tests on GNU/Hurd, #2710 - + - fix borg import issue, add comment, #2718 + - include attic.tar.gz when installing the package + also: add include_package_data=True Version 1.0.10 (2017-02-13) --------------------------- From b36b636a9e7bb6067dba08ba8753074a8108c442 Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Mon, 26 Jun 2017 22:39:02 +0200 Subject: [PATCH 2/2] build_usage --- docs/usage/break-lock.rst.inc | 3 ++- docs/usage/change-passphrase.rst.inc | 3 ++- docs/usage/check.rst.inc | 3 ++- docs/usage/create.rst.inc | 23 +++++++++++++++++------ docs/usage/delete.rst.inc | 9 ++++++--- docs/usage/extract.rst.inc | 6 ++++-- docs/usage/help.rst.inc | 9 +++++++++ docs/usage/info.rst.inc | 3 ++- docs/usage/init.rst.inc | 3 ++- docs/usage/key_export.rst.inc | 3 ++- docs/usage/key_import.rst.inc | 3 ++- docs/usage/list.rst.inc | 3 ++- docs/usage/migrate-to-repokey.rst.inc | 3 ++- docs/usage/mount.rst.inc | 3 ++- docs/usage/prune.rst.inc | 14 +++++++++----- docs/usage/rename.rst.inc | 3 ++- docs/usage/serve.rst.inc | 3 ++- docs/usage/umount.rst.inc | 3 ++- docs/usage/upgrade.rst.inc | 27 +++++++++++++++++++++++---- 19 files changed, 94 insertions(+), 33 deletions(-) diff --git a/docs/usage/break-lock.rst.inc b/docs/usage/break-lock.rst.inc index b3ba522a7..8c4f6fb1e 100644 --- a/docs/usage/break-lock.rst.inc +++ b/docs/usage/break-lock.rst.inc @@ -29,7 +29,8 @@ borg break-lock --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/change-passphrase.rst.inc b/docs/usage/change-passphrase.rst.inc index ac0019285..823401d60 100644 --- a/docs/usage/change-passphrase.rst.inc +++ b/docs/usage/change-passphrase.rst.inc @@ -30,7 +30,8 @@ borg change-passphrase --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/check.rst.inc b/docs/usage/check.rst.inc index e8e1f9fe9..dc8353489 100644 --- a/docs/usage/check.rst.inc +++ b/docs/usage/check.rst.inc @@ -31,7 +31,8 @@ borg check --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") --repository-only only perform repository checks --archives-only only perform archives checks --repair attempt to repair any inconsistencies found diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index 6237eb9b7..1db3a1e64 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -38,13 +38,16 @@ borg create --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") - -s, --stats print statistics for the created archive + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") + -s, --stats print statistics for the created archive. Requires + -v/--verbose. -p, --progress show progress display while creating the archive, showing Original, Compressed and Deduplicated sizes, followed by the Number of files seen and the path being processed, default: False - --list output verbose list of items (files, dirs, ...) + --list output verbose list of items (files, dirs, ...). + Requires -v/--verbose. --filter STATUSCHARS only display items with the given status characters -e PATTERN, --exclude PATTERN exclude paths matching PATTERN @@ -84,9 +87,12 @@ Description ~~~~~~~~~~~ This command creates a backup archive containing all files found while recursively -traversing all paths specified. When giving '-' as path, borg will read data -from standard input and create a file 'stdin' in the created archive from that -data. +traversing all paths specified. Paths are added to the archive as they are given, +that means if relative paths are desired, the command has to be run from the correct +directory. + +When giving '-' as path, borg will read data from standard input and create a +file 'stdin' in the created archive from that data. The archive will consume almost no disk space for files or parts of files that have already been stored in other archives. @@ -103,5 +109,10 @@ not provide correct inode information the --ignore-inode flag can be used. This potentially decreases reliability of change detection, while avoiding always reading all files on these file systems. +The mount points of filesystems or filesystem snapshots should be the same for every +creation of a new archive to ensure fast operation. This is because the file cache that +is used to determine changed files quickly uses absolute filenames. +If this is not possible, consider creating a bind mount to a stable location. + See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. diff --git a/docs/usage/delete.rst.inc b/docs/usage/delete.rst.inc index 64c85bdb9..f36fed1ef 100644 --- a/docs/usage/delete.rst.inc +++ b/docs/usage/delete.rst.inc @@ -30,11 +30,14 @@ borg delete --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") -p, --progress show progress display while deleting a single archive - -s, --stats print statistics for the deleted archive + -s, --stats print statistics for the deleted archive. Requires + -v/--verbose. -c, --cache-only delete only the local cache for the given repository - --force force deletion of corrupted archives + --force force deletion of corrupted archives, use --force + --force in case --force does not work. --save-space work slower, but using less space Description diff --git a/docs/usage/extract.rst.inc b/docs/usage/extract.rst.inc index 9cf2d4f9c..ff17469b4 100644 --- a/docs/usage/extract.rst.inc +++ b/docs/usage/extract.rst.inc @@ -32,8 +32,10 @@ borg extract --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") - --list output verbose list of items (files, dirs, ...) + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") + --list output verbose list of items (files, dirs, ...). + Requires -v/--verbose. -n, --dry-run do not actually change any files -e PATTERN, --exclude PATTERN exclude paths matching PATTERN diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index b1025fb5b..d6b522641 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -52,6 +52,15 @@ Prefix path, selector `pp:` This pattern style is useful to match whole sub-directories. The pattern `pp:/data/bar` matches `/data/bar` and everything therein. +.. note:: + + `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE + engine. It is very easy to formulate patterns for each of these types which + requires an inordinate amount of time to match paths. If untrusted users + are able to supply patterns, ensure they cannot supply `re:` patterns. + Further, ensure that `sh:` and `fm:` patterns only contain a handful of + wildcards at most. + Exclusions can be passed via the command line option `--exclude`. When used from within a shell the patterns should be quoted to protect them from expansion. diff --git a/docs/usage/info.rst.inc b/docs/usage/info.rst.inc index cd745b531..f88133d3e 100644 --- a/docs/usage/info.rst.inc +++ b/docs/usage/info.rst.inc @@ -29,7 +29,8 @@ borg info --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/init.rst.inc b/docs/usage/init.rst.inc index f91021415..47dc36fc9 100644 --- a/docs/usage/init.rst.inc +++ b/docs/usage/init.rst.inc @@ -29,7 +29,8 @@ borg init --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") -e {none,keyfile,repokey}, --encryption {none,keyfile,repokey} select encryption key mode (default: "repokey") -a, --append-only create an append-only mode repository diff --git a/docs/usage/key_export.rst.inc b/docs/usage/key_export.rst.inc index 389fe4ea2..ea411f199 100644 --- a/docs/usage/key_export.rst.inc +++ b/docs/usage/key_export.rst.inc @@ -31,7 +31,8 @@ borg key export --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") --paper Create an export suitable for printing and later type- in --qr-html Create an html file suitable for printing and later diff --git a/docs/usage/key_import.rst.inc b/docs/usage/key_import.rst.inc index 71a8eed46..4310afbe4 100644 --- a/docs/usage/key_import.rst.inc +++ b/docs/usage/key_import.rst.inc @@ -31,7 +31,8 @@ borg key import --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") --paper interactively import from a backup done with --paper Description diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc index 8f1d2a45e..bfdcb3e68 100644 --- a/docs/usage/list.rst.inc +++ b/docs/usage/list.rst.inc @@ -31,7 +31,8 @@ borg list --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") --short only print file/directory names, nothing else --list-format LISTFORMAT specify format for archive file listing (default: diff --git a/docs/usage/migrate-to-repokey.rst.inc b/docs/usage/migrate-to-repokey.rst.inc index c5ff21be3..2827d8d63 100644 --- a/docs/usage/migrate-to-repokey.rst.inc +++ b/docs/usage/migrate-to-repokey.rst.inc @@ -30,7 +30,8 @@ borg migrate-to-repokey --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/mount.rst.inc b/docs/usage/mount.rst.inc index 898d9c5b4..fb1067c34 100644 --- a/docs/usage/mount.rst.inc +++ b/docs/usage/mount.rst.inc @@ -31,7 +31,8 @@ borg mount --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") -f, --foreground stay in foreground, do not daemonize -o OPTIONS Extra mount options diff --git a/docs/usage/prune.rst.inc b/docs/usage/prune.rst.inc index f8247c95f..653f47a9c 100644 --- a/docs/usage/prune.rst.inc +++ b/docs/usage/prune.rst.inc @@ -9,7 +9,7 @@ borg prune usage: borg prune [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-n] [--force] [-s] [--list] - [--keep-within WITHIN] [-H HOURLY] [-d DAILY] [-w WEEKLY] + [--keep-within INTERVAL] [-H HOURLY] [-d DAILY] [-w WEEKLY] [-m MONTHLY] [-y YEARLY] [-P PREFIX] [--save-space] [REPOSITORY] @@ -31,12 +31,16 @@ borg prune --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") -n, --dry-run do not change repository --force force pruning of corrupted archives - -s, --stats print statistics for the deleted archive - --list output verbose list of archives it keeps/prunes - --keep-within WITHIN keep all archives within this time interval + -s, --stats print statistics for the deleted archive. Requires + -v/--verbose. + --list output verbose list of archives it keeps/prunes. + Requires -v/--verbose. + --keep-within INTERVAL + keep all archives within this time interval -H HOURLY, --keep-hourly HOURLY number of hourly archives to keep -d DAILY, --keep-daily DAILY diff --git a/docs/usage/rename.rst.inc b/docs/usage/rename.rst.inc index cfafb0022..9df1518c8 100644 --- a/docs/usage/rename.rst.inc +++ b/docs/usage/rename.rst.inc @@ -30,7 +30,8 @@ borg rename --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/serve.rst.inc b/docs/usage/serve.rst.inc index 48f97074c..fe05805ef 100644 --- a/docs/usage/serve.rst.inc +++ b/docs/usage/serve.rst.inc @@ -27,7 +27,8 @@ borg serve --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") --restrict-to-path PATH restrict repository access to PATH. Can be specified multiple times to allow the client access to several diff --git a/docs/usage/umount.rst.inc b/docs/usage/umount.rst.inc index febacda06..b9019e336 100644 --- a/docs/usage/umount.rst.inc +++ b/docs/usage/umount.rst.inc @@ -29,7 +29,8 @@ borg umount --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") Description ~~~~~~~~~~~ diff --git a/docs/usage/upgrade.rst.inc b/docs/usage/upgrade.rst.inc index b566cf118..1d5df05c3 100644 --- a/docs/usage/upgrade.rst.inc +++ b/docs/usage/upgrade.rst.inc @@ -30,7 +30,8 @@ borg upgrade --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) - --remote-path PATH set remote path to executable (default: "borg") + --remote-path PATH use PATH as borg executable on the remote (default: + "borg") -p, --progress show progress display while upgrading the repository -n, --dry-run do not change repository -i, --inplace rewrite repository in place, with no chance of going @@ -43,7 +44,20 @@ borg upgrade Description ~~~~~~~~~~~ -Upgrade an existing Borg repository. +Upgrade an existing, local Borg repository. + +When you do not need borg upgrade ++++++++++++++++++++++++++++++++++ + +Not every change requires that you run ``borg upgrade``. + +You do **not** need to run it when: + +- moving your repository to a different place +- upgrading to another point release (like 1.0.x to 1.0.y), + except when noted otherwise in the changelog +- upgrading from 1.0.x to 1.1.x, + except when noted otherwise in the changelog Borg 1.x.y upgrades +++++++++++++++++++ @@ -75,8 +89,13 @@ helps with converting Borg 0.xx to 1.0. Currently, only LOCAL repositories can be upgraded (issue #465). -It will change the magic strings in the repository's segments -to match the new Borg magic strings. The keyfiles found in +Please note that ``borg create`` (since 1.0.0) uses bigger chunks by +default than old borg or attic did, so the new chunks won't deduplicate +with the old chunks in the upgraded repository. +See ``--chunker-params`` option of ``borg create`` and ``borg recreate``. + +``borg upgrade`` will change the magic strings in the repository's +segments to match the new Borg magic strings. The keyfiles found in $ATTIC_KEYS_DIR or ~/.attic/keys/ will also be converted and copied to $BORG_KEYS_DIR or ~/.config/borg/keys.