Merge pull request #2750 from borgbackup/rel1011rc1

release 1.0.11rc1

docs/changes.rst

@@ -128,8 +128,8 @@ The best check that everything is ok is to run a dry-run extraction::
 Changelog
 =========
 
-Version 1.0.11rc1 (not released yet)
+Version 1.0.11rc1 (2017-06-27)
-------------------------------------
+------------------------------
 
 Bug fixes:
 
@@ -229,7 +229,9 @@ Other changes:
     to break was fixed in cygwin at least since cygwin 2.8, maybe even since
     2.7.0).
   - remove skipping the noatime tests on GNU/Hurd, #2710
-
+  - fix borg import issue, add comment, #2718
+  - include attic.tar.gz when installing the package
+    also: add include_package_data=True
 
 Version 1.0.10 (2017-02-13)
 ---------------------------

docs/usage/break-lock.rst.inc

@@ -29,7 +29,8 @@ borg break-lock
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/change-passphrase.rst.inc

@@ -30,7 +30,8 @@ borg change-passphrase
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/check.rst.inc

@@ -31,7 +31,8 @@ borg check
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       --repository-only     only perform repository checks
       --archives-only       only perform archives checks
       --repair              attempt to repair any inconsistencies found

docs/usage/create.rst.inc

@@ -38,13 +38,16 @@ borg create
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
-      -s, --stats           print statistics for the created archive
+                            "borg")
+      -s, --stats           print statistics for the created archive. Requires
+                            -v/--verbose.
       -p, --progress        show progress display while creating the archive,
                             showing Original, Compressed and Deduplicated sizes,
                             followed by the Number of files seen and the path
                             being processed, default: False
-      --list                output verbose list of items (files, dirs, ...)
+      --list                output verbose list of items (files, dirs, ...).
+                            Requires -v/--verbose.
       --filter STATUSCHARS  only display items with the given status characters
       -e PATTERN, --exclude PATTERN
                             exclude paths matching PATTERN
@@ -84,9 +87,12 @@ Description
 ~~~~~~~~~~~
 
 This command creates a backup archive containing all files found while recursively
-traversing all paths specified. When giving '-' as path, borg will read data
+traversing all paths specified. Paths are added to the archive as they are given,
-from standard input and create a file 'stdin' in the created archive from that
+that means if relative paths are desired, the command has to be run from the correct
-data.
+directory.
+
+When giving '-' as path, borg will read data from standard input and create a
+file 'stdin' in the created archive from that data.
 
 The archive will consume almost no disk space for files or parts of files that
 have already been stored in other archives.
@@ -103,5 +109,10 @@ not provide correct inode information the --ignore-inode flag can be used. This
 potentially decreases reliability of change detection, while avoiding always reading
 all files on these file systems.
 
+The mount points of filesystems or filesystem snapshots should be the same for every
+creation of a new archive to ensure fast operation. This is because the file cache that
+is used to determine changed files quickly uses absolute filenames.
+If this is not possible, consider creating a bind mount to a stable location.
+
 See the output of the "borg help patterns" command for more help on exclude patterns.
 See the output of the "borg help placeholders" command for more help on placeholders.

docs/usage/delete.rst.inc

@@ -30,11 +30,14 @@ borg delete
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       -p, --progress        show progress display while deleting a single archive
-      -s, --stats           print statistics for the deleted archive
+      -s, --stats           print statistics for the deleted archive. Requires
+                            -v/--verbose.
       -c, --cache-only      delete only the local cache for the given repository
-      --force               force deletion of corrupted archives
+      --force               force deletion of corrupted archives, use --force
+                            --force in case --force does not work.
       --save-space          work slower, but using less space
     
 Description

docs/usage/extract.rst.inc

@@ -32,8 +32,10 @@ borg extract
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
-      --list                output verbose list of items (files, dirs, ...)
+                            "borg")
+      --list                output verbose list of items (files, dirs, ...).
+                            Requires -v/--verbose.
       -n, --dry-run         do not actually change any files
       -e PATTERN, --exclude PATTERN
                             exclude paths matching PATTERN

docs/usage/help.rst.inc

@@ -52,6 +52,15 @@ Prefix path, selector `pp:`
     This pattern style is useful to match whole sub-directories. The pattern
     `pp:/data/bar` matches `/data/bar` and everything therein.
 
+.. note::
+
+    `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE
+    engine. It is very easy to formulate patterns for each of these types which
+    requires an inordinate amount of time to match paths. If untrusted users
+    are able to supply patterns, ensure they cannot supply `re:` patterns.
+    Further, ensure that `sh:` and `fm:` patterns only contain a handful of
+    wildcards at most.
+
 Exclusions can be passed via the command line option `--exclude`. When used
 from within a shell the patterns should be quoted to protect them from
 expansion.

docs/usage/info.rst.inc

@@ -29,7 +29,8 @@ borg info
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/init.rst.inc

@@ -29,7 +29,8 @@ borg init
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       -e {none,keyfile,repokey}, --encryption {none,keyfile,repokey}
                             select encryption key mode (default: "repokey")
       -a, --append-only     create an append-only mode repository

docs/usage/key_export.rst.inc

@@ -31,7 +31,8 @@ borg key export
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       --paper               Create an export suitable for printing and later type-
                             in
       --qr-html             Create an html file suitable for printing and later

docs/usage/key_import.rst.inc

@@ -31,7 +31,8 @@ borg key import
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       --paper               interactively import from a backup done with --paper
     
 Description

docs/usage/list.rst.inc

@@ -31,7 +31,8 @@ borg list
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       --short               only print file/directory names, nothing else
       --list-format LISTFORMAT
                             specify format for archive file listing (default:

docs/usage/migrate-to-repokey.rst.inc

@@ -30,7 +30,8 @@ borg migrate-to-repokey
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/mount.rst.inc

@@ -31,7 +31,8 @@ borg mount
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       -f, --foreground      stay in foreground, do not daemonize
       -o OPTIONS            Extra mount options
     

docs/usage/prune.rst.inc

@@ -9,7 +9,7 @@ borg prune
     usage: borg prune [-h] [--critical] [--error] [--warning] [--info] [--debug]
                       [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M]
                       [--remote-path PATH] [-n] [--force] [-s] [--list]
-                      [--keep-within WITHIN] [-H HOURLY] [-d DAILY] [-w WEEKLY]
+                      [--keep-within INTERVAL] [-H HOURLY] [-d DAILY] [-w WEEKLY]
                       [-m MONTHLY] [-y YEARLY] [-P PREFIX] [--save-space]
                       [REPOSITORY]
     
@@ -31,12 +31,16 @@ borg prune
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       -n, --dry-run         do not change repository
       --force               force pruning of corrupted archives
-      -s, --stats           print statistics for the deleted archive
+      -s, --stats           print statistics for the deleted archive. Requires
-      --list                output verbose list of archives it keeps/prunes
+                            -v/--verbose.
-      --keep-within WITHIN  keep all archives within this time interval
+      --list                output verbose list of archives it keeps/prunes.
+                            Requires -v/--verbose.
+      --keep-within INTERVAL
+                            keep all archives within this time interval
       -H HOURLY, --keep-hourly HOURLY
                             number of hourly archives to keep
       -d DAILY, --keep-daily DAILY

docs/usage/rename.rst.inc

@@ -30,7 +30,8 @@ borg rename
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/serve.rst.inc

@@ -27,7 +27,8 @@ borg serve
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       --restrict-to-path PATH
                             restrict repository access to PATH. Can be specified
                             multiple times to allow the client access to several

docs/usage/umount.rst.inc

@@ -29,7 +29,8 @@ borg umount
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
     
 Description
 ~~~~~~~~~~~

docs/usage/upgrade.rst.inc

@@ -30,7 +30,8 @@ borg upgrade
       --no-files-cache      do not load/update the file metadata cache used to
                             detect unchanged files
       --umask M             set umask to M (local and remote, default: 0077)
-      --remote-path PATH    set remote path to executable (default: "borg")
+      --remote-path PATH    use PATH as borg executable on the remote (default:
+                            "borg")
       -p, --progress        show progress display while upgrading the repository
       -n, --dry-run         do not change repository
       -i, --inplace         rewrite repository in place, with no chance of going
@@ -43,7 +44,20 @@ borg upgrade
 Description
 ~~~~~~~~~~~
 
-Upgrade an existing Borg repository.
+Upgrade an existing, local Borg repository.
+
+When you do not need borg upgrade
++++++++++++++++++++++++++++++++++
+
+Not every change requires that you run ``borg upgrade``.
+
+You do **not** need to run it when:
+
+- moving your repository to a different place
+- upgrading to another point release (like 1.0.x to 1.0.y),
+  except when noted otherwise in the changelog
+- upgrading from 1.0.x to 1.1.x,
+  except when noted otherwise in the changelog
 
 Borg 1.x.y upgrades
 +++++++++++++++++++
@@ -75,8 +89,13 @@ helps with converting Borg 0.xx to 1.0.
 
 Currently, only LOCAL repositories can be upgraded (issue #465).
 
-It will change the magic strings in the repository's segments
+Please note that ``borg create`` (since 1.0.0) uses bigger chunks by
-to match the new Borg magic strings. The keyfiles found in
+default than old borg or attic did, so the new chunks won't deduplicate
+with the old chunks in the upgraded repository.
+See ``--chunker-params`` option of ``borg create`` and ``borg recreate``.
+
+``borg upgrade`` will change the magic strings in the repository's
+segments to match the new Borg magic strings. The keyfiles found in
 $ATTIC_KEYS_DIR or ~/.attic/keys/ will also be converted and
 copied to $BORG_KEYS_DIR or ~/.config/borg/keys.