diff --git a/docs/usage/borgfs.rst.inc b/docs/usage/borgfs.rst.inc new file mode 100644 index 000000000..3488e936b --- /dev/null +++ b/docs/usage/borgfs.rst.inc @@ -0,0 +1,138 @@ +.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! + +.. _borg_borgfs: + +borg borgfs +----------- +.. code-block:: none + + borg [common options] borgfs [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...] + +.. only:: html + + .. class:: borg-options-table + + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **positional arguments** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``REPOSITORY_OR_ARCHIVE`` | repository/archive to mount | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``MOUNTPOINT`` | where to mount filesystem | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``PATH`` | paths to extract; patterns are supported | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **optional arguments** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-V``, ``--version`` | show version number and exit | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-f``, ``--foreground`` | stay in foreground, do not daemonize | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-o`` | Extra mount options | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Archive filters** — Archive filters can be applied to repository targets. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-P PREFIX``, ``--prefix PREFIX`` | only consider archive names starting with this prefix. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-a GLOB``, ``--glob-archives GLOB`` | only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--sort-by KEYS`` | Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--first N`` | consider first N archives after other filters were applied | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--last N`` | consider last N archives after other filters were applied | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Exclusion options** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. raw:: html + + + +.. only:: latex + + REPOSITORY_OR_ARCHIVE + repository/archive to mount + MOUNTPOINT + where to mount filesystem + PATH + paths to extract; patterns are supported + + + optional arguments + -V, --version show version number and exit + -f, --foreground stay in foreground, do not daemonize + -o Extra mount options + + + :ref:`common_options` + | + + Archive filters + -P PREFIX, --prefix PREFIX only consider archive names starting with this prefix. + -a GLOB, --glob-archives GLOB only consider archive names matching the glob. sh: rules apply, see "borg help patterns". ``--prefix`` and ``--glob-archives`` are mutually exclusive. + --sort-by KEYS Comma-separated list of sorting keys; valid keys are: timestamp, name, id; default is: timestamp + --first N consider first N archives after other filters were applied + --last N consider last N archives after other filters were applied + + + Exclusion options + -e PATTERN, --exclude PATTERN exclude paths matching PATTERN + --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line + --pattern PATTERN experimental: include/exclude paths matching PATTERN + --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. + + +Description +~~~~~~~~~~~ + +This command mounts an archive as a FUSE filesystem. This can be useful for +browsing an archive or restoring individual files. Unless the ``--foreground`` +option is given the command will run in the background until the filesystem +is ``umounted``. + +The command ``borgfs`` provides a wrapper for ``borg mount``. This can also be +used in fstab entries: +``/path/to/repo /mnt/point fuse.borgfs defaults,noauto 0 0`` + +To allow a regular user to use fstab entries, add the ``user`` option: +``/path/to/repo /mnt/point fuse.borgfs defaults,noauto,user 0 0`` + +For mount options, see the fuse(8) manual page. Additional mount options +supported by borg: + +- versions: when used with a repository mount, this gives a merged, versioned + view of the files in the archives. EXPERIMENTAL, layout may change in future. +- allow_damaged_files: by default damaged files (where missing chunks were + replaced with runs of zeros by borg check ``--repair``) are not readable and + return EIO (I/O error). Set this option to read such files. + +The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users +to tweak the performance. It sets the number of cached data chunks; additional +memory usage can be up to ~8 MiB times this number. The default is the number +of CPU cores. + +When the daemonized process receives a signal or crashes, it does not unmount. +Unmounting in these cases could cause an active rsync or similar process +to unintentionally delete data. + +When running in the foreground ^C/SIGINT unmounts cleanly, but other +signals or crashes do not. \ No newline at end of file diff --git a/docs/usage/change-passphrase.rst.inc b/docs/usage/change-passphrase.rst.inc index 1114d3503..44edba19b 100644 --- a/docs/usage/change-passphrase.rst.inc +++ b/docs/usage/change-passphrase.rst.inc @@ -43,4 +43,9 @@ Description ~~~~~~~~~~~ The key files used for repository encryption are optionally passphrase -protected. This command can be used to change this passphrase. \ No newline at end of file +protected. This command can be used to change this passphrase. + +Please note that this command only changes the passphrase, but not any +secret protected by it (like e.g. encryption/MAC keys or chunker seed). +Thus, changing the passphrase after passphrase and borg key got compromised +does not protect future (nor past) backups to the same repository. \ No newline at end of file diff --git a/docs/usage/common-options.rst.inc b/docs/usage/common-options.rst.inc index 093041c47..f6e191b0b 100644 --- a/docs/usage/common-options.rst.inc +++ b/docs/usage/common-options.rst.inc @@ -10,7 +10,6 @@ --lock-wait SECONDS wait at most SECONDS for acquiring a repository/cache lock (default: 1). --show-version show/log the borg version --show-rc show/log the return code (rc) ---no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") --remote-ratelimit RATE set remote network upload rate limit in kiByte/s (default: 0=unlimited) diff --git a/docs/usage/create.rst.inc b/docs/usage/create.rst.inc index c70e2a8f1..57ba13a68 100644 --- a/docs/usage/create.rst.inc +++ b/docs/usage/create.rst.inc @@ -31,7 +31,9 @@ borg create +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--json`` | output stats as JSON. Implies ``--stats``. | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--no-cache-sync`` | experimental: do not synchronize the cache. Implies ``--no-files-cache``. | + | | ``--no-cache-sync`` | experimental: do not synchronize the cache. Implies not using the files cache. | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--no-files-cache`` | do not load/update the file metadata cache used to detect unchanged files | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | .. class:: borg-common-opt-ref | | | @@ -53,6 +55,8 @@ borg create +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--exclude-nodump`` | exclude files flagged NODUMP | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | **Filesystem options** | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``-x``, ``--one-file-system`` | stay in the same file system and do not store mount points of other file systems | @@ -63,8 +67,14 @@ borg create +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--noctime`` | do not store ctime into archive | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--nobirthtime`` | do not store birthtime (creation date) into archive | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--nobsdflags`` | do not read and store bsdflags (e.g. NODUMP, IMMUTABLE) into archive | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--ignore-inode`` | ignore inode data in the file metadata cache used to detect unchanged files. | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--files-cache MODE`` | operate files cache in MODE. default: ctime,size,inode | + +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--read-special`` | open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files. | +-------------------------------------------------------+---------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | **Archive options** | @@ -102,7 +112,8 @@ borg create --list output verbose list of items (files, dirs, ...) --filter STATUSCHARS only display items with the given status characters (see description) --json output stats as JSON. Implies ``--stats``. - --no-cache-sync experimental: do not synchronize the cache. Implies ``--no-files-cache``. + --no-cache-sync experimental: do not synchronize the cache. Implies not using the files cache. + --no-files-cache do not load/update the file metadata cache used to detect unchanged files :ref:`common_options` @@ -116,6 +127,7 @@ borg create --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive + --exclude-nodump exclude files flagged NODUMP Filesystem options @@ -123,7 +135,10 @@ borg create --numeric-owner only store numeric user and group identifiers --noatime do not store atime into archive --noctime do not store ctime into archive + --nobirthtime do not store birthtime (creation date) into archive + --nobsdflags do not read and store bsdflags (e.g. NODUMP, IMMUTABLE) into archive --ignore-inode ignore inode data in the file metadata cache used to detect unchanged files. + --files-cache MODE operate files cache in MODE. default: ctime,size,inode --read-special open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files. @@ -157,13 +172,39 @@ In the archive name, you may use the following placeholders: {now}, {utcnow}, {fqdn}, {hostname}, {user} and some others. Backup speed is increased by not reprocessing files that are already part of -existing archives and weren't modified. Normally, detecting file modifications -will take inode information into consideration. This is problematic for files -located on sshfs and similar network file systems which do not provide stable -inode numbers, such files will always be considered modified. The -``--ignore-inode`` flag can be used to prevent this and improve performance. -This flag will reduce reliability of change detection however, with files -considered unmodified as long as their size and modification time are unchanged. +existing archives and weren't modified. The detection of unmodified files is +done by comparing multiple file metadata values with previous values kept in +the files cache. + +This comparison can operate in different modes as given by ``--files-cache``: + +- ctime,size,inode (default) +- mtime,size,inode (default behaviour of borg versions older than 1.1.0rc4) +- ctime,size (ignore the inode number) +- mtime,size (ignore the inode number) +- rechunk,ctime (all files are considered modified - rechunk, cache ctime) +- rechunk,mtime (all files are considered modified - rechunk, cache mtime) +- disabled (disable the files cache, all files considered modified - rechunk) + +inode number: better safety, but often unstable on network filesystems + +Normally, detecting file modifications will take inode information into +consideration to improve the reliability of file change detection. +This is problematic for files located on sshfs and similar network file +systems which do not provide stable inode numbers, such files will always +be considered modified. You can use modes without `inode` in this case to +improve performance, but reliability of change detection might be reduced. + +ctime vs. mtime: safety vs. speed + +- ctime is a rather safe way to detect changes to a file (metadata and contents) + as it can not be set from userspace. But, a metadata-only change will already + update the ctime, so there might be some unnecessary chunking/hashing even + without content changes. Some filesystems do not support ctime (change time). +- mtime usually works and only updates if file contents were changed. But mtime + can be arbitrarily set from userspace, e.g. to set mtime back to the same value + it had before a content change happened. This can be used maliciously as well as + well-meant, but in both cases mtime based cache modes can be problematic. The mount points of filesystems or filesystem snapshots should be the same for every creation of a new archive to ensure fast operation. This is because the file cache that @@ -174,6 +215,12 @@ The ``--progress`` option shows (from left to right) Original, Compressed and De (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path. +When using ``--stats``, you will get some statistics about how much data was +added - the "This Archive" deduplicated size there is most interesting as that is +how much your repository will grow. Please note that the "All archives" stats refer to +the state after creation. Also, the ``--stats`` and ``--dry-run`` options are mutually +exclusive because the data is not actually compressed and deduplicated during a dry run. + See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. diff --git a/docs/usage/delete.rst.inc b/docs/usage/delete.rst.inc index e965b830f..5951c71a2 100644 --- a/docs/usage/delete.rst.inc +++ b/docs/usage/delete.rst.inc @@ -6,7 +6,7 @@ borg delete ----------- .. code-block:: none - borg [common options] delete [options] [TARGET] + borg [common options] delete [options] [TARGET] [ARCHIVE...] .. only:: html @@ -17,6 +17,8 @@ borg delete +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``TARGET`` | archive or repository to delete | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``ARCHIVE`` | archives to delete | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | **optional arguments** | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``-s``, ``--stats`` | print statistics for the deleted archive | @@ -56,6 +58,8 @@ borg delete TARGET archive or repository to delete + ARCHIVE + archives to delete optional arguments @@ -81,4 +85,9 @@ Description This command deletes an archive from the repository or the complete repository. Disk space is reclaimed accordingly. If you delete the complete repository, the -local cache for it (if any) is also deleted. \ No newline at end of file +local cache for it (if any) is also deleted. + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after deletion. \ No newline at end of file diff --git a/docs/usage/diff.rst.inc b/docs/usage/diff.rst.inc index ef5b831a6..5074a0993 100644 --- a/docs/usage/diff.rst.inc +++ b/docs/usage/diff.rst.inc @@ -12,43 +12,37 @@ borg diff .. class:: borg-options-table - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **positional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **optional arguments** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--numeric-owner`` | only consider numeric user and group identifiers | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--same-chunker-params`` | Override check of chunker parameters. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--sort`` | Sort the output lines by file path. | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | .. class:: borg-common-opt-ref | - | | - | :ref:`common_options` | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | **Exclusion options** | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-------------------------------------------------------+-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **positional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``REPO_ARCHIVE1`` | repository location and ARCHIVE1 name | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``ARCHIVE2`` | ARCHIVE2 name (no repository location allowed) | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``PATH`` | paths of items inside the archives to compare; patterns are supported | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **optional arguments** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--numeric-owner`` | only consider numeric user and group identifiers | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--same-chunker-params`` | Override check of chunker parameters. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--sort`` | Sort the output lines by file path. | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | .. class:: borg-common-opt-ref | + | | + | :ref:`common_options` | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | **Exclusion options** | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + +-------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------+ .. raw:: html @@ -82,9 +76,6 @@ borg diff --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description diff --git a/docs/usage/extract.rst.inc b/docs/usage/extract.rst.inc index ef9a0c75c..495f20fca 100644 --- a/docs/usage/extract.rst.inc +++ b/docs/usage/extract.rst.inc @@ -27,6 +27,8 @@ borg extract +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--numeric-owner`` | only obey numeric user and group identifiers | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ + | | ``--nobsdflags`` | do not extract/set bsdflags (e.g. NODUMP, IMMUTABLE) | + +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--stdout`` | write all extracted data to stdout | +-------------------------------------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------------+ | | ``--sparse`` | create holes in output sparse file from all-zero chunks | @@ -68,6 +70,7 @@ borg extract --list output verbose list of items (files, dirs, ...) -n, --dry-run do not actually change any files --numeric-owner only obey numeric user and group identifiers + --nobsdflags do not extract/set bsdflags (e.g. NODUMP, IMMUTABLE) --stdout write all extracted data to stdout --sparse create holes in output sparse file from all-zero chunks diff --git a/docs/usage/help.rst.inc b/docs/usage/help.rst.inc index 152e01b54..2d5455564 100644 --- a/docs/usage/help.rst.inc +++ b/docs/usage/help.rst.inc @@ -124,10 +124,12 @@ Examples:: may specify the backup roots (starting points) and patterns for inclusion/exclusion. A root path starts with the prefix `R`, followed by a path (a plain path, not a file pattern). An include rule starts with the prefix +, an exclude rule starts - with the prefix -, both followed by a pattern. + with the prefix -, an exclude-norecurse rule starts with !, all followed by a pattern. Inclusion patterns are useful to include paths that are contained in an excluded path. The first matching pattern is used so if an include pattern matches before - an exclude pattern, the file is backed up. + an exclude pattern, the file is backed up. If an exclude-norecurse pattern matches + a directory, it won't recurse into it and won't discover any potential matches for + include rules below that directory. Note that the default pattern style for ``--pattern`` and ``--patterns-from`` is shell style (`sh:`), so those patterns behave similar to rsync include/exclude @@ -169,11 +171,11 @@ placeholders: {now} The current local date and time, by default in ISO-8601 format. - You can also supply your own `format string `_, e.g. {now:%Y-%m-%d_%H:%M:%S} + You can also supply your own `format string `_, e.g. {now:%Y-%m-%d_%H:%M:%S} {utcnow} The current UTC date and time, by default in ISO-8601 format. - You can also supply your own `format string `_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S} + You can also supply your own `format string `_, e.g. {utcnow:%Y-%m-%d_%H:%M:%S} {user} The user name (or UID, if no name is available) of the user running borg. diff --git a/docs/usage/info.rst.inc b/docs/usage/info.rst.inc index ec5d15098..5ba601f4f 100644 --- a/docs/usage/info.rst.inc +++ b/docs/usage/info.rst.inc @@ -77,6 +77,11 @@ up to the deduplicated size of the repository ("all archives"), because the two are meaning different things: This archive / deduplicated size = amount of data stored ONLY for this archive - = unique chunks of this archive. += unique chunks of this archive. All archives / deduplicated size = amount of data stored in the repo - = all chunks in the repository. \ No newline at end of file += all chunks in the repository. + +Borg archives can only contain a limited amount of file metadata. +The size of an archive relative to this limit depends on a number of factors, +mainly the number of files, the lengths of paths and other metadata stored for files. +This is shown as *utilization of maximum supported archive size*. \ No newline at end of file diff --git a/docs/usage/key_change-passphrase.rst.inc b/docs/usage/key_change-passphrase.rst.inc index 64bc409c5..ab31b9fde 100644 --- a/docs/usage/key_change-passphrase.rst.inc +++ b/docs/usage/key_change-passphrase.rst.inc @@ -43,4 +43,9 @@ Description ~~~~~~~~~~~ The key files used for repository encryption are optionally passphrase -protected. This command can be used to change this passphrase. \ No newline at end of file +protected. This command can be used to change this passphrase. + +Please note that this command only changes the passphrase, but not any +secret protected by it (like e.g. encryption/MAC keys or chunker seed). +Thus, changing the passphrase after passphrase and borg key got compromised +does not protect future (nor past) backups to the same repository. \ No newline at end of file diff --git a/docs/usage/key_export.rst.inc b/docs/usage/key_export.rst.inc index 466ac07d9..55b3e46d6 100644 --- a/docs/usage/key_export.rst.inc +++ b/docs/usage/key_export.rst.inc @@ -59,11 +59,15 @@ Description If repository encryption is used, the repository is inaccessible without the key. This command allows to backup this essential key. +Note that the backup produced does not include the passphrase itself +(i.e. the exported key stays encrypted). In order to regain access to a +repository, one needs both the exported key and the original passphrase. -There are two backup formats. The normal backup format is suitable for +There are three backup formats. The normal backup format is suitable for digital storage as a file. The ``--paper`` backup format is optimized for printing and typing in while importing, with per line checks to -reduce problems with manual input. +reduce problems with manual input. The ``--qr-html`` creates a printable +HTML template with a QR code and a copy of the ``--paper``-formatted key. For repositories using keyfile encryption the key is saved locally on the system that is capable of doing backups. To guard against loss diff --git a/docs/usage/list.rst.inc b/docs/usage/list.rst.inc index 3358a74bf..fbdfc09f7 100644 --- a/docs/usage/list.rst.inc +++ b/docs/usage/list.rst.inc @@ -23,7 +23,7 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--short`` | only print file/directory names, nothing else | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}") | + | | ``--format FORMAT``, ``--list-format FORMAT`` | specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}") | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--json`` | Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -55,12 +55,6 @@ borg list +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-caches`` | exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--exclude-if-present NAME`` | exclude directories that are tagged by containing a filesystem object with the given NAME | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | ``--keep-exclude-tags``, ``--keep-tag-files`` | if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive | - +-----------------------------------------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. raw:: html @@ -80,7 +74,7 @@ borg list optional arguments --short only print file/directory names, nothing else - --format FORMAT, --list-format FORMAT specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NL}") + --format FORMAT, --list-format FORMAT specify format for file listing (default: "{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NL}") --json Only valid for listing repository contents. Format output as JSON. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "barchive" key is therefore not available. --json-lines Only valid for listing archive contents. Format output as JSON Lines. The form of ``--format`` is ignored, but keys used in it are added to the JSON output. Some keys are always present. Note: JSON can only represent text. A "bpath" key is therefore not available. @@ -101,9 +95,6 @@ borg list --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --pattern PATTERN experimental: include/exclude paths matching PATTERN --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line - --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) - --exclude-if-present NAME exclude directories that are tagged by containing a filesystem object with the given NAME - --keep-exclude-tags, --keep-tag-files if tag objects are specified with ``--exclude-if-present``, don't omit the tag objects themselves from the backup archive Description @@ -127,15 +118,15 @@ The following keys are available for ``--format``: Keys for listing repository archives: -- name: archive name interpreted as text (might be missing non-text characters, see barchive) - archive: archive name interpreted as text (might be missing non-text characters, see barchive) +- name: alias of "archive" - barchive: verbatim archive name, can contain any character except NUL - comment: archive comment interpreted as text (might be missing non-text characters, see bcomment) - bcomment: verbatim archive comment, can contain any character except NUL - id: internal ID of the archive -- time: time (start) of creation of the archive - start: time (start) of creation of the archive +- time: alias of "start" - end: time (end) of creation of the archive diff --git a/docs/usage/mount.rst.inc b/docs/usage/mount.rst.inc index deff36657..ee7b80a07 100644 --- a/docs/usage/mount.rst.inc +++ b/docs/usage/mount.rst.inc @@ -6,7 +6,7 @@ borg mount ---------- .. code-block:: none - borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT + borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...] .. only:: html @@ -19,6 +19,8 @@ borg mount +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``MOUNTPOINT`` | where to mount filesystem | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``PATH`` | paths to extract; patterns are supported | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | **optional arguments** | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``-f``, ``--foreground`` | stay in foreground, do not daemonize | @@ -41,6 +43,18 @@ borg mount +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ | | ``--last N`` | consider last N archives after other filters were applied | +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Exclusion options** | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``-e PATTERN``, ``--exclude PATTERN`` | exclude paths matching PATTERN | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--exclude-from EXCLUDEFILE`` | read exclude patterns from EXCLUDEFILE, one per line | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--pattern PATTERN`` | experimental: include/exclude paths matching PATTERN | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--patterns-from PATTERNFILE`` | experimental: read include/exclude patterns from PATTERNFILE, one per line | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | ``--strip-components NUMBER`` | Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. | + +-----------------------------------------------------------------------------+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+ .. raw:: html @@ -56,6 +70,8 @@ borg mount repository/archive to mount MOUNTPOINT where to mount filesystem + PATH + paths to extract; patterns are supported optional arguments @@ -74,6 +90,14 @@ borg mount --last N consider last N archives after other filters were applied + Exclusion options + -e PATTERN, --exclude PATTERN exclude paths matching PATTERN + --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line + --pattern PATTERN experimental: include/exclude paths matching PATTERN + --patterns-from PATTERNFILE experimental: read include/exclude patterns from PATTERNFILE, one per line + --strip-components NUMBER Remove the specified number of leading path elements. Paths with fewer elements will be silently skipped. + + Description ~~~~~~~~~~~ diff --git a/docs/usage/prune.rst.inc b/docs/usage/prune.rst.inc index f98fbab84..81c2d32f9 100644 --- a/docs/usage/prune.rst.inc +++ b/docs/usage/prune.rst.inc @@ -134,4 +134,9 @@ negative number of archives to keep means that there is no limit. The ``--keep-last N`` option is doing the same as ``--keep-secondly N`` (and it will keep the last N archives under the assumption that you do not create more than one -backup archive in the same second). \ No newline at end of file +backup archive in the same second). + +When using ``--stats``, you will get some statistics about how much data was +deleted - the "Deleted data" deduplicated size there is most interesting as +that is how much your repository will shrink. +Please note that the "All archives" stats refer to the state after pruning. \ No newline at end of file diff --git a/docs/usage/recreate.rst.inc b/docs/usage/recreate.rst.inc index ef258e694..b3071ab3d 100644 --- a/docs/usage/recreate.rst.inc +++ b/docs/usage/recreate.rst.inc @@ -150,4 +150,15 @@ With ``--target`` the original archive is not replaced, instead a new archive is When rechunking space usage can be substantial, expect at least the entire deduplicated size of the archives using the previous chunker params. When recompressing expect approx. (throughput / checkpoint-interval) in space usage, -assuming all chunks are recompressed. \ No newline at end of file +assuming all chunks are recompressed. + +If you recently ran borg check --repair and it had to fix lost chunks with all-zero +replacement chunks, please first run another backup for the same data and re-run +borg check --repair afterwards to heal any archives that had lost chunks which are +still generated from the input data. + +Important: running borg recreate to re-chunk will remove the chunks_healthy +metadata of all items with replacement chunks, so healing will not be possible +any more after re-chunking (it is also unlikely it would ever work: due to the +change of chunking parameters, the missing chunk likely will never be seen again +even if you still have the data that produced it). \ No newline at end of file diff --git a/docs/usage/upgrade.rst.inc b/docs/usage/upgrade.rst.inc index eafa4362a..47a240dd0 100644 --- a/docs/usage/upgrade.rst.inc +++ b/docs/usage/upgrade.rst.inc @@ -130,13 +130,13 @@ make sure the cache files are also removed: borg delete borg -Unless ``--inplace`` is specified, the upgrade process first -creates a backup copy of the repository, in -REPOSITORY.before-upgrade-DATETIME, using hardlinks. This takes -longer than in place upgrades, but is much safer and gives -progress information (as opposed to ``cp -al``). Once you are -satisfied with the conversion, you can safely destroy the -backup copy. +Unless ``--inplace`` is specified, the upgrade process first creates a backup +copy of the repository, in REPOSITORY.before-upgrade-DATETIME, using hardlinks. +This requires that the repository and its parent directory reside on same +filesystem so the hardlink copy can work. +This takes longer than in place upgrades, but is much safer and gives +progress information (as opposed to ``cp -al``). Once you are satisfied +with the conversion, you can safely destroy the backup copy. WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back diff --git a/docs/usage/with-lock.rst.inc b/docs/usage/with-lock.rst.inc index fc4541164..6f4fc741a 100644 --- a/docs/usage/with-lock.rst.inc +++ b/docs/usage/with-lock.rst.inc @@ -60,5 +60,6 @@ code as borg's return code. .. note:: If you copy a repository with the lock held, the lock will be present in - the copy, obviously. Thus, before using borg on the copy, you need to - use "borg break-lock" on it. \ No newline at end of file + the copy. Thus, before using borg on the copy from a different host, + you need to use "borg break-lock" on the copied repository, because + Borg is cautious and does not automatically remove stale locks made by a different host. \ No newline at end of file