diff --git a/docs/changes.rst b/docs/changes.rst index 256e652f4..d7ce391bf 100644 --- a/docs/changes.rst +++ b/docs/changes.rst @@ -2607,36 +2607,6 @@ Other changes: - added FAQ section about backing up root partition -Version 1.0.10 (2017-02-13) ---------------------------- - -Bug fixes: - -- Manifest timestamps are now monotonically increasing, - this fixes issues when the system clock jumps backwards - or is set inconsistently across computers accessing the same repository, #2115 -- Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on - py.test >= 3.0, #2112 - -New features: - -- "key export" can now generate a printable HTML page with both a QR code and - a human-readable "paperkey" representation (and custom text) through the - ``--qr-html`` option. - - The same functionality is also available through `paperkey.html `_, - which is the same HTML page generated by ``--qr-html``. It works with existing - "key export" files and key files. - -Other changes: - -- docs: - - - language clarification - "borg create --one-file-system" option does not respect - mount points, but considers different file systems instead, #2141 -- setup.py: build_api: sort file list for determinism - - Version 1.1.0b3 (2017-01-15) ---------------------------- @@ -2718,6 +2688,438 @@ Other changes: - remove all BORG_* env vars from the outer environment +Version 1.1.0b2 (2016-10-01) +---------------------------- + +Bug fixes: + +- fix incorrect preservation of delete tags, leading to "object count mismatch" + on borg check, #1598. This only occurred with 1.1.0b1 (not with 1.0.x) and is + normally fixed by running another borg create/delete/prune. +- fix broken --progress for double-cell paths (e.g. CJK), #1624 +- borg recreate: also catch SIGHUP +- FUSE: + + - fix hardlinks in versions view, #1599 + - add parameter check to ItemCache.get to make potential failures more clear + +New features: + +- Archiver, RemoteRepository: add --remote-ratelimit (send data) +- borg help compression, #1582 +- borg check: delete chunks with integrity errors, #1575, so they can be + "repaired" immediately and maybe healed later. +- archives filters concept (refactoring/unifying older code) + + - covers --first/--last/--prefix/--sort-by options + - currently used for borg list/info/delete + +Other changes: + +- borg check --verify-data slightly tuned (use get_many()) +- change {utcnow} and {now} to ISO-8601 format ("T" date/time separator) +- repo check: log transaction IDs, improve object count mismatch diagnostic +- Vagrantfile: use TW's fresh-bootloader pyinstaller branch +- fix module names in api.rst +- hashindex: bump api_version + + +Version 1.1.0b1 (2016-08-28) +---------------------------- + +New features: + +- new commands: + + - borg recreate: re-create existing archives, #787 #686 #630 #70, also see + #757, #770. + + - selectively remove files/dirs from old archives + - re-compress data + - re-chunkify data, e.g. to have upgraded Attic / Borg 0.xx archives + deduplicate with Borg 1.x archives or to experiment with chunker-params. + - borg diff: show differences between archives + - borg with-lock: execute a command with the repository locked, #990 +- borg create: + + - Flexible compression with pattern matching on path/filename, + and LZ4 heuristic for deciding compressibility, #810, #1007 + - visit files in inode order (better speed, esp. for large directories and rotating disks) + - in-file checkpoints, #1217 + - increased default checkpoint interval to 30 minutes (was 5 minutes), #896 + - added uuid archive format tag, #1151 + - save mountpoint directories with --one-file-system, makes system restore easier, #1033 + - Linux: added support for some BSD flags, #1050 + - add 'x' status for excluded paths, #814 + + - also means files excluded via UF_NODUMP, #1080 +- borg check: + + - will not produce the "Checking segments" output unless new --progress option is passed, #824. + - --verify-data to verify data cryptographically on the client, #975 +- borg list, #751, #1179 + + - removed {formatkeys}, see "borg list --help" + - --list-format is deprecated, use --format instead + - --format now also applies to listing archives, not only archive contents, #1179 + - now supports the usual [PATH [PATHS…]] syntax and excludes + - new keys: csize, num_chunks, unique_chunks, NUL + - supports guaranteed_available hashlib hashes + (to avoid varying functionality depending on environment), + which includes the SHA1 and SHA2 family as well as MD5 +- borg prune: + + - to better visualize the "thinning out", we now list all archives in + reverse time order. rephrase and reorder help text. + - implement --keep-last N via --keep-secondly N, also --keep-minutely. + assuming that there is not more than 1 backup archive made in 1s, + --keep-last N and --keep-secondly N are equivalent, #537 + - cleanup checkpoints except the latest, #1008 +- borg extract: + + - added --progress, #1449 + - Linux: limited support for BSD flags, #1050 +- borg info: + + - output is now more similar to borg create --stats, #977 +- borg mount: + + - provide "borgfs" wrapper for borg mount, enables usage via fstab, #743 + - "versions" mount option - when used with a repository mount, this gives + a merged, versioned view of the files in all archives, #729 +- repository: + + - added progress information to commit/compaction phase (often takes some time when deleting/pruning), #1519 + - automatic recovery for some forms of repository inconsistency, #858 + - check free space before going forward with a commit, #1336 + - improved write performance (esp. for rotating media), #985 + + - new IO code for Linux + - raised default segment size to approx 512 MiB + - improved compaction performance, #1041 + - reduced client CPU load and improved performance for remote repositories, #940 + +- options that imply output (--show-rc, --show-version, --list, --stats, + --progress) don't need -v/--info to have that output displayed, #865 +- add archive comments (via borg (re)create --comment), #842 +- borg list/prune/delete: also output archive id, #731 +- --show-version: shows/logs the borg version, #725 +- added --debug-topic for granular debug logging, #1447 +- use atomic file writing/updating for configuration and key files, #1377 +- BORG_KEY_FILE environment variable, #1001 +- self-testing module, #970 + + +Bug fixes: + +- list: fixed default output being produced if --format is given with empty parameter, #1489 +- create: fixed overflowing progress line with CJK and similar characters, #1051 +- prune: fixed crash if --prefix resulted in no matches, #1029 +- init: clean up partial repo if passphrase input is aborted, #850 +- info: quote cmdline arguments that have spaces in them +- fix hardlinks failing in some cases for extracting subtrees, #761 + +Other changes: + +- replace stdlib hmac with OpenSSL, zero-copy decrypt (10-15% increase in + performance of hash-lists and extract). +- improved chunker performance, #1021 +- open repository segment files in exclusive mode (fail-safe), #1134 +- improved error logging, #1440 +- Source: + + - pass meta-data around, #765 + - move some constants to new constants module + - better readability and fewer errors with namedtuples, #823 + - moved source tree into src/ subdirectory, #1016 + - made borg.platform a package, #1113 + - removed dead crypto code, #1032 + - improved and ported parts of the test suite to py.test, #912 + - created data classes instead of passing dictionaries around, #981, #1158, #1161 + - cleaned up imports, #1112 +- Docs: + + - better help texts and sphinx reproduction of usage help: + + - Group options + - Nicer list of options in Sphinx + - Deduplicate 'Common options' (including --help) + - chunker: added some insights by "Voltara", #903 + - clarify what "deduplicated size" means + - fix / update / add package list entries + - added a SaltStack usage example, #956 + - expanded FAQ + - new contributors in AUTHORS! +- Tests: + + - vagrant: add ubuntu/xenial 64bit - this box has still some issues + - ChunkBuffer: add test for leaving partial chunk in buffer, fixes #945 + + +Version 1.0.13 (2019-02-15) +--------------------------- + +Please note: this is very likely the last 1.0.x release, please upgrade to 1.1.x. + +Bug fixes: + +- security fix: configure FUSE with "default_permissions", #3903. + "default_permissions" is now enforced by borg by default to let the + kernel check uid/gid/mode based permissions. + "ignore_permissions" can be given to not enforce "default_permissions". +- xattrs: fix borg exception handling on ENOSPC error, #3808. + +New features: + +- Read a passphrase from a file descriptor specified in the + BORG_PASSPHRASE_FD environment variable. + +Other changes: + +- acl platform code: fix acl set return type +- xattr: + + - add linux {list,get,set}xattr ctypes prototypes + - fix darwin flistxattr ctypes prototype +- testing / travis-ci: + + - fix the homebrew 1.9 issues on travis-ci, #4254 + - travis OS X: use xcode 8.3 (not broken) + - tox.ini: lock requirements + - unbreak 1.0-maint on travis, fixes #4123 +- vagrant: + + - misc. fixes + - FUSE for macOS: upgrade 3.7.1 to 3.8.3 + - Python: upgrade 3.5.5 to 3.5.6 +- docs: + + - Update installation instructions for macOS + - update release workflow using twine (docs, scripts), #4213 + +Version 1.0.12 (2018-04-08) +--------------------------- + +Bug fixes: + +- repository: cleanup/write: invalidate cached FDs, tests +- serve: fix exitcode, #2910 +- extract: set bsdflags last (include immutable flag), #3263 +- create --timestamp: set start time, #2957 +- create: show excluded dir with "x" for tagged dirs / caches, #3189 +- migrate locks to child PID when daemonize is used +- Buffer: fix wrong thread-local storage use, #2951 +- fix detection of non-local path, #3108 +- fix LDLP restoration for subprocesses, #3077 +- fix subprocess environments (xattr module's fakeroot version check, + borg umount, BORG_PASSCOMMAND), #3050 +- remote: deal with partial lines, #2637 +- get rid of datetime.isoformat, use safe parse_timestamp to parse + timestamps, #2994 +- build: do .h file content checks in binary mode, fixes build issue for + non-ascii header files on pure-ascii locale platforms, #3544 #3639 +- remove platform.uname() call which caused library mismatch issues, #3732 +- add exception handler around deprecated platform.linux_distribution() call + +Other changes: + +- require msgpack-python >= 0.4.6 and < 0.5.0, see #3753 +- add parens for C preprocessor macro argument usages (did not cause + malfunction) +- ignore corrupt files cache, #2939 +- replace "modulo" with "if" to check for wraparound in hashmap +- keymanager: don't depend on optional readline module, #2980 +- exclude broken pytest 3.3.0 release +- exclude broken Cython 0.27(.0) release, #3066 +- flake8: add some ignores +- docs: + + - create: document exclusion through nodump + - document good and problematic option placements, fix examples, #3356 + - update docs about hardlinked symlinks limitation + - faq: we do not implement futile attempts of ETA / progress displays + - simplified rate limiting wrapper in FAQ + - twitter account @borgbackup, #2948 + - add note about metadata dedup and --no[ac]time, #2518 + - change-passphrase only changes the passphrase, #2990 + - clarify encrypted key format for borg key export, #3296 + - document sshfs rename workaround, #3315 + - update release checklist about security fixes + - docs about how to verify a signed release, #3634 + - chunk seed is generated per /repository/ +- vagrant: + + - use FUSE for macOS 3.7.1 to build the macOS binary + - use python 3.5.5 to build the binaries + - add exe location to PATH when we build an exe + - use https pypi url for wheezy + - netbsd: bash is already installed + - netbsd: fix netbsd version in PKG_PATH + - use self-made FreeBSD 10.3 box, #3022 + - backport fs_init (including related updates) from 1.1 + - the boxcutter wheezy boxes are 404, use local ones +- travis: + + - don't perform full Travis build on docs-only changes, #2531 + - only short-circuit docs-only changes for pull requests + + +Version 1.0.11 (2017-07-21) +--------------------------- + +Bug fixes: + +- use limited unpacker for outer key (security precaution), #2174 +- fix paperkey import bug + +Other changes: + +- change --checkpoint-interval default from 600s to 1800s, #2841. + this improves efficiency for big repositories a lot. +- docs: fix OpenSUSE command and add OpenSUSE section +- tests: add tests for split_lstring and paperkey +- vagrant: + + - fix openbsd shell + - backport cpu/ram setup from master + - add stretch64 VM + +Version 1.0.11rc1 (2017-06-27) +------------------------------ + +Bug fixes: + +- performance: rebuild hashtable if we have too few empty buckets, #2246. + this fixes some sporadic, but severe performance breakdowns. +- Archive: allocate zeros when needed, #2308 + fixes huge memory usage of mount (8 MiB × number of archives) +- IPv6 address support + also: Location: more informative exception when parsing fails +- borg single-file binary: use pyinstaller v3.2.1, #2396 + this fixes that the prelink cronjob on some distros kills the + borg binary by stripping away parts of it. +- extract: + + - warning for unextracted big extended attributes, #2258 + - also create parent dir for device files, if needed. + - don't write to disk with --stdout, #2645 +- archive check: detect and fix missing all-zero replacement chunks, #2180 +- fix (de)compression exceptions, #2224 #2221 +- files cache: update inode number, #2226 +- borg rpc: use limited msgpack.Unpacker (security precaution), #2139 +- Manifest: use limited msgpack.Unpacker (security precaution), #2175 +- Location: accept //servername/share/path +- fix ChunkIndex.__contains__ assertion for big-endian archs (harmless) +- create: handle BackupOSError on a per-path level in one spot +- fix error msg, there is no --keep-last in borg 1.0.x, #2282 +- clamp (nano)second values to unproblematic range, #2304 +- fuse / borg mount: + + - fix st_blocks to be an integer (not float) value + - fix negative uid/gid crash (they could come into archives e.g. when + backing up external drives under cygwin), #2674 + - fix crash if empty (None) xattr is read + - do pre-mount checks before opening repository + - check llfuse is installed before asking for passphrase +- borg rename: expand placeholders, #2386 +- borg serve: fix forced command lines containing BORG_ env vars +- fix error msg, it is --keep-within, not --within +- fix borg key/debug/benchmark crashing without subcommand, #2240 +- chunker: fix invalid use of types, don't do uint32_t >> 32 +- document follow_symlinks requirements, check libc, #2507 + +New features: + +- added BORG_PASSCOMMAND environment variable, #2573 +- add minimal version of in repository mandatory feature flags, #2134 + + This should allow us to make sure older borg versions can be cleanly + prevented from doing operations that are no longer safe because of + repository format evolution. This allows more fine grained control than + just incrementing the manifest version. So for example a change that + still allows new archives to be created but would corrupt the repository + when an old version tries to delete an archive or check the repository + would add the new feature to the check and delete set but leave it out + of the write set. +- borg delete --force --force to delete severely corrupted archives, #1975 + +Other changes: + +- embrace y2038 issue to support 32bit platforms +- be more clear that this is a "beyond repair" case, #2427 +- key file names: limit to 100 characters and remove colons from host name +- upgrade FUSE for macOS to 3.5.8, #2346 +- split up parsing and filtering for --keep-within, better error message, #2610 +- docs: + + - fix caskroom link, #2299 + - address SSH batch mode, #2202 #2270 + - improve remote-path description + - document snapshot usage, #2178 + - document relative path usage, #1868 + - one link per distro in the installation page + - development: new branching model in git repository + - kill api page + - added FAQ section about backing up root partition + - add bountysource badge, #2558 + - create empty docs.txt reequirements, #2694 + - README: how to help the project + - note -v/--verbose requirement on affected options, #2542 + - document borg init behaviour via append-only borg serve, #2440 + - be clear about what buzhash is used for (chunking) and want it is not + used for (deduplication)- also say already in the readme that we use a + cryptohash for dedupe, so people don't worry, #2390 + - add hint about chunker params to borg upgrade docs, #2421 + - clarify borg upgrade docs, #2436 + - quickstart: delete problematic BORG_PASSPRHASE use, #2623 + - faq: specify "using inline shell scripts" + - document pattern denial of service, #2624 +- tests: + + - remove attic dependency of the tests, #2505 + - travis: + + - enhance travis setuptools_scm situation + - install fakeroot for Linux + - add test for borg delete --force + - enable remote tests on cygwin (the cygwin issue that caused these tests + to break was fixed in cygwin at least since cygwin 2.8, maybe even since + 2.7.0). + - remove skipping the noatime tests on GNU/Hurd, #2710 + - fix borg import issue, add comment, #2718 + - include attic.tar.gz when installing the package + also: add include_package_data=True + +Version 1.0.10 (2017-02-13) +--------------------------- + +Bug fixes: + +- Manifest timestamps are now monotonically increasing, + this fixes issues when the system clock jumps backwards + or is set inconsistently across computers accessing the same repository, #2115 +- Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on + py.test >= 3.0, #2112 + +New features: + +- "key export" can now generate a printable HTML page with both a QR code and + a human-readable "paperkey" representation (and custom text) through the + ``--qr-html`` option. + + The same functionality is also available through `paperkey.html `_, + which is the same HTML page generated by ``--qr-html``. It works with existing + "key export" files and key files. + +Other changes: + +- docs: + + - language clarification - "borg create --one-file-system" option does not respect + mount points, but considers different file systems instead, #2141 +- setup.py: build_api: sort file list for determinism + + Version 1.0.10rc1 (2017-01-29) ------------------------------ @@ -2887,173 +3289,6 @@ Other changes: - add windows virtual machine with cygwin - Vagrantfile cleanup / code deduplication -Version 1.1.0b2 (2016-10-01) ----------------------------- - -Bug fixes: - -- fix incorrect preservation of delete tags, leading to "object count mismatch" - on borg check, #1598. This only occurred with 1.1.0b1 (not with 1.0.x) and is - normally fixed by running another borg create/delete/prune. -- fix broken --progress for double-cell paths (e.g. CJK), #1624 -- borg recreate: also catch SIGHUP -- FUSE: - - - fix hardlinks in versions view, #1599 - - add parameter check to ItemCache.get to make potential failures more clear - -New features: - -- Archiver, RemoteRepository: add --remote-ratelimit (send data) -- borg help compression, #1582 -- borg check: delete chunks with integrity errors, #1575, so they can be - "repaired" immediately and maybe healed later. -- archives filters concept (refactoring/unifying older code) - - - covers --first/--last/--prefix/--sort-by options - - currently used for borg list/info/delete - -Other changes: - -- borg check --verify-data slightly tuned (use get_many()) -- change {utcnow} and {now} to ISO-8601 format ("T" date/time separator) -- repo check: log transaction IDs, improve object count mismatch diagnostic -- Vagrantfile: use TW's fresh-bootloader pyinstaller branch -- fix module names in api.rst -- hashindex: bump api_version - - -Version 1.1.0b1 (2016-08-28) ----------------------------- - -New features: - -- new commands: - - - borg recreate: re-create existing archives, #787 #686 #630 #70, also see - #757, #770. - - - selectively remove files/dirs from old archives - - re-compress data - - re-chunkify data, e.g. to have upgraded Attic / Borg 0.xx archives - deduplicate with Borg 1.x archives or to experiment with chunker-params. - - borg diff: show differences between archives - - borg with-lock: execute a command with the repository locked, #990 -- borg create: - - - Flexible compression with pattern matching on path/filename, - and LZ4 heuristic for deciding compressibility, #810, #1007 - - visit files in inode order (better speed, esp. for large directories and rotating disks) - - in-file checkpoints, #1217 - - increased default checkpoint interval to 30 minutes (was 5 minutes), #896 - - added uuid archive format tag, #1151 - - save mountpoint directories with --one-file-system, makes system restore easier, #1033 - - Linux: added support for some BSD flags, #1050 - - add 'x' status for excluded paths, #814 - - - also means files excluded via UF_NODUMP, #1080 -- borg check: - - - will not produce the "Checking segments" output unless new --progress option is passed, #824. - - --verify-data to verify data cryptographically on the client, #975 -- borg list, #751, #1179 - - - removed {formatkeys}, see "borg list --help" - - --list-format is deprecated, use --format instead - - --format now also applies to listing archives, not only archive contents, #1179 - - now supports the usual [PATH [PATHS…]] syntax and excludes - - new keys: csize, num_chunks, unique_chunks, NUL - - supports guaranteed_available hashlib hashes - (to avoid varying functionality depending on environment), - which includes the SHA1 and SHA2 family as well as MD5 -- borg prune: - - - to better visualize the "thinning out", we now list all archives in - reverse time order. rephrase and reorder help text. - - implement --keep-last N via --keep-secondly N, also --keep-minutely. - assuming that there is not more than 1 backup archive made in 1s, - --keep-last N and --keep-secondly N are equivalent, #537 - - cleanup checkpoints except the latest, #1008 -- borg extract: - - - added --progress, #1449 - - Linux: limited support for BSD flags, #1050 -- borg info: - - - output is now more similar to borg create --stats, #977 -- borg mount: - - - provide "borgfs" wrapper for borg mount, enables usage via fstab, #743 - - "versions" mount option - when used with a repository mount, this gives - a merged, versioned view of the files in all archives, #729 -- repository: - - - added progress information to commit/compaction phase (often takes some time when deleting/pruning), #1519 - - automatic recovery for some forms of repository inconsistency, #858 - - check free space before going forward with a commit, #1336 - - improved write performance (esp. for rotating media), #985 - - - new IO code for Linux - - raised default segment size to approx 512 MiB - - improved compaction performance, #1041 - - reduced client CPU load and improved performance for remote repositories, #940 - -- options that imply output (--show-rc, --show-version, --list, --stats, - --progress) don't need -v/--info to have that output displayed, #865 -- add archive comments (via borg (re)create --comment), #842 -- borg list/prune/delete: also output archive id, #731 -- --show-version: shows/logs the borg version, #725 -- added --debug-topic for granular debug logging, #1447 -- use atomic file writing/updating for configuration and key files, #1377 -- BORG_KEY_FILE environment variable, #1001 -- self-testing module, #970 - - -Bug fixes: - -- list: fixed default output being produced if --format is given with empty parameter, #1489 -- create: fixed overflowing progress line with CJK and similar characters, #1051 -- prune: fixed crash if --prefix resulted in no matches, #1029 -- init: clean up partial repo if passphrase input is aborted, #850 -- info: quote cmdline arguments that have spaces in them -- fix hardlinks failing in some cases for extracting subtrees, #761 - -Other changes: - -- replace stdlib hmac with OpenSSL, zero-copy decrypt (10-15% increase in - performance of hash-lists and extract). -- improved chunker performance, #1021 -- open repository segment files in exclusive mode (fail-safe), #1134 -- improved error logging, #1440 -- Source: - - - pass meta-data around, #765 - - move some constants to new constants module - - better readability and fewer errors with namedtuples, #823 - - moved source tree into src/ subdirectory, #1016 - - made borg.platform a package, #1113 - - removed dead crypto code, #1032 - - improved and ported parts of the test suite to py.test, #912 - - created data classes instead of passing dictionaries around, #981, #1158, #1161 - - cleaned up imports, #1112 -- Docs: - - - better help texts and sphinx reproduction of usage help: - - - Group options - - Nicer list of options in Sphinx - - Deduplicate 'Common options' (including --help) - - chunker: added some insights by "Voltara", #903 - - clarify what "deduplicated size" means - - fix / update / add package list entries - - added a SaltStack usage example, #956 - - expanded FAQ - - new contributors in AUTHORS! -- Tests: - - - vagrant: add ubuntu/xenial 64bit - this box has still some issues - - ChunkBuffer: add test for leaving partial chunk in buffer, fixes #945 - Version 1.0.8 (2016-10-29) --------------------------