From 6043d8b25722ec5db27a3a8eec5780711cefcadc Mon Sep 17 00:00:00 2001 From: Thomas Waldmann Date: Tue, 19 Dec 2017 02:19:15 +0100 Subject: [PATCH] update CHANGES (1.1-maint) --- docs/changes.rst | 89 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/docs/changes.rst b/docs/changes.rst index 2a2cc1a40..3569984d5 100644 --- a/docs/changes.rst +++ b/docs/changes.rst @@ -6,6 +6,46 @@ Important notes This section provides information about security and corruption issues. +.. _broken_validator: + +Pre-1.1.4 potential data corruption issue +----------------------------------------- + +A data corruption bug was discovered in borg check --repair, see issue #3444. + +This is a 1.1.x regression, releases < 1.1 (e.g. 1.0.x) are not affected. + +To avoid data loss, you must not run borg check --repair using an unfixed version +of borg 1.1.x. The first official release that has the fix is 1.1.4. + +Package maintainers may have applied the fix to updated packages of 1.1.x (x<4) +though, see the package maintainer's package changelog to make sure. + +If you never had missing item metadata chunks, the bug has not affected you +even if you did run borg check --repair with an unfixed version. + +When borg check --repair tried to repair corrupt archives that miss item metadata +chunks, the resync to valid metadata in still present item metadata chunks +malfunctioned. This was due to a broken validator that considered all (even valid) +item metadata as invalid. As they were considered invalid, borg discarded them. +Practically, that means the affected files, directories or other fs objects were +discarded from the archive. + +Due to the malfunction, the process was extremely slow, but if you let it +complete, borg would have created a "repaired" archive that has lost a lot of items. +If you interrupted borg check --repair because it was so strangely slow (killing +borg somehow, e.g. Ctrl-C) the transaction was rolled back and no corruption occurred. + +The log message indicating the precondition for the bug triggering looks like: + + item metadata chunk missing [chunk: 001056_bdee87d...a3e50d] + +If you never had that in your borg check --repair runs, you're not affected. + +But if you're unsure or you actually have seen that, better check your archives. +By just using "borg list repo::archive" you can see if all expected filesystem +items are listed. + .. _tam_vuln: Pre-1.0.9 manifest spoofing vulnerability (CVE-2016-10099) @@ -131,8 +171,8 @@ The best check that everything is ok is to run a dry-run extraction:: Changelog ========= -Version 1.1.3 (2017-11-27) --------------------------- +Version 1.1.4 (not released yet) +-------------------------------- Compatibility notes: @@ -150,6 +190,51 @@ Compatibility notes: You can avoid the one-time slowdown by using the pre-1.1.0rc4-compatible mode (but that is less safe for detecting changed files than the default). See the --files-cache docs for details. +- borg 1.1.4 changes: + + - zstd compression is new in borg 1.1.4, older borg can't handle it. + - new minimum requirements for the compression libraries - if the required + versions (header and lib) can't be found at build time, bundled code will + be used: + + - added: libzstd >= 1.3.0 (bundled: 1.3.2) + - updated: liblz4 >= 1.7.0 / r129 (bundled: 1.8.0) + +Fixes: + +- data corruption fix: fix for borg check --repair malfunction, #3444. + See the more detailled notes close to the top of this document. +- also delete security dir when deleting a repo, #3427 +- fix building the "borg prune" man page, #3398 + +New features: + +- added zstd compression. try it! +- added placeholder for fqdn in reverse notation + +Other changes: + +- list help topics when invalid topic is requested +- add auto-generated docs for borg config +- don't generate HTML docs page for borgfs, #3404 +- refactored build of the compress and crypto.low_level extensions, #3415: + + - move some lib/build related code to setup_{zstd,lz4,b2}.py + - bundle lz4 1.8.0 (requirement: >= 1.7.0 / r129) + - bundle zstd 1.3.2 (requirement: >= 1.3.0) + - blake2 was already bundled + - rename BORG_LZ4_PREFIX env var to BORG_LIBLZ4_PREFIX for better consistency: + we also have BORG_LIBB2_PREFIX and BORG_LIBZSTD_PREFIX now. + - add prefer_system_lib* = True settings to setup.py - by default the build + will prefer a shared library over the bundled code, if library and headers + can be found and meet the minimum requirements. +- fix lz4 deprecation warning, requires lz4 >= 1.7.0 (r129) +- add parens for C preprocessor macro argument usages (did not cause malfunction) +- exclude broken pytest 3.3.0 release + + +Version 1.1.3 (2017-11-27) +-------------------------- Fixes: