mirror of
https://github.com/borgbackup/borg.git
synced 2024-12-24 16:55:36 +00:00
Merge pull request #5018 from ThomasWaldmann/update-changes-from-1.1
update changes.rst from 1.1-maint branch, fixes #5006
This commit is contained in:
TW
GitHub
commit
425f0919df
1 changed files with 230 additions and 0 deletions
230
docs/changes.rst
230
docs/changes.rst
|
|
@ -6,6 +6,53 @@ Important notes
|
||||||
|
|
||||||
This section provides information about security and corruption issues.
|
This section provides information about security and corruption issues.
|
||||||
|
|
||||||
|
.. _hashindex_set_bug:
|
||||||
|
|
||||||
|
Pre-1.1.11 potential index corruption / data loss issue
|
||||||
|
-------------------------------------------------------
|
||||||
|
|
||||||
|
A bug was discovered in our hashtable code, see issue #4829.
|
||||||
|
The code is used for the client-side chunks cache and the server-side repo index.
|
||||||
|
|
||||||
|
Although borg uses the hashtables very heavily, the index corruption did not
|
||||||
|
happen too frequently, because it needed specific conditions to happen.
|
||||||
|
|
||||||
|
Data loss required even more specific conditions, so it should be rare (and
|
||||||
|
also detectable via borg check).
|
||||||
|
|
||||||
|
You might be affected if borg crashed with / complained about:
|
||||||
|
|
||||||
|
- AssertionError: Corrupted segment reference count - corrupted index or hints
|
||||||
|
- ObjectNotFound: Object with key ... not found in repository ...
|
||||||
|
- Index mismatch for key b'...'. (..., ...) != (-1, -1)
|
||||||
|
- ValueError: stats_against: key contained in self but not in master_index.
|
||||||
|
|
||||||
|
Advised procedure to fix any related issue in your indexes/caches:
|
||||||
|
|
||||||
|
- install fixed borg code (on client AND server)
|
||||||
|
- for all of your clients and repos remove the cache by:
|
||||||
|
|
||||||
|
borg delete --cache-only YOURREPO
|
||||||
|
|
||||||
|
(later, the cache will be re-built automatically)
|
||||||
|
- for all your repos, rebuild the repo index by:
|
||||||
|
|
||||||
|
borg check --repair YOURREPO
|
||||||
|
|
||||||
|
This will also check all archives and detect if there is any data-loss issue.
|
||||||
|
|
||||||
|
Affected branches / releases:
|
||||||
|
|
||||||
|
- fd06497 introduced the bug into 1.1-maint branch - it affects all borg 1.1.x since 1.1.0b4.
|
||||||
|
- fd06497 introduced the bug into master branch - it affects all borg 1.2.0 alpha releases.
|
||||||
|
- c5cd882 introduced the bug into 1.0-maint branch - it affects all borg 1.0.x since 1.0.11rc1.
|
||||||
|
|
||||||
|
The bug was fixed by:
|
||||||
|
|
||||||
|
- 701159a fixes the bug in 1.1-maint branch - will be released with borg 1.1.11.
|
||||||
|
- fa63150 fixes the bug in master branch - will be released with borg 1.2.0a8.
|
||||||
|
- 7bb90b6 fixes the bug in 1.0-maint branch. Branch is EOL, no new release is planned as of now.
|
||||||
|
|
||||||
.. _broken_validator:
|
.. _broken_validator:
|
||||||
|
|
||||||
Pre-1.1.4 potential data corruption issue
|
Pre-1.1.4 potential data corruption issue
|
||||||
|
|
@ -470,6 +517,189 @@ Other changes:
|
||||||
- vagrant: new VMs for linux/bsd/darwin, most with OpenSSL 1.1 and py36
|
- vagrant: new VMs for linux/bsd/darwin, most with OpenSSL 1.1 and py36
|
||||||
|
|
||||||
|
|
||||||
|
Version 1.1.11 (2020-03-08)
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Compatibility notes:
|
||||||
|
|
||||||
|
- When upgrading from borg 1.0.x to 1.1.x, please note:
|
||||||
|
|
||||||
|
- read all the compatibility notes for 1.1.0*, starting from 1.1.0b1.
|
||||||
|
- borg upgrade: you do not need to and you also should not run it.
|
||||||
|
- borg might ask some security-related questions once after upgrading.
|
||||||
|
You can answer them either manually or via environment variable.
|
||||||
|
One known case is if you use unencrypted repositories, then it will ask
|
||||||
|
about a unknown unencrypted repository one time.
|
||||||
|
- your first backup with 1.1.x might be significantly slower (it might
|
||||||
|
completely read, chunk, hash a lot files) - this is due to the
|
||||||
|
--files-cache mode change (and happens every time you change mode).
|
||||||
|
You can avoid the one-time slowdown by using the pre-1.1.0rc4-compatible
|
||||||
|
mode (but that is less safe for detecting changed files than the default).
|
||||||
|
See the --files-cache docs for details.
|
||||||
|
- 1.1.11 removes WSL autodetection (Windows 10 Subsystem for Linux).
|
||||||
|
If WSL still has a problem with sync_file_range, you need to set
|
||||||
|
BORG_WORKAROUNDS=basesyncfile in the borg process environment to
|
||||||
|
work around the WSL issue.
|
||||||
|
|
||||||
|
Fixes:
|
||||||
|
|
||||||
|
- fixed potential index corruption / data loss issue due to bug in hashindex_set, #4829.
|
||||||
|
Please read and follow the more detailled notes close to the top of this document.
|
||||||
|
- upgrade bundled xxhash to 0.7.3, #4891.
|
||||||
|
0.7.2 is the minimum requirement for correct operations on ARMv6 in non-fixup
|
||||||
|
mode, where unaligned memory accesses cause bus errors.
|
||||||
|
0.7.3 adds some speedups and libxxhash 0.7.3 even has a pkg-config file now.
|
||||||
|
- upgrade bundled lz4 to 1.9.2
|
||||||
|
- upgrade bundled zstd to 1.4.4
|
||||||
|
- fix crash when upgrading erroneous hints file, #4922
|
||||||
|
- extract:
|
||||||
|
|
||||||
|
- fix KeyError for "partial" extraction, #4607
|
||||||
|
- fix "partial" extract for hardlinked contentless file types, #4725
|
||||||
|
- fix preloading for old (0.xx) remote servers, #4652
|
||||||
|
- fix confusing output of borg extract --list --strip-components, #4934
|
||||||
|
- delete: after double-force delete, warn about necessary repair, #4704
|
||||||
|
- create: give invalid repo error msg if repo config not found, #4411
|
||||||
|
- mount: fix FUSE mount missing st_birthtime, #4763 #4767
|
||||||
|
- check: do not stumble over invalid item key, #4845
|
||||||
|
- info: if the archive doesn't exist, print a pretty message, #4793
|
||||||
|
- SecurityManager.known(): check all files, #4614
|
||||||
|
- Repository.open: use stat() to check for repo dir, #4695
|
||||||
|
- Repository.check_can_create_repository: use stat() to check, #4695
|
||||||
|
- fix invalid archive error message
|
||||||
|
- fix optional/non-optional location arg, #4541
|
||||||
|
- commit-time free space calc: ignore bad compact map entries, #4796
|
||||||
|
- ignore EACCES (errno 13) when hardlinking the old config, #4730
|
||||||
|
- --prefix / -P: fix processing, avoid argparse issue, #4769
|
||||||
|
|
||||||
|
New features:
|
||||||
|
|
||||||
|
- enable placeholder usage in all extra archive arguments
|
||||||
|
- new BORG_WORKAROUNDS mechanism, basesyncfile, #4710
|
||||||
|
- recreate: support --timestamp option, #4745
|
||||||
|
- support platforms without os.link (e.g. Android with Termux), #4901.
|
||||||
|
if we don't have os.link, we just extract another copy instead of making a hardlink.
|
||||||
|
- support linux platforms without sync_file_range (e.g. Android 7 with Termux), #4905
|
||||||
|
|
||||||
|
Other:
|
||||||
|
|
||||||
|
- ignore --stats when given with --dry-run, but continue, #4373
|
||||||
|
- add some ProgressIndicator msgids to code / fix docs, #4935
|
||||||
|
- elaborate on "Calculating size" message
|
||||||
|
- argparser: always use REPOSITORY in metavar, also use more consistent help phrasing.
|
||||||
|
- check: improve error output for matching index size, see #4829
|
||||||
|
- docs:
|
||||||
|
|
||||||
|
- changelog: add advisory about hashindex_set bug #4829
|
||||||
|
- better describe BORG_SECURITY_DIR, BORG_CACHE_DIR, #4919
|
||||||
|
- infos about cache security assumptions, #4900
|
||||||
|
- add FAQ describing difference between a local repo vs. repo on a server.
|
||||||
|
- document how to test exclusion patterns without performing an actual backup
|
||||||
|
- timestamps in the files cache are now usually ctime, #4583
|
||||||
|
- fix bad reference to borg compact (does not exist in 1.1), #4660
|
||||||
|
- create: borg 1.1 is not future any more
|
||||||
|
- extract: document limitation "needs empty destination", #4598
|
||||||
|
- how to supply a passphrase, use crypto devices, #4549
|
||||||
|
- fix osxfuse github link in installation docs
|
||||||
|
- add example of exclude-norecurse rule in help patterns
|
||||||
|
- update macOS Brew link
|
||||||
|
- add note about software for automating backups, #4581
|
||||||
|
- AUTHORS: mention copyright+license for bundled msgpack
|
||||||
|
- fix various code blocks in the docs, #4708
|
||||||
|
- updated docs to cover use of temp directory on remote, #4545
|
||||||
|
- add restore docs, #4670
|
||||||
|
- add a pull backup / push restore how-to, #1552
|
||||||
|
- add FAQ how to retain original paths, #4532
|
||||||
|
- explain difference between --exclude and --pattern, #4118
|
||||||
|
- add FAQs for SSH connection issues, #3866
|
||||||
|
- improve password FAQ, #4591
|
||||||
|
- reiterate that 'file cache names are absolute' in FAQ
|
||||||
|
- tests:
|
||||||
|
|
||||||
|
- cope with ANY error when importing pytest into borg.testsuite, #4652
|
||||||
|
- fix broken test that relied on improper zlib assumptions
|
||||||
|
- test_fuse: filter out selinux xattrs, #4574
|
||||||
|
- travis / vagrant:
|
||||||
|
|
||||||
|
- misc python versions removed / changed (due to openssl 1.1 compatibility)
|
||||||
|
or added (3.7 and 3.8, for better borg compatibility testing)
|
||||||
|
- binary building is on python 3.5.9 now
|
||||||
|
- vagrant:
|
||||||
|
|
||||||
|
- add new boxes: ubuntu 18.04 and 20.04, debian 10
|
||||||
|
- update boxes: openindiana, darwin, netbsd
|
||||||
|
- remove old boxes: centos 6
|
||||||
|
- darwin: updated osxfuse to 3.10.4
|
||||||
|
- use debian/ubuntu pip/virtualenv packages
|
||||||
|
- rather use python 3.6.2 than 3.6.0, fixes coverage/sqlite3 issue
|
||||||
|
- use requirements.d/development.lock.txt to avoid compat issues
|
||||||
|
- travis:
|
||||||
|
|
||||||
|
- darwin: backport some install code / order from master
|
||||||
|
- remove deprecated keyword "sudo" from travis config
|
||||||
|
- allow osx builds to fail, #4955
|
||||||
|
this is due to travis-ci frequently being so slow that the OS X builds
|
||||||
|
just fail because they exceed 50 minutes and get killed by travis.
|
||||||
|
|
||||||
|
|
||||||
|
Version 1.1.10 (2019-05-16)
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Fixes:
|
||||||
|
|
||||||
|
- extract: hang on partial extraction with ssh: repo, when hardlink master
|
||||||
|
is not matched/extracted and borg hangs on related slave hardlink, #4350
|
||||||
|
- lrucache: regularly remove old FDs, #4427
|
||||||
|
- avoid stale filehandle issues, #3265
|
||||||
|
- freebsd: make xattr platform code api compatible with linux, #3952
|
||||||
|
- use whitelist approach for borg serve, #4097
|
||||||
|
- borg command shall terminate with rc 2 for ImportErrors, #4424
|
||||||
|
- create: only run stat_simple_attrs() once, this increases
|
||||||
|
backup with lots of unchanged files performance by ~ 5%.
|
||||||
|
- prune: fix incorrect borg prune --stats output with --dry-run, #4373
|
||||||
|
- key export: emit user-friendly error if repo key is exported to a directory,
|
||||||
|
#4348
|
||||||
|
|
||||||
|
New features:
|
||||||
|
|
||||||
|
- bundle latest supported msgpack-python release (0.5.6), remove msgpack-python
|
||||||
|
from setup.py install_requires - by default we use the bundled code now.
|
||||||
|
optionally, we still support using an external msgpack (see hints in
|
||||||
|
setup.py), but this requires solid requirements management within
|
||||||
|
distributions and is not recommended.
|
||||||
|
borgbackup will break if you upgrade msgpack to an unsupported version.
|
||||||
|
- display msgpack version as part of sysinfo (e.g. in tracebacks)
|
||||||
|
- timestamp for borg delete --info added, #4359
|
||||||
|
- enable placeholder usage in --comment and --glob-archives, #4559, #4495
|
||||||
|
|
||||||
|
Other:
|
||||||
|
|
||||||
|
- serve: do not check python/libc for borg serve, #4483
|
||||||
|
- shell completions: borg diff second archive
|
||||||
|
- release scripts: signing binaries with Qubes OS support
|
||||||
|
- testing:
|
||||||
|
|
||||||
|
- vagrant: upgrade openbsd box to 6.4
|
||||||
|
- travis-ci: lock test env to py 3.4 compatible versions, #4343
|
||||||
|
- get rid of confusing coverage warning, #2069
|
||||||
|
- rename test_mount_hardlinks to test_fuse_mount_hardlinks,
|
||||||
|
so both can be excluded by "not test_fuse".
|
||||||
|
- pure-py msgpack warning shall not make a lot of tests fail, #4558
|
||||||
|
- docs:
|
||||||
|
|
||||||
|
- add "SSH Configuration" section to "borg serve", #3988, #636, #4485
|
||||||
|
- README: new URL for funding options
|
||||||
|
- add a sample logging.conf in docs/misc, #4380
|
||||||
|
- elaborate on append-only mode docs, #3504
|
||||||
|
- installation: added Alpine Linux to distribution list, #4415
|
||||||
|
- usage.html: only modify window.location when redirecting, #4133
|
||||||
|
- add msgpack license to docs/3rd_party/msgpack
|
||||||
|
- vagrant / binary builds:
|
||||||
|
|
||||||
|
- use python 3.5.7 for builds
|
||||||
|
- use osxfuse 3.8.3
|
||||||
|
|
||||||
|
|
||||||
Version 1.1.9 (2019-02-10)
|
Version 1.1.9 (2019-02-10)
|
||||||
--------------------------
|
--------------------------
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue