Commit Graph

7322 Commits

Author SHA1 Message Date
Thomas Waldmann 9315ecb89e update CHANGES 2022-09-28 15:20:15 +02:00
TW f2fe22522d
Merge pull request #7052 from ThomasWaldmann/fix-upgrader-user-group-none
transfer: fix user/group == None crash with borg1 archives
2022-09-28 14:35:40 +02:00
TW b8f86abaa3
Merge pull request #7059 from ThomasWaldmann/ci-py311-fuse3
CI: test pyfuse3 with python 3.11
2022-09-28 13:14:39 +02:00
Thomas Waldmann c3e5ff613f CI: test pyfuse3 with python 3.11 2022-09-28 02:57:20 +02:00
TW 1a89db0a70
Merge pull request #7058 from ThomasWaldmann/fix-memoryview-crash
avoid memoryview related TypeError in compressors
2022-09-28 00:52:55 +02:00
Thomas Waldmann 515833c664 avoid memoryview related TypeError in compressors
fixes:

https://github.com/borgbackup/borg/discussions/7020#discussioncomment-3741024
2022-09-27 23:41:21 +02:00
TW 58c375678a
Merge pull request #7055 from ThomasWaldmann/borg-lock-wait-env-var
BORG_LOCK_WAIT=n env var to set default for --lock-wait option, fixes #5279
2022-09-27 19:41:14 +02:00
TW ba7b9cfd30
Merge pull request #7037 from ThomasWaldmann/rcompress
rcompress: do a repo-wide (re)compression
2022-09-27 19:13:58 +02:00
Thomas Waldmann e57ff29f22 BORG_LOCK_WAIT=n env var to set default for --lock-wait option, fixes #5279 2022-09-27 17:09:19 +02:00
Thomas Waldmann 9455c1e278 rcompress: do a repo-wide (re)compression
reads all chunks in on-disk order and recompresses them if they are not already using
the desired compression type and level (and obfuscation level).

supports SIGINT/ctrl-c and --checkpoint-interval (default: 1800s).

this is a borg command that compacts when committing (without this, it would have
a huge space usage). it commits/compacts every checkpoint interval or when
pressing ctrl-c / receiving SIGINT.
2022-09-27 16:50:19 +02:00
Thomas Waldmann 803b2a8d38 remember obfuscation level 2022-09-27 11:39:32 +02:00
TW 78b1301b98
Merge pull request #7028 from ThomasWaldmann/match-archives
implement pattern support for --match-archives, fixes #6504
2022-09-27 11:38:27 +02:00
Thomas Waldmann b71ab084ba transfer: fix user/group == None crash with borg1 archives 2022-09-26 23:00:29 +02:00
TW 098d80f843
Merge pull request #7043 from ThomasWaldmann/misc-fixes
misc fixes
2022-09-26 17:32:50 +02:00
TW 600606897e
Merge pull request #7044 from ThomasWaldmann/repoobj-fixes
repoobj: mutate meta + misc minor fixes / updates
2022-09-21 11:55:41 +02:00
Thomas Waldmann 55b137bdae repoobj: consistency / cleanups 2022-09-21 11:19:40 +02:00
Thomas Waldmann acc5d9870a repoobj.format: mutate meta
we should modify the meta dict given by the caller, so the caller can know
about e.g. the compression/obfuscation that was done (this is useful for rcompress).
2022-09-21 11:19:40 +02:00
Thomas Waldmann a07c93d7f4 add assertion to key_factory, fail early with zero-len manifest data 2022-09-21 09:06:59 +02:00
Thomas Waldmann 8c9fed105d hashindex: make NSIndex1 api compatible
some new stuff is not supported for NSIndex1,
but we can avoid crashing due to function signature mismatches or
missing methods and rather have more clear exceptions.
2022-09-21 08:56:37 +02:00
TW d671707226
Merge pull request #7042 from ThomasWaldmann/fix3-scan
repository.scan: fix start_offset
2022-09-20 21:55:34 +02:00
Thomas Waldmann b3c7d8f5f3 repository.scan: fix start_offset
start_offset (from state) is only valid for start_segment, otherwise we must use 0.
2022-09-20 21:25:36 +02:00
TW 63f736be4f
Merge pull request #7040 from ThomasWaldmann/fix2-scan
repository.scan: use same end_segment within same scan
2022-09-19 22:47:40 +02:00
Thomas Waldmann c4e54ca44e repository.scan: use same end_segment within same scan
achieved by putting it into the state that is now used instead of the marker.
2022-09-19 21:14:25 +02:00
TW c0e674ce61
Merge pull request #7038 from ThomasWaldmann/fix-scan
Fix repository.scan
2022-09-19 12:28:09 +02:00
Thomas Waldmann 49a4884cfe repository.scan: do not use chunkid as marker, but (segment, offset)
when using .scan(limit, marker), we used to use the last chunkid from
the previously returned scan result to remember how far we got and
from where we need to continue.

as this approach used the repo index to look up the respective segment/offset,
it was problematic if the code using scan was re-writing the chunk to
a new segment/offset, updating the repo index (e.g. when recompressing a chunk)
and basically destroying the memory about from where we need to continue
scanning.

thus, directly returning (segment, offset) as marker is easier and solves this issue.
2022-09-19 12:03:13 +02:00
Thomas Waldmann ce08f92090 repository.scan: only iterate over committed chunks
otherwise, if we scan+get+put (e.g. if we read/modify/write chunks to
recompress them), it would scan past the last commit and run into the
newly written chunks (and potentially never terminate).
2022-09-19 11:05:07 +02:00
TW 7e3b6752cb
Merge pull request #7036 from ThomasWaldmann/fix-7034-master
check: fix uninitialised variable if repo is completely empty, fixes #7034
2022-09-17 21:22:43 +02:00
Thomas Waldmann 78de77e67a check: fix uninitialised variable if repo is completely empty, fixes #7034 2022-09-17 20:54:13 +02:00
TW d1cd017bd5
Merge pull request #7032 from ThomasWaldmann/compat-setuptoolsscm4-master
do not use version_tuple placeholder in setuptools_scm template
2022-09-17 13:54:16 +02:00
Thomas Waldmann 3d311143f3 do not use version_tuple placeholder in setuptools_scm template
that would require setuptools_scm>=5.0.0 but some dists do not have that yet.

also, we do not use the version_tuple from _version.py, so it is not required anyway.

forward port of #7024.
2022-09-17 13:37:38 +02:00
TW ca2cd9ad25
Merge pull request #7026 from ThomasWaldmann/empty-zero-unknown
metadata: differentiate between empty/zero and unknown
2022-09-16 21:52:31 +02:00
TW 2f803f83bf
Merge pull request #7030 from ThomasWaldmann/beta
beta status now
2022-09-16 21:52:07 +02:00
Thomas Waldmann dae1b9cd82 beta status now 2022-09-16 21:26:41 +02:00
Thomas Waldmann 1a6b60f415 mode, user/group id/name: minor code refactor, remove None values at transfer time, #6908
https://github.com/borgbackup/borg/issues/6908#issuecomment-1224910916
2022-09-16 21:12:29 +02:00
TW 16787e48b7
Merge pull request #7029 from ThomasWaldmann/relative-imports
Relative imports
2022-09-16 20:48:32 +02:00
Thomas Waldmann 785cb4c21a __main__: remove absolute "borg" import 2022-09-16 19:37:54 +02:00
Thomas Waldmann f348c86f08 mount cmd tests: remove absolute "borg" import 2022-09-16 19:09:46 +02:00
Thomas Waldmann 4493d396e6 implement pattern support for --match-archives, fixes #6504
also:
- rename --glob-archives option to --match-archives (short: -a, unchanged)
- globbing patterns now need sh: prefix
- regex patterns need re: prefix
- "identical" match "patterns" use an id: prefix
- new default style is id: pattern (--glob-archives used sh: glob pattern)
- source code: glob -> match, GLOB -> PATTERN
2022-09-16 15:10:13 +02:00
Thomas Waldmann 6e2419f3b2 timestamps: minor code refactor, nothing else to do, #6908
https://github.com/borgbackup/borg/issues/6908#issuecomment-1224886207
2022-09-14 18:19:35 +02:00
Thomas Waldmann 3280603e43 acls cleanup (linux), #6908
https://github.com/borgbackup/borg/issues/6908#issuecomment-1224872992

also:
- added some type assertions
- made NULL pointer checks more explicit
- fsencode only called when needed
- structure of code more similar now on all OSes
2022-09-14 13:57:40 +02:00
Thomas Waldmann 6a1c64b0dc xattrs cleanup, #6908
https://github.com/borgbackup/borg/issues/6908#issuecomment-1224870018
2022-09-14 13:57:40 +02:00
Thomas Waldmann 287907b218 bsdflags cleanup, #6908
https://github.com/borgbackup/borg/issues/6908#issuecomment-1224839170
2022-09-14 11:24:50 +02:00
TW f5df35b36e
Merge pull request #7022 from ThomasWaldmann/split-archiver-tests
Split archiver tests
2022-09-14 09:51:14 +02:00
Thomas Waldmann 5d7b48ced8 add remote repo / borg binary testing 2022-09-14 09:11:45 +02:00
Thomas Waldmann f410de690d remove log output checks
the intention of this test is testing whether borg check
returns an error when checking a corrupted repository.

the removed assertions were rather testing the test logging
configuration, which seems flaky:

- when running all tests, assertions failed
- when running only this one test, assertions succeeded
- assertions also succeeded when running all the tests before
  they were refactored to separate test modules, although the
  test code was not changed, just moved.
2022-09-13 16:22:16 +02:00
Thomas Waldmann 1cd105d9f5 update flake8 ignores in setup.cfg 2022-09-13 14:58:46 +02:00
Thomas Waldmann f94eec199f move patterns tests to own module 2022-09-13 14:18:17 +02:00
Thomas Waldmann 87c3f11796 remove unneeded code 2022-09-13 14:14:56 +02:00
Thomas Waldmann db0d15c182 move another test to argparsing 2022-09-13 14:12:15 +02:00
Thomas Waldmann 6ec6fc2496 move corruption tests to own module 2022-09-13 14:08:03 +02:00