1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2025-02-26 16:04:06 +00:00
Commit graph

7372 commits

Author SHA1 Message Date
Thomas Waldmann
64cc16a9f4 Item: fix xattr processing
Item.xattrs is now always a StableDict mapping bytes keys -> bytes values.

The special casing of empty values (b'') getting replaced by None was removed.
2022-06-09 17:57:28 +02:00
Thomas Waldmann
9d684120a2 Item: assert type also in property getter
also: fixed Item.xattrs to be StableDict (not just a dict, as the
msgpack unpacker gives us)
2022-06-09 17:57:28 +02:00
Thomas Waldmann
7b138cc710 Item: convert timestamps once, get rid of bigint code 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e58525fc6 Item: remove some decode= params
update_internal() makes sure they have the desired type already.
2022-06-09 17:57:28 +02:00
Thomas Waldmann
655c1b9cc2 update docstrings / comments 2022-06-09 17:57:28 +02:00
Thomas Waldmann
33444be926 more str vs bytes fixing 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e87f1111b cleanup msgpack related str/bytes mess, fixes #968
see ticket and borg.helpers.msgpack docstring.

this changeset implements the full migration to
msgpack 2.0 spec (use_bin_type=True, raw=False).

still needed compat to the past is done via want_bytes decoder in borg.item.
2022-06-09 17:57:28 +02:00
Thomas Waldmann
f8dbe5b542 cleanup msgpack related str/bytes mess, see #968
see ticket and borg.helpers.msgpack docstring.
2022-06-09 17:57:28 +02:00
TW
86fe8bdd57
Merge pull request #6703 from ThomasWaldmann/r2r-transfer
borg2: "borg transfer" cmd (and also getting rid of legacy)
2022-06-09 17:56:03 +02:00
Thomas Waldmann
c5540c2dd9 upgrade compressed chunk: fix treatment of ObfuscateSize chunks
the inner payload of ObfuscateSize chunks are compressed chunks and need
the same zlib fix and level patching as non-obfuscated compressed chunks.
2022-06-09 17:49:16 +02:00
Thomas Waldmann
72c68c49d0 obfuscation: fix byte order for size, fixes #6701 2022-06-09 17:49:16 +02:00
Thomas Waldmann
6584a92c81 compression: use the 2 bytes for type and level, fixes #6698
adapt borg transfer, transferred chunks are set to compression level "unknown".
2022-06-09 17:49:16 +02:00
Thomas Waldmann
32a3601e4a compute hlid from inode / device 2022-06-09 17:49:16 +02:00
Thomas Waldmann
8798b0340a use whitelist approach to make sure item._dict is clean 2022-06-09 17:49:16 +02:00
Thomas Waldmann
d3dfa3be30 use version 2 for new archives
but still be able to read v1 archives
for borg transfer.
2022-06-09 17:49:16 +02:00
TW
0e8c9941bb
Merge pull request #6747 from fantasya-pbem/docs/4776_FAW-swapped-targets
[DOCS] #4776 - FAQ: How to swap backup media
2022-06-05 18:20:36 +02:00
TW
eed5038922
Merge pull request #6750 from ThomasWaldmann/update-changes-rel1118-master
update CHANGES with 1.1.18 changelog
2022-06-05 18:09:03 +02:00
Thalian
c7c98ca0fd [DOCS] #4776 - FAQ: How to swap backup media
fixes #4776
2022-06-05 18:05:16 +02:00
Thomas Waldmann
7634426195 update CHANGES with 1.1.18 changelog 2022-06-05 17:47:41 +02:00
TW
4ff0a29209
Merge pull request #6741 from fantasya-pbem/docs/5310_overhaul-help-patterns
docs: overhaul borg help patterns, fixes #5310
2022-06-05 10:36:33 +02:00
Thalian
421a7ef52f [DOCS] #5310 - Overhaul borg help patterns
fixes #5310
2022-06-05 09:02:04 +02:00
TW
34bb4d904b
Merge pull request #6743 from ThomasWaldmann/msgpack104-master
msgpack 1.0.4 / py 3.11 (master)
2022-06-03 10:36:59 +02:00
Thomas Waldmann
c07afb26e4 add python 3.11 to pypi metadata 2022-06-03 10:05:04 +02:00
Thomas Waldmann
18a7debf75 CI: also test on python 3.11-dev 2022-06-03 10:05:04 +02:00
Thomas Waldmann
a970f000b0 allow msgpack 1.0.4, fixes #6716 2022-06-03 10:03:49 +02:00
TW
c4b05c18e0
Merge pull request #6739 from ThomasWaldmann/vagrant-updates-master
Vagrant updates (master)
2022-06-01 20:31:16 +02:00
Thomas Waldmann
de4b9198c3 vagrant: use python 3.9.13 for binary build 2022-06-01 19:56:52 +02:00
Thomas Waldmann
c00c48a27b vagrant: use pyinstaller 4.10
when installed via pip, this automatically build the bootloader now.
2022-06-01 19:54:48 +02:00
Thomas Waldmann
bd005c11a2 vagrant/testing: upgrade development.lock.txt
esp. the Cython version upgrade to 0.29.30 is important for python 3.11.
2022-06-01 19:54:11 +02:00
Elmar Hoffmann
c2317c4cce
make constants for files cache mode more clear (#6724)
* make constants for files cache mode more clear

Traditionally, DEFAULT_FILES_CACHE_MODE_UI and DEFAULT_FILES_CACHE_MODE
were - as the naming scheme implies - the same setting, one being the UI
representation as given to the --files-cache command line option and the
other being the same default value in the internal representation.

It happended that the actual value used in borg create always comes from
DEFAULT_FILES_CACHE_MODE_UI (because that does have the --files-cache
option) whereas for all other commands (that do not use the files cache) it
comes from DEFAULT_FILES_CACHE_MODE.

PR #5777 then abused this fact to implement the optimisation to skip loading
of the files cache in those other commands by changing the value of
DEFAULT_FILES_CACHE_MODE to disabled.
This however also changes the meaning of that variable and thus redesignates
it to something not matching the original naming anymore.

Anyone not aware of this change and the intention behind it looking at the
code would have a hard time figuring this out and be easily mislead.

This does away with the confusion making the code more maintainable by
renaming DEFAULT_FILES_CACHE_MODE to FILES_CACHE_MODE_DISABLED, making the
new intention of that internal default clear.

* make constant for files cache mode UI default match naming scheme
2022-05-30 14:01:19 +02:00
TW
d064f63ad7
Merge pull request #6726 from elho/use-relative-imports
Use relative imports
2022-05-29 14:05:23 +02:00
Elmar Hoffmann
fd34fa2d02 use relative imports
Use relative imports where trivially possible for more consistency and to
avoid using the borg module name explicitly.
2022-05-29 09:36:52 +02:00
Elmar Hoffmann
c34df51e3e import IntegrityError used as base class with according name
This not only brings code style in line with the other helpers that do the
same thing this way, but also does away with an unnecessary absolute import
using the borg module name explicitly.
2022-05-28 16:07:20 +02:00
TW
e8046bc761
Merge pull request #6723 from KN4CK3R/forwardport-6722
Forwardport: borg debug dump-repo-objs --ghost (#6722)
2022-05-28 13:37:56 +02:00
TW
7b08222256 Merge pull request #6722 from ThomasWaldmann/debug-get-chunk-1.2
borg debug dump-repo-objs --ghost: new --segment=S --offset=O options
2022-05-28 01:42:26 +02:00
TW
1614abd4b9
Merge pull request #6719 from fantasya-pbem/task/6407_document-pattern-changes
#6407 - Document Borg 1.2 pattern behavior change
2022-05-27 15:23:15 +02:00
Thalian
45fac2dc54 #6407 - Document Borg 1.2 pattern behavior change
Fix wrong root path that was accidently changed in last commit.
2022-05-27 15:17:43 +02:00
TW
b209fcce12
Merge pull request #6717 from fantasya-pbem/task/6407_document-pattern-changes
docs: document borg 1.2 pattern behaviour change, fixes #6407
2022-05-27 15:04:43 +02:00
Thalian
b4d3859b9e #6407 - Document Borg 1.2 pattern behavior change
Make clear that absolute paths always go into the matcher as if they are relative (without leading slash). Adapt all examples accordingly.

fixes #6407
2022-05-27 14:13:12 +02:00
TW
b10d661185
Merge pull request #6706 from ThomasWaldmann/revert-no-scp-in-master
Revert "Remove scp syntax for locations (#6697)"
2022-05-18 17:58:56 +02:00
Thomas Waldmann
61299d23db Revert "Remove scp syntax for locations (#6697)"
This reverts commit 1b4b84dfd8.
2022-05-18 17:31:10 +02:00
Thomas Waldmann
e5f1a4fb4d recreate: cachedir_masters not needed any more
now all hardlinked regular file items have chunks.
2022-05-18 14:20:01 +02:00
Thomas Waldmann
6bfdb3f630 refactor hardlink_master processing globally
borg now has the chunks list in every item with content.
due to the symmetric way how borg now deals with hardlinks using
item.hlid, processing gets much simpler.

but some places where borg deals with other "sources" of hardlinks
still need to do some hardlink management:
borg uses the HardLinkManager there now (which is not much more
than a dict, but keeps documentation at one place and avoids some
code duplication we had before).

item.hlid is computed via hardlink_id function.

support hardlinked symlinks, fixes #2379
as we use item.hlid now to group hardlinks together,
there is no conflict with the item.source usage for
symlink targets any more.

2nd+ hardlinks now add to the files count as did the 1st one.
for borg, now all hardlinks are created equal.
so any hardlink item with chunks now adds to the "file" count.

ItemFormatter: support {hlid} instead of {source} for hardlinks
2022-05-18 14:20:01 +02:00
Thomas Waldmann
7903dad183 transfer: convert timestamps int/bigint -> msgpack.Timestamp, see #2323
Timestamp scales to 64 or 96bit serialization formats, that should be enough for everybody.

We use this in archived items and also in the files cache.
2022-05-18 14:20:01 +02:00
Thomas Waldmann
e4a97ea8cc transfer: all hardlinks have chunks, maybe chunks_healty, hlid
Item.hlid: same id, same hardlink (xxh64 digest)
Item.hardlink_master: not used for new archives any more
Item.source: not used for hardlink slaves any more
2022-05-18 14:20:01 +02:00
Thomas Waldmann
01f72d15b4 transfer: remove the zlib type bytes hack
hack: see the docstring of ZLIB_legacy class.

New clean ZLIB class that works as every other compressor.

ZLIB ID 0x0500, ZLIB_legacy ID 0x.8..
2022-05-18 14:20:01 +02:00
Thomas Waldmann
ba1dbe6111 transfer: make sure items with chunks have precomputed size 2022-05-18 14:20:01 +02:00
Thomas Waldmann
98b7dc0bf5 transfer: clean item of attic 0.13 'acl' bug remnants
also: remove attic bug support code from borg check.

borg transfer removes the acl key. we do not run borg check on old repos.
2022-05-18 14:20:00 +02:00
Thomas Waldmann
116f67036f transfer: copy archives from another repo
this is somehow similar to borg recreate,
but with different focus and way simpler:

not changing compression algo
not changing chunking
not excluding files inside an archive by path match
only dealing with complete archives

but:
different src and dst repo
only reading each chunk once
keeping the compressed payload (no decompression/recompression effort)
--dry-run can be used before and afterwards to check
2022-05-18 14:20:00 +02:00
Emil M George
1b4b84dfd8
Remove scp syntax for locations (#6697)
remove scp syntax support from cli and from Location parser, add note about converting scp-style URLs to ssh-style, fixes #6691
2022-05-15 21:55:19 +02:00