mirror/borg
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2024-12-25 17:27:31 +00:00
Code Issues Releases Wiki Activity
6852 commits 9 branches 139 tags 60 MiB
Commit graph

6852 commits

This branch
This branch
All branches
Author SHA1 Message Date
Thomas Waldmann
822aefc7b0 re-add dsize placeholder 2022-06-12 17:15:13 +02:00
Thomas Waldmann
49adb77157 calc_stats: deduplicated size now, was deduplicated csize 
also: remove pre12_meta cache
 2022-06-12 17:15:13 +02:00
Thomas Waldmann
19dfbe5c5c compute the deduplicated size before compression 
so we do not need csize for it.
 2022-06-12 17:15:13 +02:00
Thomas Waldmann
1fd571a4d0 fix comments 2022-06-12 17:15:13 +02:00
Thomas Waldmann
2c1f7951c4 remove csize from ChunkIndexEntry 2022-06-12 17:15:13 +02:00
Thomas Waldmann
b82a39c3b3 remove csize from stats_against() 2022-06-12 15:48:33 +02:00
Thomas Waldmann
0211948cac remove csize from summarize return tuple 2022-06-12 15:48:33 +02:00
Thomas Waldmann
b726aa5665 remove csize support from get_size 2022-06-12 15:48:33 +02:00
Thomas Waldmann
ace5957524 remove csize from item.chunks elements 2022-06-12 15:48:33 +02:00
Thomas Waldmann
b9f9623a6d prepare to remove csize (set it to 0 for now) 2022-06-12 15:48:33 +02:00
TW
2c9be35886
Merge pull request #6704 from ThomasWaldmann/msgpack-str-bytes-cleanup 
borg2: cleanup msgpack related str/bytes mess
 2022-06-09 18:18:13 +02:00
Thomas Waldmann
d4ee968b07 use borg 2.0 to refer to this, not 1.3 
also, some type conversions are now done in update_internal once,
not in the decode methods of the classes in item.pyx.
 2022-06-09 18:13:40 +02:00
Thomas Waldmann
08228fbd32 Item: remove unused hardlink_masters param 2022-06-09 17:57:28 +02:00
Thomas Waldmann
421d4bdfb0 docs: fix bytes -> str in data-structures docs 2022-06-09 17:57:28 +02:00
Thomas Waldmann
58009f6773 Key: fix once, remove decode=... 2022-06-09 17:57:28 +02:00
Thomas Waldmann
ed22f721f3 EncryptedKey: fix once, remove decode=... 2022-06-09 17:57:28 +02:00
Thomas Waldmann
f2b085787b Item: disallow None value for .user/group/chunks/chunks_healthy 
If we do not know the value, just do not have that key/value pair in the item.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
64cc16a9f4 Item: fix xattr processing 
Item.xattrs is now always a StableDict mapping bytes keys -> bytes values.

The special casing of empty values (b'') getting replaced by None was removed.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
9d684120a2 Item: assert type also in property getter 
also: fixed Item.xattrs to be StableDict (not just a dict, as the
msgpack unpacker gives us)
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
7b138cc710 Item: convert timestamps once, get rid of bigint code 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e58525fc6 Item: remove some decode= params 
update_internal() makes sure they have the desired type already.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
655c1b9cc2 update docstrings / comments 2022-06-09 17:57:28 +02:00
Thomas Waldmann
33444be926 more str vs bytes fixing 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e87f1111b cleanup msgpack related str/bytes mess, fixes #968 
see ticket and borg.helpers.msgpack docstring.

this changeset implements the full migration to
msgpack 2.0 spec (use_bin_type=True, raw=False).

still needed compat to the past is done via want_bytes decoder in borg.item.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
f8dbe5b542 cleanup msgpack related str/bytes mess, see #968 
see ticket and borg.helpers.msgpack docstring.
 2022-06-09 17:57:28 +02:00
TW
86fe8bdd57
Merge pull request #6703 from ThomasWaldmann/r2r-transfer 
borg2: "borg transfer" cmd (and also getting rid of legacy)
 2022-06-09 17:56:03 +02:00
Thomas Waldmann
c5540c2dd9 upgrade compressed chunk: fix treatment of ObfuscateSize chunks 
the inner payload of ObfuscateSize chunks are compressed chunks and need
the same zlib fix and level patching as non-obfuscated compressed chunks.
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
72c68c49d0 obfuscation: fix byte order for size, fixes #6701 2022-06-09 17:49:16 +02:00
Thomas Waldmann
6584a92c81 compression: use the 2 bytes for type and level, fixes #6698 
adapt borg transfer, transferred chunks are set to compression level "unknown".
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
32a3601e4a compute hlid from inode / device 2022-06-09 17:49:16 +02:00
Thomas Waldmann
8798b0340a use whitelist approach to make sure item._dict is clean 2022-06-09 17:49:16 +02:00
Thomas Waldmann
d3dfa3be30 use version 2 for new archives 
but still be able to read v1 archives
for borg transfer.
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
e5f1a4fb4d recreate: cachedir_masters not needed any more 
now all hardlinked regular file items have chunks.
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
6bfdb3f630 refactor hardlink_master processing globally 
borg now has the chunks list in every item with content.
due to the symmetric way how borg now deals with hardlinks using
item.hlid, processing gets much simpler.

but some places where borg deals with other "sources" of hardlinks
still need to do some hardlink management:
borg uses the HardLinkManager there now (which is not much more
than a dict, but keeps documentation at one place and avoids some
code duplication we had before).

item.hlid is computed via hardlink_id function.

support hardlinked symlinks, fixes #2379
as we use item.hlid now to group hardlinks together,
there is no conflict with the item.source usage for
symlink targets any more.

2nd+ hardlinks now add to the files count as did the 1st one.
for borg, now all hardlinks are created equal.
so any hardlink item with chunks now adds to the "file" count.

ItemFormatter: support {hlid} instead of {source} for hardlinks
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
7903dad183 transfer: convert timestamps int/bigint -> msgpack.Timestamp, see #2323 
Timestamp scales to 64 or 96bit serialization formats, that should be enough for everybody.

We use this in archived items and also in the files cache.
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
e4a97ea8cc transfer: all hardlinks have chunks, maybe chunks_healty, hlid 
Item.hlid: same id, same hardlink (xxh64 digest)
Item.hardlink_master: not used for new archives any more
Item.source: not used for hardlink slaves any more
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
01f72d15b4 transfer: remove the zlib type bytes hack 
hack: see the docstring of ZLIB_legacy class.

New clean ZLIB class that works as every other compressor.

ZLIB ID 0x0500, ZLIB_legacy ID 0x.8..
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
ba1dbe6111 transfer: make sure items with chunks have precomputed size 2022-05-18 14:20:01 +02:00
Thomas Waldmann
98b7dc0bf5 transfer: clean item of attic 0.13 'acl' bug remnants 
also: remove attic bug support code from borg check.

borg transfer removes the acl key. we do not run borg check on old repos.
 2022-05-18 14:20:00 +02:00
Thomas Waldmann
116f67036f transfer: copy archives from another repo 
this is somehow similar to borg recreate,
but with different focus and way simpler:

not changing compression algo
not changing chunking
not excluding files inside an archive by path match
only dealing with complete archives

but:
different src and dst repo
only reading each chunk once
keeping the compressed payload (no decompression/recompression effort)
--dry-run can be used before and afterwards to check
 2022-05-18 14:20:00 +02:00
Emil M George
1b4b84dfd8
Remove scp syntax for locations (#6697) 
remove scp syntax support from cli and from Location parser, add note about converting scp-style URLs to ssh-style, fixes #6691
 2022-05-15 21:55:19 +02:00
Andrea Gelmini
c79fd61b5c
Fix typos (#6688) 
fix typos

Co-authored-by: Thomas Waldmann <tw@waldmann-edv.de>
 2022-05-15 21:23:48 +02:00
TW
ab4bef0372
Merge pull request #6694 from ThomasWaldmann/versions-mount-1archive-error-master 
mount -o versions: give clear error msg instead of crashing
 2022-05-15 19:52:32 +02:00
Thomas Waldmann
2bed467e92 mount -o versions: give clear error msg instead of crashing 
it does not make sense to request versions view if you only
look at 1 archive, but the code shall not crash in that case
as it did, but give a clear error msg.
 2022-05-15 00:37:45 +02:00
Christopher Klooz
594d83aad5
docs: authentication primitives: improved security and performance infos (master) (#6667) 
docs: authentication primitives: improved security and performance infos
 2022-05-14 22:57:22 +02:00
ReethuVinta
ac4666d7f4
documented workaround for getting help for sub-sub-commands (#6346) 
docs: how to get help for sub-sub-commands
 2022-05-13 18:29:43 +02:00
ReethuVinta
c85bcfd3ad
documented secret key usage against fingerprinting (#6345) 
docs: secret key usage against fingerprinting

Co-authored-by: Thomas Waldmann <tw@waldmann-edv.de>
 2022-05-10 21:18:40 +02:00
TW
0e3ff0ab70
Merge pull request #6682 from ThomasWaldmann/fix-archive-progress-json-master 
show_progress: add finished=true/false to archive_progress json
 2022-05-08 19:45:39 +02:00
TW
09a84fbd24
Merge pull request #6685 from ThomasWaldmann/fix-create-filter-master 
fix create_filter_process exception handler, fixes #6681
 2022-05-08 19:15:35 +02:00
Thomas Waldmann
18eb696a17 fix create_filter_process exception handler, fixes #6681 
if cmd was falsy (e.g. None), there is no proc.
then, if "yield stream" raises an exception, the exception handler crashed at "proc.kill()".
 2022-05-08 18:46:13 +02:00