mirror/borg
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2024-12-25 17:27:31 +00:00
Code Issues Releases Wiki Activity
6819 commits 9 branches 139 tags 60 MiB
Commit graph

6819 commits

This branch
This branch
All branches
Author SHA1 Message Date
Thomas Waldmann
6bfdb3f630 refactor hardlink_master processing globally 
borg now has the chunks list in every item with content.
due to the symmetric way how borg now deals with hardlinks using
item.hlid, processing gets much simpler.

but some places where borg deals with other "sources" of hardlinks
still need to do some hardlink management:
borg uses the HardLinkManager there now (which is not much more
than a dict, but keeps documentation at one place and avoids some
code duplication we had before).

item.hlid is computed via hardlink_id function.

support hardlinked symlinks, fixes #2379
as we use item.hlid now to group hardlinks together,
there is no conflict with the item.source usage for
symlink targets any more.

2nd+ hardlinks now add to the files count as did the 1st one.
for borg, now all hardlinks are created equal.
so any hardlink item with chunks now adds to the "file" count.

ItemFormatter: support {hlid} instead of {source} for hardlinks
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
7903dad183 transfer: convert timestamps int/bigint -> msgpack.Timestamp, see #2323 
Timestamp scales to 64 or 96bit serialization formats, that should be enough for everybody.

We use this in archived items and also in the files cache.
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
e4a97ea8cc transfer: all hardlinks have chunks, maybe chunks_healty, hlid 
Item.hlid: same id, same hardlink (xxh64 digest)
Item.hardlink_master: not used for new archives any more
Item.source: not used for hardlink slaves any more
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
01f72d15b4 transfer: remove the zlib type bytes hack 
hack: see the docstring of ZLIB_legacy class.

New clean ZLIB class that works as every other compressor.

ZLIB ID 0x0500, ZLIB_legacy ID 0x.8..
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
ba1dbe6111 transfer: make sure items with chunks have precomputed size 2022-05-18 14:20:01 +02:00
Thomas Waldmann
98b7dc0bf5 transfer: clean item of attic 0.13 'acl' bug remnants 
also: remove attic bug support code from borg check.

borg transfer removes the acl key. we do not run borg check on old repos.
 2022-05-18 14:20:00 +02:00
Thomas Waldmann
116f67036f transfer: copy archives from another repo 
this is somehow similar to borg recreate,
but with different focus and way simpler:

not changing compression algo
not changing chunking
not excluding files inside an archive by path match
only dealing with complete archives

but:
different src and dst repo
only reading each chunk once
keeping the compressed payload (no decompression/recompression effort)
--dry-run can be used before and afterwards to check
 2022-05-18 14:20:00 +02:00
Emil M George
1b4b84dfd8
Remove scp syntax for locations (#6697) 
remove scp syntax support from cli and from Location parser, add note about converting scp-style URLs to ssh-style, fixes #6691
 2022-05-15 21:55:19 +02:00
Andrea Gelmini
c79fd61b5c
Fix typos (#6688) 
fix typos

Co-authored-by: Thomas Waldmann <tw@waldmann-edv.de>
 2022-05-15 21:23:48 +02:00
TW
ab4bef0372
Merge pull request #6694 from ThomasWaldmann/versions-mount-1archive-error-master 
mount -o versions: give clear error msg instead of crashing
 2022-05-15 19:52:32 +02:00
Thomas Waldmann
2bed467e92 mount -o versions: give clear error msg instead of crashing 
it does not make sense to request versions view if you only
look at 1 archive, but the code shall not crash in that case
as it did, but give a clear error msg.
 2022-05-15 00:37:45 +02:00
Christopher Klooz
594d83aad5
docs: authentication primitives: improved security and performance infos (master) (#6667) 
docs: authentication primitives: improved security and performance infos
 2022-05-14 22:57:22 +02:00
ReethuVinta
ac4666d7f4
documented workaround for getting help for sub-sub-commands (#6346) 
docs: how to get help for sub-sub-commands
 2022-05-13 18:29:43 +02:00
ReethuVinta
c85bcfd3ad
documented secret key usage against fingerprinting (#6345) 
docs: secret key usage against fingerprinting

Co-authored-by: Thomas Waldmann <tw@waldmann-edv.de>
 2022-05-10 21:18:40 +02:00
TW
0e3ff0ab70
Merge pull request #6682 from ThomasWaldmann/fix-archive-progress-json-master 
show_progress: add finished=true/false to archive_progress json
 2022-05-08 19:45:39 +02:00
TW
09a84fbd24
Merge pull request #6685 from ThomasWaldmann/fix-create-filter-master 
fix create_filter_process exception handler, fixes #6681
 2022-05-08 19:15:35 +02:00
Thomas Waldmann
18eb696a17 fix create_filter_process exception handler, fixes #6681 
if cmd was falsy (e.g. None), there is no proc.
then, if "yield stream" raises an exception, the exception handler crashed at "proc.kill()".
 2022-05-08 18:46:13 +02:00
Thomas Waldmann
1c0937958d show_progress: add finished=true/false to archive_progress json, fixes #6570 
also:
- remove empty values from final json
- add test
 2022-05-08 18:32:07 +02:00
TW
a3e99a2b51
Merge pull request #6659 from ThomasWaldmann/docs-fc-suffix-master 
docs: mention BORG_FILES_CACHE_SUFFIX as alternative to BORG_FILES_CACHE_TTL
 2022-05-07 14:39:57 +02:00
TW
4acfcf23e9
Merge pull request #6670 from ThomasWaldmann/no-pytest-check-master 
check that borg does not require pytest for normal usage, fixes #6563
 2022-05-07 10:41:34 +02:00
Thomas Waldmann
2c25123284 check that borg does not require pytest for normal usage, fixes #6563 
also: move the note about this to the very top of the affected modules.
 2022-05-07 02:24:18 +02:00
TW
dc89798496
Merge pull request #6665 from ThomasWaldmann/fix-key-idhash-compat 
fix key id hash compat check
 2022-05-04 00:00:21 +02:00
Thomas Waldmann
af776ce7a6 refactor: borg.key.uses_same_id_hash(key_old, key_new) 2022-05-03 21:32:35 +02:00
Thomas Waldmann
3239836dce fix key id hash compat check: also support new->new 
the check only considered old key -> new key changes, but
new key to new key is of course also fine.

e.g. repokey-aes-ocb -> repokey-aes-ocb (both use hmac-sha256
as id hash)
 2022-05-03 21:25:44 +02:00
TW
b0f7dd0904
Merge pull request #6664 from ThomasWaldmann/api-give-compressed-master 
api: enable giving already compressed data
 2022-05-03 11:43:47 +02:00
Thomas Waldmann
0e53dc040a api: enable giving already compressed data 
the api already offered getting compressed data,
but not giving compressed data.

thus: cache.add_chunk and key.encrypt improved.
 2022-05-03 04:15:01 +02:00
TW
19ad926c70
Merge pull request #6662 from ThomasWaldmann/decrypt-fixes 
fix key.decrypt calls
 2022-05-02 21:25:26 +02:00
Thomas Waldmann
cc0e33da65 fix key.decrypt calls 
the id must now always be given correctly because
the AEAD crypto modes authenticate the chunk id.

the special case when id == MANIFEST_ID is now handled
inside assert_id, so we never need to give a None id.
 2022-05-02 20:56:50 +02:00
TW
3ba9bab66c
Merge pull request #6660 from ThomasWaldmann/reuse-key 
init --other-location=OTHER_REPO: reuse key material from OTHER_REPO
 2022-05-02 20:52:13 +02:00
Thomas Waldmann
4b070040d6 init --other-location=OTHER_REPO: reuse key material from OTHER_REPO, fixes #6554 
it potentially will ask for the passphrase for the key of OTHERREPO.
for the newly created repo, it will use the same passphrase.

it will copy: enc_key, enc_hmac_key, id_key, chunker_seed.

keeping the id_key (and id algorithm) and the chunker seed (and chunker
algorithm and parameters) is desirable for deduplication.
the id algorithm is usually either HMAC-SHA256 or BLAKE2b.

keeping the enc_key / enc_hmac_key must be implemented carefully:
A) AES-CTR -> AES-CTR is INSECURE due to nonce reuse, thus not allowed.
B) AES-CTR -> AEAD with session keys is secure.
C) AEAD with session keys -> AEAD with session keys is secure.

AEAD modes with session keys: AES-OCB and CHACHA20-POLY1305.
 2022-05-02 18:50:27 +02:00
Thomas Waldmann
020e2defaf implement with_other_repository and BORG_OTHER_REPO 2022-05-02 18:48:14 +02:00
Thomas Waldmann
a375335859 make some code from with_repository reusable 2022-05-02 18:45:14 +02:00
Thomas Waldmann
9f09a9e096 docs: mention BORG_FILES_CACHE_SUFFIX as alternative to BORG_FILES_CACHE_TTL, fixes #5602 2022-04-30 20:28:46 +02:00
TW
65c7829216
Merge pull request #6637 from ThomasWaldmann/docs-files-cache-master 
FAQ: add a hint about --debug-topic=files_cache
 2022-04-20 13:02:09 +02:00
Thomas Waldmann
c2ad118434 FAQ: add a hint about --debug-topic=files_cache 2022-04-20 13:00:02 +02:00
TW
12d27d7944
Merge pull request #6630 from ThomasWaldmann/fix-iec-master 
fix --iec mode output for borg delete and prune
 2022-04-19 23:39:13 +02:00
TW
f937a72a13
Merge pull request #6633 from ThomasWaldmann/fix-info-authenticated-master 
info: fix authenticated mode repo to show "Encrypted: No", fixes #6462
 2022-04-19 23:38:43 +02:00
Thomas Waldmann
16f6983b1d info: fix authenticated mode repo to show "Encrypted: No", fixes #6462 2022-04-19 22:12:23 +02:00
Thomas Waldmann
c86d19cc06 prune: fix --stats --iec output to use binary units, see #6606 2022-04-19 21:56:29 +02:00
Thomas Waldmann
8d1e012ecc delete: fix --stats --iec output to use binary units, see #6606 2022-04-19 21:56:19 +02:00
TW
5743ec7e09
Merge pull request #6616 from ThomasWaldmann/fix-files-cache-6353-master 
fix transaction rollback: use files cache filename as found in txn.active
 2022-04-19 21:16:29 +02:00
TW
0e9431b174
Merge pull request #6624 from ThomasWaldmann/fix-diff-6483-master 
diff: support presence change for blkdev, chrdev and fifo items (master)
 2022-04-19 17:16:13 +02:00
TW
c60a314ee0 diff: support presence change for blkdev, chrdev and fifo items (1.2-maint) (#6615) 
diff: support presence change for blkdev, chrdev and fifo items

also: refactor / clean up / reuse code.
 2022-04-19 16:49:21 +02:00
TW
299196ebcb
Merge pull request #6619 from ThomasWaldmann/invalid-repo-msg-master 
better error msg for defect or unsupported repo configs, fixes #6566
 2022-04-18 09:53:09 +02:00
Thomas Waldmann
1aba534c5a better error msg for defect or unsupported repo configs, fixes #6566 2022-04-18 09:27:26 +02:00
Thomas Waldmann
37430beda3 fix transaction rollback: use files cache filename as found in txn.active, fixes #6353 2022-04-18 09:03:37 +02:00
TW
7fd53903f4
Merge pull request #6604 from ThomasWaldmann/argon2-chacha 
argon2 key: use chacha20-poly1305 instead of aes256-ctr+hmac-sha256
 2022-04-17 07:10:46 +02:00
TW
195a45d1c1
Merge pull request #6609 from ThomasWaldmann/docs-table-formatting-master 
init: docs table formatting fixed (master)
 2022-04-17 01:59:30 +02:00
TW
f766a21fc7
Merge pull request #6610 from ThomasWaldmann/more-benchmarks-master 
benchmark cpu: add some more compression benchmarks
 2022-04-17 01:59:17 +02:00
Thomas Waldmann
77a8464a96 benchmark cpu: add some more compression benchmarks 2022-04-17 00:25:28 +02:00