this is similar to #4777.
borg check must not crash if an archive metadata block does not decrypt.
Instead, report the archive_id, remove the archive from the manifest and skip to the next archive.
selftest imports testsuite.crypto
I did not realise this and imported pytest from testsuite.crypto
This broke the selftest.
Solution: move the tests that depend on pytest to testsuite.key.
All three affected tests are tests for the Key classes, so
this is probably a better plase for them anyway.
when migrating from repokey to keyfile, we just store an empty key into the repo config,
because we do not have a "delete key" RPC api. thus, empty key means "there is no key".
here we fix load_key, so that it does not behave differently for no key and empty key:
in both cases, it just returns an empty value.
additionally, we strip the value we get from the config, so whitespace does not matter.
All callers now check for the repokey not being empty, otherwise RepoKeyNotFoundError
is raised.
for now, this code shall only work on v2 repos (created by this code).
the code to read v1 repos is still present though, so for experiments,
it is possible to change the repo version in the repo config from 1 to
2 manually.
having version 2 in the repo config also avoids that borg < 1.3 is
used on such a repo, which would cause damage:
old borg would not recognize the PUT2 tagged segment entries and
old borg check --repair would likely kill them all due to that.
also: keep repo version in Repository.version
note: this required a slight increase of MAX_OBJECT_SIZE so that MAX_DATA_SIZE
could stay the same as before.
For PUT2, compute the hash over the whole entry (header and content, excluding
hash and crc32 fields, because the crc32 computation includes the hash).
Also: refactor crc32 checks into function, use f-strings, structure _read in
a more logical sequential order.
write_put: avoid creating a large temporary bytes object
why use xxh64?
- fast even without hw acceleration
- borg depends on it already anyway
- stronger than crc32 and strong enough for this purpose
Argon2 the second part: implement encryption/decryption of argon2 keys
borg init --key-algorithm=argon2 (new default, older pbkdf2 also still available)
borg key change-passphrase: keep key algorithm the same
borg key change-location: keep key algorithm the same
use env var BORG_TESTONLY_WEAKEN_KDF=1 to resource limit (cpu, memory, ...) the kdf when running the automated tests.
OpenBSD does not have `lchmod()` causing `os.lchmod` to be unavailable
on this platform. As a result ArchiverTestCase::test_basic_functionality
fails when run manually (#2055).
OpenBSD does have `fchmodat()`, which has a flag that makes it behave
like `lchmod()`. In Python this can be used via `os.chmod(path, mode,
follow_symlinks=False)`.
As of Python 3.3 `os.lchmod(path, mode)` is equivalent to
`os.chmod(path, mode, follow_symlinks=False)`. As such, switching to the
latter is preferred as it enables more platforms to do the right thing.
although bug #6526 did not show with ssh style URLs, we should
not have different regexes for the host part for ssh and scp style.
thus i extracted the host_re from both and also cleaned up a bit.
added a negative lookahead/lookbehind to make sure an ipv6 addr
(enclosed in square brackets) does not get badly matched by the
regex part intended for hostnames and ipv4 addrs only.
the other part of that regex which is actually intended to match
ipv6 addrs only matches if they are enclosed in square brackets.
also added tests for ssh and scp style repo URLs with ipv6 addrs
in brackets.
also: made regex more readable, putting these 2 cases on separate lines.
The previous sample for creating a ~/.borg-passphrase file creates it first and then chmod's it to 400 permissions. That's probably fine in practice, but means there's a tiny window where the passphrase file is sitting with default permissions (likely world readable, depending on the system umask).
It seems safer to first change the umask to remove all group & world bits (0077) _before_ creating the file. To be polite and avoid messing with the user's previous umask, we do this in a subshell. (Note that umask 0077 leads to a mode of 600 rather than the previous 400, because removing the owner write bit doesn't seem to buy much since the owner can just chmod the file anyway.)
export-tar: just msgpack and b64encode all item metadata and
put that into a BORG specific PAX header.
this is *additional* to the standard tar metadata.
import-tar: when detecting the BORG specific PAX header, just get
all metadata from there (and ignore the standard tar
metadata).
--tar-format=GNU|PAX (default: GNU)
changed the tests which use GNU tar cli tool to use --tar-format=GNU
explicitly, so they don't break in case we change the default.
atime timestamp is only present in output if the archive item has it
(which is not the case by default, needs "borg create --atime ...").
if LZ4/ZSTD.decompress gets called with a memoryview idata, keep
it until after the super().decompress(idata) call, so we save one
copy operation just to remove the 2 bytes long compression type
header.
attic is borg's parent project, but it stalled in 2015 and was not updated since then.
guess we can assume that most attic users have meanwhile noticed this and already
converted their repos to borg.
if some did not yet, they are advised to use borg < 1.3 to do that ASAP.
note: borg can still DETECT an attic repo by recognizing its ATTIC_MAGIC value
and then gives exactly that advice.
Code gets simpler if we always only use the (shorter) header_fmt.
That format ALWAYS applies, to all tags borg writes.
If the tag unpacked from there indicates that there is also a chunkid
to read (like for PUT and DEL), we can decide that inside _read and
then read the chunkid from the fd.
