Commit Graph

6697 Commits

Author SHA1 Message Date
Thomas Waldmann df23f3ed22 micro opt lz4/zstd decompress: keep memoryview a bit longer, fixes #3412
if LZ4/ZSTD.decompress gets called with a memoryview idata, keep
it until after the super().decompress(idata) call, so we save one
copy operation just to remove the 2 bytes long compression type
header.
2022-04-01 21:03:28 +02:00
TW e8456ff7d9
Merge pull request #6516 from ThomasWaldmann/remove-attic
upgrade: remove the "attic backup" repo upgrader and tests
2022-04-01 13:53:03 +02:00
Thomas Waldmann c7b1cd56d8 upgrade: remove the "attic backup" repo upgrader and tests
attic is borg's parent project, but it stalled in 2015 and was not updated since then.

guess we can assume that most attic users have meanwhile noticed this and already
converted their repos to borg.

if some did not yet, they are advised to use borg < 1.3 to do that ASAP.

note: borg can still DETECT an attic repo by recognizing its ATTIC_MAGIC value
      and then gives exactly that advice.
2022-04-01 12:41:11 +02:00
TW 7abc62b308
Merge pull request #6511 from ThomasWaldmann/repo-cleanup
repo code cleanup
2022-03-31 21:33:29 +02:00
Thomas Waldmann cfa34bdf71 repository: simplify LoggedIO._read
Code gets simpler if we always only use the (shorter) header_fmt.
That format ALWAYS applies, to all tags borg writes.

If the tag unpacked from there indicates that there is also a chunkid
to read (like for PUT and DEL), we can decide that inside _read and
then read the chunkid from the fd.
2022-03-31 20:50:55 +02:00
TW b9ea17da77
Merge pull request #6506 from ThomasWaldmann/update-security-support-master
borg 1.2 is security supported
2022-03-30 00:11:38 +02:00
Thomas Waldmann 2dbf11b154 borg 1.2 is security supported 2022-03-29 23:58:37 +02:00
TW da565f53ea
Merge pull request #6499 from ThomasWaldmann/usage-fixes
docs: usage: add benchmark cpu and key change-location
2022-03-28 00:13:18 +02:00
Thomas Waldmann a91dd04d11 docs: usage: add benchmark cpu and key change-location 2022-03-26 23:40:03 +01:00
TW 28731c56d1
Merge pull request #6498 from ThomasWaldmann/vagrant-fixes-master
vagrant: use generic/freebsd13, FUSE fixes
2022-03-26 22:55:32 +01:00
Thomas Waldmann 705336a7b6 vagrant: use generic/freebsd13, FUSE fixes
also: some stuff has pkg-config support now, remove env vars.
2022-03-26 22:44:18 +01:00
TW 9390f54225
Merge pull request #6497 from ThomasWaldmann/vagrant-py3.9.12
vagrant: use Python 3.9.12
2022-03-26 21:47:09 +01:00
Thomas Waldmann 417e790fee vagrant: use Python 3.9.12 2022-03-26 21:38:37 +01:00
TW 312cae5ebb
Merge pull request #6495 from ThomasWaldmann/kdf-timings
borg benchmark cpu: add kdf timings
2022-03-26 20:29:45 +01:00
TW e5ab19c4c3
Merge pull request #6496 from ThomasWaldmann/fix-manifest-id
bugfix: correctly give manifest id
2022-03-26 20:29:19 +01:00
Thomas Waldmann 7083e6738f bugfix: correctly give manifest id
but do not verify chunk-id for it (does not match because the ID is
fixed at all-zero).
2022-03-26 20:07:52 +01:00
TW 09e2afe951
Merge pull request #6492 from ThomasWaldmann/use-compare_digest
use hmac.compare_digest instead of ==, fixes #6470
2022-03-26 19:48:18 +01:00
Thomas Waldmann 3fe622dd92 borg benchmark cpu: add kdf timings
Note: A KDF shall be expensive to compute to slow down
attacks on your key encryption / your passphrase.
2022-03-26 19:47:43 +01:00
Thomas Waldmann fe7eb5702b use hmac.compare_digest instead of ==, fixes #6470 2022-03-26 18:51:36 +01:00
TW e443588329
Merge pull request #6487 from ThomasWaldmann/update-ubuntu-link
docs: update link to ubuntu packages, fixes #6485
2022-03-26 18:21:06 +01:00
Thomas Waldmann 359a347fdd docs: update link to ubuntu packages, fixes #6485 2022-03-26 18:06:08 +01:00
TW 23b27cfd0c
Merge pull request #6486 from ThomasWaldmann/update-docs
docs: add libdeflate requirement to install docs, fixes #6446
2022-03-26 18:03:38 +01:00
Thomas Waldmann b2ab823fea docs: update cygwin install notes to python 3.9 2022-03-26 17:56:54 +01:00
Thomas Waldmann 07cf06f9e1 docs: add libdeflate requirement to install docs, fixes #6446
there was no change required due to argon2-cffi, because that will be automatically installed by pip.
2022-03-26 17:52:46 +01:00
TW fbdeaa89bc
Merge pull request #6463 from ThomasWaldmann/new-crypto
new AEAD crypto with session keys
2022-03-26 17:27:58 +01:00
Thomas Waldmann c66826528b init olen to avoid some (false positive) compiler warnings
olen is assigned by OpenSSL, but the compiler can't know that and generates these warnings:

  warning: src/borg/crypto/low_level.pyx:271:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:274:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:314:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:317:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:514:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:517:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:566:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:572:22: local variable 'olen' referenced before assignment
2022-03-26 17:05:57 +01:00
Thomas Waldmann 10cbdcc67e add encryption-aead diagram 2022-03-26 17:05:57 +01:00
Thomas Waldmann af26835dfc delete pointless assert
thanks @hexagonrecursion for finding this.
2022-03-26 17:05:57 +01:00
Thomas Waldmann dd2a054ac4 crypto: key: reduce class inheritance depth 2022-03-26 17:05:57 +01:00
Thomas Waldmann 3a0e1a1cc2 crypto: low_level: reduce class inheritance depth 2022-03-26 17:05:57 +01:00
Thomas Waldmann e4b65dea76 crypto: add IV overflow check
will never happen, but better play safe.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 900a812e9c crypto: bump API_VERSION to 1.3_01 2022-03-26 17:05:57 +01:00
Thomas Waldmann ce247526c2 docs: update borg init examples 2022-03-26 17:05:57 +01:00
Thomas Waldmann 298c5ee539 docs: security infos only applying to legacy encryption 2022-03-26 17:05:57 +01:00
Thomas Waldmann b3383a4d53 update borg init docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann ccf0875053 EVP_DecryptFinal_ex: fix check for return value
seems like the current docs were updated.
was "positive return code".
now specifically mentions 0 and 1.
2022-03-26 17:05:57 +01:00
Thomas Waldmann e1313ccc05 crypto.low_level: simplify return code checks (legacy) 2022-03-26 17:05:57 +01:00
Thomas Waldmann 948d67efd8 crypto.low_level: simplify return code checks (AEAD) 2022-03-26 17:05:57 +01:00
Thomas Waldmann f4a6ad080b docs: add new AEAD modes to security docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann c50e1124b5 also authenticate the chunkid when using the AEAD ciphers (AES-OCB/CHACHA-POLY) 2022-03-26 17:05:57 +01:00
Thomas Waldmann 8bd9477b96 add aad parameter to borg.crypto.low_level api
added it for all classes there, so the caller just give it.

for the legacy AES-CTR based classes, the given aad is completely ignored.
this is to stay compatible with repo data of borg < 1.3.

for the new AEAD based classes:
encrypt: the aad is fed into the auth tag computation
decrypt: same. decrypt will fail on auth tag mismatch.
2022-03-26 17:05:57 +01:00
Thomas Waldmann d3b78a6cf5 minor key.encrypt api change/cleanup
we already have .decrypt(id, data, ...).
i changed .encrypt(chunk) to .encrypt(id, data).

the old borg crypto won't really need or use the id,
but the new AEAD crypto will authenticate the id in future.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 41b8a04d82 use faster hmac.digest api 2022-03-26 17:05:57 +01:00
Thomas Waldmann 74ecb63671 fix new crypto benchmarks for api change 2022-03-26 17:05:57 +01:00
Thomas Waldmann 0b5a21275f avoid losing the key (old crypto)
if we just have a pointer to a bytes object which might go out of scope, we can lose it.

also: cython can directly assign a bytes object into a same-size char array.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 6d6d3caa24 avoid losing the key
if we just have a pointer to a bytes object which might go out of scope, we can lose it.

also: cython can directly assign a bytes object into a same-size char array.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 41082f558f crypto: add some tests for new key types 2022-03-26 17:05:57 +01:00
Thomas Waldmann 6f2c587080 tests: consistently give iv_int to ciphersuite 2022-03-26 17:05:57 +01:00
Thomas Waldmann 6c7b499d3f set aead auth tag directly before EVP_DecryptFinal_ev
https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption
2022-03-26 17:05:57 +01:00
Thomas Waldmann bb949b25ea EVP_DecryptFinal_ex: fix check for return value
seems like the current docs were updated.
was "positive return code".
now specifically mentions 0 and 1.
2022-03-26 17:05:57 +01:00