Commit Graph

6753 Commits

Author SHA1 Message Date
Jonathan Rascher da07c36d6b
(docs) Recommend umask for passphrase file perms
The previous sample for creating a ~/.borg-passphrase file creates it first and then chmod's it to 400 permissions. That's probably fine in practice, but means there's a tiny window where the passphrase file is sitting with default permissions (likely world readable, depending on the system umask).

It seems safer to first change the umask to remove all group & world bits (0077) _before_ creating the file. To be polite and avoid messing with the user's previous umask, we do this in a subshell. (Note that umask 0077 leads to a mode of 600 rather than the previous 400, because removing the owner write bit doesn't seem to buy much since the owner can just chmod the file anyway.)
2022-04-03 00:34:33 -05:00
Thomas Waldmann e8069a8f80 import/export-tar: --tar-format=BORG: roundtrip ALL item metadata, fixes #5830
export-tar: just msgpack and b64encode all item metadata and
            put that into a BORG specific PAX header.
            this is *additional* to the standard tar metadata.

import-tar: when detecting the BORG specific PAX header, just get
            all metadata from there (and ignore the standard tar
            metadata).
2022-04-02 22:25:44 +02:00
TW 22fc6d1bdd
Merge pull request #6522 from ThomasWaldmann/tar-pax-master
import/export-tar: PAX format, ctime and atime support
2022-04-02 19:08:53 +02:00
TW 3c0779afa7
Merge pull request #6520 from ThomasWaldmann/memoryviews
memoryviews
2022-04-02 19:04:42 +02:00
Thomas Waldmann 78e92fa9e1 import/export-tar: --tar-format, support ctime/atime
--tar-format=GNU|PAX (default: GNU)

changed the tests which use GNU tar cli tool to use --tar-format=GNU
explicitly, so they don't break in case we change the default.

atime timestamp is only present in output if the archive item has it
(which is not the case by default, needs "borg create --atime ...").
2022-04-02 18:30:55 +02:00
Thomas Waldmann 154e5d87e7 micro opt: callers shall provide a memoryview to .decompress()
if they would call with a bytes object, it would usually cause
1 object copy to remove the compression type bytes.
2022-04-01 21:22:31 +02:00
Thomas Waldmann df23f3ed22 micro opt lz4/zstd decompress: keep memoryview a bit longer, fixes #3412
if LZ4/ZSTD.decompress gets called with a memoryview idata, keep
it until after the super().decompress(idata) call, so we save one
copy operation just to remove the 2 bytes long compression type
header.
2022-04-01 21:03:28 +02:00
TW e8456ff7d9
Merge pull request #6516 from ThomasWaldmann/remove-attic
upgrade: remove the "attic backup" repo upgrader and tests
2022-04-01 13:53:03 +02:00
Thomas Waldmann c7b1cd56d8 upgrade: remove the "attic backup" repo upgrader and tests
attic is borg's parent project, but it stalled in 2015 and was not updated since then.

guess we can assume that most attic users have meanwhile noticed this and already
converted their repos to borg.

if some did not yet, they are advised to use borg < 1.3 to do that ASAP.

note: borg can still DETECT an attic repo by recognizing its ATTIC_MAGIC value
      and then gives exactly that advice.
2022-04-01 12:41:11 +02:00
TW 7abc62b308
Merge pull request #6511 from ThomasWaldmann/repo-cleanup
repo code cleanup
2022-03-31 21:33:29 +02:00
Thomas Waldmann cfa34bdf71 repository: simplify LoggedIO._read
Code gets simpler if we always only use the (shorter) header_fmt.
That format ALWAYS applies, to all tags borg writes.

If the tag unpacked from there indicates that there is also a chunkid
to read (like for PUT and DEL), we can decide that inside _read and
then read the chunkid from the fd.
2022-03-31 20:50:55 +02:00
TW b9ea17da77
Merge pull request #6506 from ThomasWaldmann/update-security-support-master
borg 1.2 is security supported
2022-03-30 00:11:38 +02:00
Thomas Waldmann 2dbf11b154 borg 1.2 is security supported 2022-03-29 23:58:37 +02:00
TW da565f53ea
Merge pull request #6499 from ThomasWaldmann/usage-fixes
docs: usage: add benchmark cpu and key change-location
2022-03-28 00:13:18 +02:00
Thomas Waldmann a91dd04d11 docs: usage: add benchmark cpu and key change-location 2022-03-26 23:40:03 +01:00
TW 28731c56d1
Merge pull request #6498 from ThomasWaldmann/vagrant-fixes-master
vagrant: use generic/freebsd13, FUSE fixes
2022-03-26 22:55:32 +01:00
Thomas Waldmann 705336a7b6 vagrant: use generic/freebsd13, FUSE fixes
also: some stuff has pkg-config support now, remove env vars.
2022-03-26 22:44:18 +01:00
TW 9390f54225
Merge pull request #6497 from ThomasWaldmann/vagrant-py3.9.12
vagrant: use Python 3.9.12
2022-03-26 21:47:09 +01:00
Thomas Waldmann 417e790fee vagrant: use Python 3.9.12 2022-03-26 21:38:37 +01:00
TW 312cae5ebb
Merge pull request #6495 from ThomasWaldmann/kdf-timings
borg benchmark cpu: add kdf timings
2022-03-26 20:29:45 +01:00
TW e5ab19c4c3
Merge pull request #6496 from ThomasWaldmann/fix-manifest-id
bugfix: correctly give manifest id
2022-03-26 20:29:19 +01:00
Thomas Waldmann 7083e6738f bugfix: correctly give manifest id
but do not verify chunk-id for it (does not match because the ID is
fixed at all-zero).
2022-03-26 20:07:52 +01:00
TW 09e2afe951
Merge pull request #6492 from ThomasWaldmann/use-compare_digest
use hmac.compare_digest instead of ==, fixes #6470
2022-03-26 19:48:18 +01:00
Thomas Waldmann 3fe622dd92 borg benchmark cpu: add kdf timings
Note: A KDF shall be expensive to compute to slow down
attacks on your key encryption / your passphrase.
2022-03-26 19:47:43 +01:00
Thomas Waldmann fe7eb5702b use hmac.compare_digest instead of ==, fixes #6470 2022-03-26 18:51:36 +01:00
TW e443588329
Merge pull request #6487 from ThomasWaldmann/update-ubuntu-link
docs: update link to ubuntu packages, fixes #6485
2022-03-26 18:21:06 +01:00
Thomas Waldmann 359a347fdd docs: update link to ubuntu packages, fixes #6485 2022-03-26 18:06:08 +01:00
TW 23b27cfd0c
Merge pull request #6486 from ThomasWaldmann/update-docs
docs: add libdeflate requirement to install docs, fixes #6446
2022-03-26 18:03:38 +01:00
Thomas Waldmann b2ab823fea docs: update cygwin install notes to python 3.9 2022-03-26 17:56:54 +01:00
Thomas Waldmann 07cf06f9e1 docs: add libdeflate requirement to install docs, fixes #6446
there was no change required due to argon2-cffi, because that will be automatically installed by pip.
2022-03-26 17:52:46 +01:00
TW fbdeaa89bc
Merge pull request #6463 from ThomasWaldmann/new-crypto
new AEAD crypto with session keys
2022-03-26 17:27:58 +01:00
Thomas Waldmann c66826528b init olen to avoid some (false positive) compiler warnings
olen is assigned by OpenSSL, but the compiler can't know that and generates these warnings:

  warning: src/borg/crypto/low_level.pyx:271:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:274:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:314:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:317:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:514:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:517:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:566:22: local variable 'olen' referenced before assignment
  warning: src/borg/crypto/low_level.pyx:572:22: local variable 'olen' referenced before assignment
2022-03-26 17:05:57 +01:00
Thomas Waldmann 10cbdcc67e add encryption-aead diagram 2022-03-26 17:05:57 +01:00
Thomas Waldmann af26835dfc delete pointless assert
thanks @hexagonrecursion for finding this.
2022-03-26 17:05:57 +01:00
Thomas Waldmann dd2a054ac4 crypto: key: reduce class inheritance depth 2022-03-26 17:05:57 +01:00
Thomas Waldmann 3a0e1a1cc2 crypto: low_level: reduce class inheritance depth 2022-03-26 17:05:57 +01:00
Thomas Waldmann e4b65dea76 crypto: add IV overflow check
will never happen, but better play safe.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 900a812e9c crypto: bump API_VERSION to 1.3_01 2022-03-26 17:05:57 +01:00
Thomas Waldmann ce247526c2 docs: update borg init examples 2022-03-26 17:05:57 +01:00
Thomas Waldmann 298c5ee539 docs: security infos only applying to legacy encryption 2022-03-26 17:05:57 +01:00
Thomas Waldmann b3383a4d53 update borg init docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann ccf0875053 EVP_DecryptFinal_ex: fix check for return value
seems like the current docs were updated.
was "positive return code".
now specifically mentions 0 and 1.
2022-03-26 17:05:57 +01:00
Thomas Waldmann e1313ccc05 crypto.low_level: simplify return code checks (legacy) 2022-03-26 17:05:57 +01:00
Thomas Waldmann 948d67efd8 crypto.low_level: simplify return code checks (AEAD) 2022-03-26 17:05:57 +01:00
Thomas Waldmann f4a6ad080b docs: add new AEAD modes to security docs 2022-03-26 17:05:57 +01:00
Thomas Waldmann c50e1124b5 also authenticate the chunkid when using the AEAD ciphers (AES-OCB/CHACHA-POLY) 2022-03-26 17:05:57 +01:00
Thomas Waldmann 8bd9477b96 add aad parameter to borg.crypto.low_level api
added it for all classes there, so the caller just give it.

for the legacy AES-CTR based classes, the given aad is completely ignored.
this is to stay compatible with repo data of borg < 1.3.

for the new AEAD based classes:
encrypt: the aad is fed into the auth tag computation
decrypt: same. decrypt will fail on auth tag mismatch.
2022-03-26 17:05:57 +01:00
Thomas Waldmann d3b78a6cf5 minor key.encrypt api change/cleanup
we already have .decrypt(id, data, ...).
i changed .encrypt(chunk) to .encrypt(id, data).

the old borg crypto won't really need or use the id,
but the new AEAD crypto will authenticate the id in future.
2022-03-26 17:05:57 +01:00
Thomas Waldmann 41b8a04d82 use faster hmac.digest api 2022-03-26 17:05:57 +01:00
Thomas Waldmann 74ecb63671 fix new crypto benchmarks for api change 2022-03-26 17:05:57 +01:00