#6338 introduces regression when building with LibreSSL (3.5.0).
```
cc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -g -fPIC -O2 -pipe -g -O2 -pipe -g -O2 -pipe -fPIC -Isrc/borg/crypto -I/usr/local/include/python3.9 -c src/borg/crypto/low_level.c -o /tmp/ports/pobj/borgbackup-1.2.1/borg-eec359cf228caf00d9c72bde07bf939872e9d3fa/temp.openbsd-7.1-amd64-3.9/src/borg/crypto/low_level.o
src/borg/crypto/low_level.c:12439:48: error: use of undeclared identifier 'EVP_chacha20_poly1305'; did you mean 'EVP_aead_chacha20_poly1305'?
__pyx_v_self->__pyx_base.__pyx_base.cipher = EVP_chacha20_poly1305;
^~~~~~~~~~~~~~~~~~~~~
EVP_aead_chacha20_poly1305
/usr/include/openssl/evp.h:1161:17: note: 'EVP_aead_chacha20_poly1305' declared here
const EVP_AEAD *EVP_aead_chacha20_poly1305(void);
^
1 error generated.
```
Unfortunately `EVP_aead_chacha20_poly1305`, offered by LibreSSL, is not
a drop in replacement for `EVP_chacha20_poly1305`. More info on the
first can be found at https://man.openbsd.org/EVP_AEAD_CTX_init.3.
All these are unsupported since long.
Newer versions of LibreSSL have gained chacha20-poly1305 support,
but still lack aes256-ocb support.
Also they have the HMAC_CTX_new/free api now.
docs: openssl >= 1.1.1 is required now
anything older is out of support anyway.
The generated source code was producing a compiler warning due to
the pointers differing in constness. The called function expects
a non-const pointer while the generated code produces a const pointer
via a cast. This changes the cast to drop 'const' to make the compiler
happy.
If we create a new repo (and a new keyfile key, create=True),
there must not already exist a keyfile at the path/filename
where we want to write the new one.
In other use cases (e.g. if we overwrite a keyfile due
to the user changing their passphrase, create=False),
of course overwriting at the same path/fname is desired.
Running 'borg key import' on a keyfile repository with the BORG_KEY_FILE
environment variable set works correctly if the BORG_KEY_FILE file
already exists. However, the command crashes if the BORG_KEY_FILE file
does not exist:
$ BORG_KEY_FILE=newborgkey borg key import /home/strager/borg-backups/straglum borgkey
Local Exception
Traceback (most recent call last):
[snip]
File "[snip]/borg/crypto/key.py", line 713, in sanity_check
with open(filename, 'rb') as fd:
FileNotFoundError: [Errno 2] No such file or directory: '[snip]/newborgkey'
Platform: Linux straglum 5.0.0-25-generic #26~18.04.1-Ubuntu SMP Thu Aug 1 13:51:02 UTC 2019 x86_64
Linux: debian buster/sid
Borg: 1.1.11 Python: CPython 3.7.7 msgpack: 0.5.6
PID: 15306 CWD: /home/strager/Projects/borg
sys.argv: ['[snip]/borg', 'key', 'import', '/home/strager/borg-backups/straglum', 'borgkey']
SSH_ORIGINAL_COMMAND: None
Make 'borg key import' not require the BORG_KEY_FILE file to already
exist.
This commit does not change the behavior of 'borg key import' without
BORG_KEY_FILE. This commit also does not change the behavior of 'borg
key import' on a repokey repository.
I want to change the key lookup logic for the 'borg key import' command.
Extract methods out of the KeyfileKey.find_key and
KeyfileKey.get_new_target to make this future change possible without
duplicating code.
This commit should not change behavior.
wrap msgpack to avoid future upstream api changes making troubles
or that we would have to globally spoil our code with extra params.
make sure the packing is always with use_bin_type=False,
thus generating "old" msgpack format (as borg always did) from
bytes objects.
make sure the unpacking is always with raw=True,
thus generating bytes objects.
note:
safe unicode encoding/decoding for some kinds of data types is done in Item
class (see item.pyx), so it is enough if we care for bytes objects on the
msgpack level.
also wrap exception handling, so borg code can catch msgpack specific
exceptions even if the upstream msgpack code raises way too generic
exceptions typed Exception, TypeError or ValueError.
We use own Exception classes for this, upstream classes are deprecated
if interactive passphrase query fails and the env vars are not set,
show a clear error message about this.
users often do 'BORG_PASSPHRASE=secret', forgetting the 'export'.
or they use sudo (and not sudo -E).
in both cases, the env vars won't be available for the borg process.
Hopefully it is better dealing with a lot of small-object allocations than malloc/free is.
Small allocs happen if the input file is small, so it results only in 1 small chunk.
This breaks it on OpenSSL 1.0.x as there is no HMAC_CTX_new/free() yet.
OTOH, this change is consistent with the previous change done for
EVP_CIPHER_CTX (which works on 1.0 and 1.1).
includes:
- aes256-ctr-hmac-sha256 (attic/borg legacy, optional aad support)
- aes256-gcm (new, optional aad support)
uses 96bits for iv, 128bit for auth tag.
- header support
the caller-provided header will be just copied in front of the rest -
this avoids expensive operations (memcpy, garbage collection) in Python.
the first bytes in the header may be non-authenticated data if aad_offset > 0.
this is to support legacy attic/borg envelope layout, where the type byte
is not authenticated.
- aad support
additional authenticated data - it just contributes to the computed mac,
but is not encrypted). the current api assumes that aad starts at some
aad_offset inside the given header and extends to the end of it.
- iv handling helpers, compute next iv based on amount of processed data
- unit tests
Note: the changes are intentionally kept isolated / not integrated into the
rest of the code, so this has to be done later.
rename authenticated to authenticated-blake2, consistent with the other
blake2 key modes
add authenticated mode that fills the blank and is consistent with the
other "unqualified" key modes
Implement IntegrityCheckedFile
This is based on much earlier work from October 2016 by me, but is
overall simplified and changed terminology (from "signing" to
hashing and integrity checking).
See #1688 for the full history.
Thomas Waldmann