if we know both archives' chunker params, use "are different" not "might be".
also do not recommend to enforce it using --same-chunker-params in this case.
that would require setuptools_scm>=5.0.0 but some dists do not have that yet.
also, we do not use the version_tuple from _version.py, so it is not required
anyway.
looks like we can not rely on listxattr only returning
valid, acceptable names for getxattr.
so getxattr can still fail with EINVAL.
also:
- do not raise an exception if getting a single
xattr fails, rather emit a warning and continue
processing the remaining xattrs (and also the whole
file). we already had that for EPERM (and similar
for ENOATTR), just do it for all errors.
- _check, when it raises OSError, gives us a nice
exception object, use it.
- more helpful error msgs, try not to lose error information
There are some other places with subprocesses:
- borg create --content-from-command
- borg create --paths-from-command
- (de)compression filter process of import-tar / export-tar
work around setuptools puking about:
############################
# Package would be ignored #
############################
Python recognizes 'borg.cache_sync' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'borg.cache_sync' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'borg.cache_sync' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
hopefully this is the final fix.
after first fixing of #6400 (by using os.umask after mkstemp), there
was a new problem that chmod was not supported on some fs.
even after fixing that, there were other issues, see the ACLs issue
documented in #6933.
the root cause of all this is tempfile.mkstemp internally using a
very secure, but hardcoded and for our use case problematic mode
of 0o600.
mkstemp_mode (mosty copy&paste from python stdlib tempfile module +
"black" formatting applied) supports giving the mode via the api,
that is the only change needed.
slightly dirty due to the _xxx imports from tempfile, but hopefully
this will be supported in some future python version.
i suspect that compact_segments wrongly drops some DEL tags.
this could make already deleted chunks re-appear, leading to:
- suddenly more repo index entries than previously (check, repo part)
- suddenly orphaned chunks appearing (check, archives part)
if this is the case, the issue is harmless, but annoying/confusing.
to fix the issue, i need the complete log line "dropping DEL for id X".
the id X it mentions is one of the chunk ids borg check complains about.
While writing my own out-of-band decoder, I had a hard time figuring out
how to unpack the manifest. From the description, I was only able to
read that the manifest is msgpack'd, but I had not been able to figure
out that it's also going through the same encryption+compression logic
as all other things do.
This should make it a little clearer and provide the necessary
information to understand how the compression works.
the old code did just 1 attempt to detect the repo decryption key.
if the first chunkid we got from the chunks hashtable iterator was accidentally
the id of the chunk we intentionally corrupted in test_delete_double_force,
setup of the key failed and that made the test crash.
in practice, this could of course also happen if chunks are corrupted, thus
we now do many retries with other chunks before giving up.
error handling was improved: do not return None (instead of a key), it just
leads to weird crashes elsewhere, but fail early with IntegrityError and a
reasonable error msg.
rename method to make_key to avoid confusion with borg.crypto.key.identify_key.
manifest, repo and cache are committed every checkpoint interval.
also, when ctrl-c is pressed, finish deleting the current archive, commit and then terminate.
