Didn't add dll folder since it wasn't copied when updating zstd to
1.3.4.
Didn't add extra files from the lib directory (Makefile, README.md, etc)
that were also left out when updating zstd to 1.3.4.
Add new zstd files to setup_zstd.py (DDict was refactored in
facebook/zstd#1388)
before this, it over-eagerly compacted "small" segments ("small"
being < 100MB by default) if there were only a few bytes to be freed.
also:
- improve debug logging
- as compaction is a separate borg command now, use the module logger
scenario:
- x is a regular file
- borg does stat on x: is a regular file
- so borg dispatches to process_file
- attack: x gets replaced by a symlink (mv symlink x)
- in process_file, borg opens x and must not follow the symlink nor
continue processing as a normal file, but rather error in open()
due to NOFOLLOW.
on linux, acls are based on xattrs, so do these closeby:
1. listxattr -> keys (without acl related keys)
2. for all keys: getxattr
3. acl-related getxattr by acl library
for fd-based operations, we would have to open the file, but for
char / block devices this has unwanted effects, even if we do not
read from the device.
thus, we use path (or dir_fd + name) based ops here.
acl_get:
remove assumption that having an FD means it is a regular file, we try
to use FDs a much as possible.
only get the default acl for directories - other fs objects are not
expected to have a default acl.
the path needs to be encoded also for the case when we have an fd,
it is needed to get the default acl for directories.
also: micro-opt: encode path later, not needed for ISLNK check.
acl_set:
remove the "if False" branch, it is the same here: the fd-based api
only supports access ACLs, but not default ACLs, so we always need
to use the path-based api here.
if scandir does not get a path, it can't prefix it in front of the
filename in the direntries it returns, so dirent.path == dirent.name.
thus, we just only use dirent.name and construct the full path.
races via changing path components can be avoided by opening the
parent directory and using parent_fd + file_name combination with
*at style functions to access the directories' contents.
avoiding too large chunks that the repository can not store.
avoiding too small chunks that would create excessively many chunks
and way to much storage and management overhead. we only disallow
extreme cases, this does not mean that everything that is allowed
also makes sense in practice (and does not eat lots of memory and
storage space).
"default_permissions" is now enforced by borg by default to let the
kernel check uid/gid/mode based permissions.
"ignore_permissions" can be given to not enforce "default_permissions".
note: man mount.fuse explicitly tells about the security issue:
default_permissions
By default FUSE doesn't check file access permissions, ...
This option enables permission checking, restricting access
based on file mode.
This option is usually useful together with the allow_other
mount option.
We consider this a pitfall waiting for someone to fall into and this is
why we chose to change the default behaviour for borg.
Thomas Waldmann