1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2024-12-25 17:27:31 +00:00
Commit graph

5345 commits

Author SHA1 Message Date
Thomas Waldmann
23eeded7c5 fix --read-special behaviour: follow symlinks pointing to special files
also: added a test for this.
2019-02-20 10:13:09 +01:00
Thomas Waldmann
ec17f0a607 check for stat race conditions, see #908
we must avoid a handler processing a fs item of wrong file type,
so check if it has changed.
2019-02-20 09:16:57 +01:00
Thomas Waldmann
b4ca919d02 add O_NOFOLLOW to base flags, see #908
scenario:

- x is a regular file
- borg does stat on x: is a regular file
- so borg dispatches to process_file
- attack: x gets replaced by a symlink (mv symlink x)
- in process_file, borg opens x and must not follow the symlink nor
  continue processing as a normal file, but rather error in open()
  due to NOFOLLOW.
2019-02-17 05:17:52 +01:00
Thomas Waldmann
39922e88e5 micro-opt: get xattrs directly before acls
on linux, acls are based on xattrs, so do these closeby:

1. listxattr -> keys (without acl related keys)
2. for all keys: getxattr
3. acl-related getxattr by acl library
2019-02-17 02:46:03 +01:00
Thomas Waldmann
85b711fc88 opening device files is troublesome, don't do it
for fd-based operations, we would have to open the file, but for
char / block devices this has unwanted effects, even if we do not
read from the device.

thus, we use path (or dir_fd + name) based ops here.
2019-02-14 09:20:04 +01:00
Thomas Waldmann
b960d3cd23 linux: acl_(get|set) - adapt to more FD usage / default acl for dirs
acl_get:

remove assumption that having an FD means it is a regular file, we try
to use FDs a much as possible.

only get the default acl for directories - other fs objects are not
expected to have a default acl.

the path needs to be encoded also for the case when we have an fd,
it is needed to get the default acl for directories.

also: micro-opt: encode path later, not needed for ISLNK check.

acl_set:

remove the "if False" branch, it is the same here: the fd-based api
only supports access ACLs, but not default ACLs, so we always need
to use the path-based api here.
2019-02-14 09:20:04 +01:00
Thomas Waldmann
66dd25ebc4 when scandir gets called with an FD, dirent.path is not usable
if scandir does not get a path, it can't prefix it in front of the
filename in the direntries it returns, so dirent.path == dirent.name.

thus, we just only use dirent.name and construct the full path.
2019-02-14 09:20:04 +01:00
Thomas Waldmann
833c49f834 use *at style functions (e.g. openat, statat) to avoid races
races via changing path components can be avoided by opening the
parent directory and using parent_fd + file_name combination with
*at style functions to access the directories' contents.
2019-02-14 09:20:04 +01:00
Thomas Waldmann
ad5b9a1dfd _process / process_*: change to kwargs only
we'll add/remove some args soon, so many pos args would be just bad.
2019-02-14 09:20:03 +01:00
Thomas Waldmann
8220c6eac8 move/refactor Archive._open_rb function to helpers.os_open
also:
- add and use OsOpen context manager
- add O_NONBLOCK, O_NOFOLLOW, O_NOCTTY (inspired by gnu tar)
2019-02-14 09:20:03 +01:00
Thomas Waldmann
677102f292 process_file: avoid race condition: stat data vs. content
always open the file and then do all operations with the fd:
- fstat
- read
- get xattrs, acls, bsdflags
2019-02-14 09:20:03 +01:00
Thomas Waldmann
dca04ba538 _process: remove an infrequently used micro-opt
st param was only given at the root paths of the recursion.
we can just drop that and make the code simpler.
2019-02-14 09:20:03 +01:00
TW
ce69fc1e19
Merge pull request #4238 from ThomasWaldmann/fixed-block-chunker
implement a fixed block size chunker, fixes #1086
2019-02-14 09:04:02 +01:00
Thomas Waldmann
7f46eb99aa update docs about fixed chunker and chunker algo spec needed 2019-02-13 06:30:13 +01:00
Thomas Waldmann
ac0803fe0b chunker algorithms: use constants to avoid typos 2019-02-13 04:36:09 +01:00
Thomas Waldmann
be2c061733 chunker params parsing: add more validation
avoiding too large chunks that the repository can not store.

avoiding too small chunks that would create excessively many chunks
and way to much storage and management overhead. we only disallow
extreme cases, this does not mean that everything that is allowed
also makes sense in practice (and does not eat lots of memory and
storage space).
2019-02-13 04:24:14 +01:00
Thomas Waldmann
80e0b42f7d add fixed blocksize chunker, fixes #1086 2019-02-13 04:24:14 +01:00
Thomas Waldmann
c4ffbd2a17 prepare to support multiple chunkers 2019-02-13 04:24:14 +01:00
TW
de55d763a4
Merge pull request #4357 from ThomasWaldmann/xattr-tests
re-add the code that checks if we run under fakeroot, fixes #4291
2019-02-12 14:37:39 +01:00
Thomas Waldmann
12a18b955e re-add the code that checks if we run under fakeroot, fixes #4291
code taken from 1.1-maint.

running as a user, with or without fakeroot does not have the test
fails in test_extract_capabilities any more.
2019-02-12 05:08:06 +01:00
TW
9041a315ba
Merge pull request #4349 from donnyward/master
Update bundled lz4 code to 1.8.3, fixes #4209
2019-02-12 02:43:12 +01:00
TW
095a109af4
Merge pull request #4347 from ThomasWaldmann/fix-fuse-test-fakeroot
test_mount_hardlinks: get rid of fakeroot-caused test fails, fixes #3389
2019-02-12 00:13:38 +01:00
Thomas Waldmann
e878ca5189 test_mount_hardlinks: get rid of fakeroot-caused test fails, fixes #3389 2019-02-11 17:47:34 +01:00
TW
6de90d9460
Merge pull request #4354 from ThomasWaldmann/fuse-default-options-master
security fix: configure FUSE with "default_permissions", fixes #3903
2019-02-11 17:46:30 +01:00
Thomas Waldmann
672c2c99a7 security fix: configure FUSE with "default_permissions", fixes #3903
"default_permissions" is now enforced by borg by default to let the
kernel check uid/gid/mode based permissions.

"ignore_permissions" can be given to not enforce "default_permissions".

note: man mount.fuse explicitly tells about the security issue:

    default_permissions
	By  default FUSE doesn't check file access permissions, ...
	This option enables permission checking, restricting access
	based on file mode.
	This option is usually useful together with the allow_other
	mount option.

We consider this a pitfall waiting for someone to fall into and this is
why we chose to change the default behaviour for borg.
2019-02-11 13:01:24 +01:00
Donny Ward
af449cdd1f Update bundled lz4 code to 1.8.3, fixes #4209 2019-02-10 20:36:29 -08:00
TW
fa9d3263de
Merge pull request #4345 from ThomasWaldmann/update-scripts-master
signing binaries with Qubes OS support, docs
2019-02-11 02:45:46 +01:00
Thomas Waldmann
305efb5b01 signing binaries with Qubes OS support, docs 2019-02-11 02:44:44 +01:00
TW
ba5185cbdb
Merge pull request #4340 from SanskritFritz/master
More complete shell completions for borg mount -o
2019-02-11 01:16:40 +01:00
SanskritFritz
bb98a1dae4 Option arguments for zsh tab completion 2019-02-10 20:33:23 +01:00
SanskritFritz
7becc03608 More complete shell completions for borg mount -o 2019-02-10 20:33:15 +01:00
anarcat
10a4e011c0 add warnings on repository copies to avoid future problem like #4272 (#4285)
docs: add warnings about repo copies to avoid problems like #4272
2019-02-10 20:23:59 +01:00
TW
9259000d73
Merge pull request #4335 from SanskritFritz/master
master - Shell completions.
2019-02-08 20:05:04 +01:00
SanskritFritz
8435206cc3 Shell completions for borg compact 2019-02-08 16:44:36 +01:00
SanskritFritz
0a97274bd1 New shell completions for borg 1.1.9 2019-02-08 14:27:14 +01:00
SanskritFritz
ff52d4225b Added shell completions for borg help 2019-02-06 16:49:31 +01:00
TW
2b1c223917
Merge pull request #4327 from ThomasWaldmann/forward-ports-16
master forward ports (16)
2019-02-05 17:08:38 +01:00
Thomas Waldmann
a9d562de17 work around some Microsoft WSL issues
see there:

https://github.com/borgbackup/borg/issues/1961

and especially there (not implemented sync_file_range):

https://github.com/Microsoft/WSL/issues/645
2019-02-05 15:15:50 +01:00
Thomas Waldmann
4fc23f532e work around the backslash issue, see #4280
thanks to AJ Jordan @strugee for finding this.

while using \\ gives a single backslash when using "borg help pattern"
on the shell, our rendered html docs did not show the single backslash
at all.

i was lazy do dig into that issue, so i just used the word "backslash".

:-)
2019-02-05 15:08:11 +01:00
TW
9a3fbdd751
Merge pull request #4316 from ThomasWaldmann/init-make-parentdirs
borg init --make-parent-dirs parent1/parent2/repo_dir, fixes #4235
2019-02-05 04:49:22 +01:00
TW
b79c26a45f
Merge pull request #4319 from ThomasWaldmann/msgpack-061
support msgpack 0.6.1
2019-02-05 03:59:21 +01:00
Thomas Waldmann
f479850652 add comment about msgpack >= 0.6.1 features for later 2019-02-05 03:08:19 +01:00
Thomas Waldmann
940e107add cleanup msgpack requirements
we do not expect 0.5.x with x > 6 any more.

also remove outdated comments from setup.py.
2019-02-05 02:37:22 +01:00
Thomas Waldmann
0ebfaa5b61 allow msgpack 0.6.1, fixes #4308 2019-02-05 02:32:36 +01:00
Thomas Waldmann
600e798201 borg init --make-parent-dirs parent1/parent2/repo_dir, fixes #4235 2019-02-04 17:12:11 +01:00
TW
b204201fb5
Merge pull request #4302 from ThomasWaldmann/repair-output
add archive name to check --repair output, fixes #3447
2019-02-04 03:29:58 +01:00
TW
c3f40de606
cache_sync: compute size/count stats, borg info: consider part files (#4286)
cache_sync: compute size/count stats, borg info: consider part files

fixes #3522
2019-02-04 03:26:45 +01:00
TW
50666c3e44
Merge pull request #4315 from ThomasWaldmann/check-msgpack-master
check msgpack version, terminate for unsupported versions
2019-02-03 04:34:51 +01:00
TW
de288cd6b2
Merge pull request #4314 from ThomasWaldmann/fix-tests-master
fix duplicate test method name, fixes #4311
2019-02-03 04:10:15 +01:00
Thomas Waldmann
18c9feb7e3 check msgpack version, terminate for unsupported versions 2019-02-03 03:54:41 +01:00