Commit Graph

5350 Commits

Author SHA1 Message Date
TW d644323333
Merge pull request #4043 from ThomasWaldmann/use-more-fds
use more FDs, avoid race conditions on active fs
2019-02-22 12:54:16 +01:00
TW 5ba14e8d95
Merge pull request #4383 from milkey-mouse/fix-4133
Only modify window.location when redirecting
2019-02-21 11:25:43 +01:00
Milkey Mouse 7ea2bd1389
Only modify window.location when redirecting (fixes #4133)
window.location.pathname = window.location.pathname can trigger a redirect,
which created a refresh loop in usage.html.
2019-02-20 11:12:45 -08:00
Thomas Waldmann 23eeded7c5 fix --read-special behaviour: follow symlinks pointing to special files
also: added a test for this.
2019-02-20 10:13:09 +01:00
Thomas Waldmann ec17f0a607 check for stat race conditions, see #908
we must avoid a handler processing a fs item of wrong file type,
so check if it has changed.
2019-02-20 09:16:57 +01:00
TW 091bd2bd79
Merge pull request #4377 from milkey-mouse/copyright-2019
Update copyright year
2019-02-20 06:10:11 +01:00
Milkey Mouse 453e4a4b92
Update copyright year 2019-02-19 20:20:13 -08:00
Thomas Waldmann b4ca919d02 add O_NOFOLLOW to base flags, see #908
scenario:

- x is a regular file
- borg does stat on x: is a regular file
- so borg dispatches to process_file
- attack: x gets replaced by a symlink (mv symlink x)
- in process_file, borg opens x and must not follow the symlink nor
  continue processing as a normal file, but rather error in open()
  due to NOFOLLOW.
2019-02-17 05:17:52 +01:00
Thomas Waldmann 39922e88e5 micro-opt: get xattrs directly before acls
on linux, acls are based on xattrs, so do these closeby:

1. listxattr -> keys (without acl related keys)
2. for all keys: getxattr
3. acl-related getxattr by acl library
2019-02-17 02:46:03 +01:00
Thomas Waldmann 85b711fc88 opening device files is troublesome, don't do it
for fd-based operations, we would have to open the file, but for
char / block devices this has unwanted effects, even if we do not
read from the device.

thus, we use path (or dir_fd + name) based ops here.
2019-02-14 09:20:04 +01:00
Thomas Waldmann b960d3cd23 linux: acl_(get|set) - adapt to more FD usage / default acl for dirs
acl_get:

remove assumption that having an FD means it is a regular file, we try
to use FDs a much as possible.

only get the default acl for directories - other fs objects are not
expected to have a default acl.

the path needs to be encoded also for the case when we have an fd,
it is needed to get the default acl for directories.

also: micro-opt: encode path later, not needed for ISLNK check.

acl_set:

remove the "if False" branch, it is the same here: the fd-based api
only supports access ACLs, but not default ACLs, so we always need
to use the path-based api here.
2019-02-14 09:20:04 +01:00
Thomas Waldmann 66dd25ebc4 when scandir gets called with an FD, dirent.path is not usable
if scandir does not get a path, it can't prefix it in front of the
filename in the direntries it returns, so dirent.path == dirent.name.

thus, we just only use dirent.name and construct the full path.
2019-02-14 09:20:04 +01:00
Thomas Waldmann 833c49f834 use *at style functions (e.g. openat, statat) to avoid races
races via changing path components can be avoided by opening the
parent directory and using parent_fd + file_name combination with
*at style functions to access the directories' contents.
2019-02-14 09:20:04 +01:00
Thomas Waldmann ad5b9a1dfd _process / process_*: change to kwargs only
we'll add/remove some args soon, so many pos args would be just bad.
2019-02-14 09:20:03 +01:00
Thomas Waldmann 8220c6eac8 move/refactor Archive._open_rb function to helpers.os_open
also:
- add and use OsOpen context manager
- add O_NONBLOCK, O_NOFOLLOW, O_NOCTTY (inspired by gnu tar)
2019-02-14 09:20:03 +01:00
Thomas Waldmann 677102f292 process_file: avoid race condition: stat data vs. content
always open the file and then do all operations with the fd:
- fstat
- read
- get xattrs, acls, bsdflags
2019-02-14 09:20:03 +01:00
Thomas Waldmann dca04ba538 _process: remove an infrequently used micro-opt
st param was only given at the root paths of the recursion.
we can just drop that and make the code simpler.
2019-02-14 09:20:03 +01:00
TW ce69fc1e19
Merge pull request #4238 from ThomasWaldmann/fixed-block-chunker
implement a fixed block size chunker, fixes #1086
2019-02-14 09:04:02 +01:00
Thomas Waldmann 7f46eb99aa update docs about fixed chunker and chunker algo spec needed 2019-02-13 06:30:13 +01:00
Thomas Waldmann ac0803fe0b chunker algorithms: use constants to avoid typos 2019-02-13 04:36:09 +01:00
Thomas Waldmann be2c061733 chunker params parsing: add more validation
avoiding too large chunks that the repository can not store.

avoiding too small chunks that would create excessively many chunks
and way to much storage and management overhead. we only disallow
extreme cases, this does not mean that everything that is allowed
also makes sense in practice (and does not eat lots of memory and
storage space).
2019-02-13 04:24:14 +01:00
Thomas Waldmann 80e0b42f7d add fixed blocksize chunker, fixes #1086 2019-02-13 04:24:14 +01:00
Thomas Waldmann c4ffbd2a17 prepare to support multiple chunkers 2019-02-13 04:24:14 +01:00
TW de55d763a4
Merge pull request #4357 from ThomasWaldmann/xattr-tests
re-add the code that checks if we run under fakeroot, fixes #4291
2019-02-12 14:37:39 +01:00
Thomas Waldmann 12a18b955e re-add the code that checks if we run under fakeroot, fixes #4291
code taken from 1.1-maint.

running as a user, with or without fakeroot does not have the test
fails in test_extract_capabilities any more.
2019-02-12 05:08:06 +01:00
TW 9041a315ba
Merge pull request #4349 from donnyward/master
Update bundled lz4 code to 1.8.3, fixes #4209
2019-02-12 02:43:12 +01:00
TW 095a109af4
Merge pull request #4347 from ThomasWaldmann/fix-fuse-test-fakeroot
test_mount_hardlinks: get rid of fakeroot-caused test fails, fixes #3389
2019-02-12 00:13:38 +01:00
Thomas Waldmann e878ca5189 test_mount_hardlinks: get rid of fakeroot-caused test fails, fixes #3389 2019-02-11 17:47:34 +01:00
TW 6de90d9460
Merge pull request #4354 from ThomasWaldmann/fuse-default-options-master
security fix: configure FUSE with "default_permissions", fixes #3903
2019-02-11 17:46:30 +01:00
Thomas Waldmann 672c2c99a7 security fix: configure FUSE with "default_permissions", fixes #3903
"default_permissions" is now enforced by borg by default to let the
kernel check uid/gid/mode based permissions.

"ignore_permissions" can be given to not enforce "default_permissions".

note: man mount.fuse explicitly tells about the security issue:

    default_permissions
	By  default FUSE doesn't check file access permissions, ...
	This option enables permission checking, restricting access
	based on file mode.
	This option is usually useful together with the allow_other
	mount option.

We consider this a pitfall waiting for someone to fall into and this is
why we chose to change the default behaviour for borg.
2019-02-11 13:01:24 +01:00
Donny Ward af449cdd1f Update bundled lz4 code to 1.8.3, fixes #4209 2019-02-10 20:36:29 -08:00
TW fa9d3263de
Merge pull request #4345 from ThomasWaldmann/update-scripts-master
signing binaries with Qubes OS support, docs
2019-02-11 02:45:46 +01:00
Thomas Waldmann 305efb5b01 signing binaries with Qubes OS support, docs 2019-02-11 02:44:44 +01:00
TW ba5185cbdb
Merge pull request #4340 from SanskritFritz/master
More complete shell completions for borg mount -o
2019-02-11 01:16:40 +01:00
SanskritFritz bb98a1dae4 Option arguments for zsh tab completion 2019-02-10 20:33:23 +01:00
SanskritFritz 7becc03608 More complete shell completions for borg mount -o 2019-02-10 20:33:15 +01:00
anarcat 10a4e011c0 add warnings on repository copies to avoid future problem like #4272 (#4285)
docs: add warnings about repo copies to avoid problems like #4272
2019-02-10 20:23:59 +01:00
TW 9259000d73
Merge pull request #4335 from SanskritFritz/master
master - Shell completions.
2019-02-08 20:05:04 +01:00
SanskritFritz 8435206cc3 Shell completions for borg compact 2019-02-08 16:44:36 +01:00
SanskritFritz 0a97274bd1 New shell completions for borg 1.1.9 2019-02-08 14:27:14 +01:00
SanskritFritz ff52d4225b Added shell completions for borg help 2019-02-06 16:49:31 +01:00
TW 2b1c223917
Merge pull request #4327 from ThomasWaldmann/forward-ports-16
master forward ports (16)
2019-02-05 17:08:38 +01:00
Thomas Waldmann a9d562de17 work around some Microsoft WSL issues
see there:

https://github.com/borgbackup/borg/issues/1961

and especially there (not implemented sync_file_range):

https://github.com/Microsoft/WSL/issues/645
2019-02-05 15:15:50 +01:00
Thomas Waldmann 4fc23f532e work around the backslash issue, see #4280
thanks to AJ Jordan @strugee for finding this.

while using \\ gives a single backslash when using "borg help pattern"
on the shell, our rendered html docs did not show the single backslash
at all.

i was lazy do dig into that issue, so i just used the word "backslash".

:-)
2019-02-05 15:08:11 +01:00
TW 9a3fbdd751
Merge pull request #4316 from ThomasWaldmann/init-make-parentdirs
borg init --make-parent-dirs parent1/parent2/repo_dir, fixes #4235
2019-02-05 04:49:22 +01:00
TW b79c26a45f
Merge pull request #4319 from ThomasWaldmann/msgpack-061
support msgpack 0.6.1
2019-02-05 03:59:21 +01:00
Thomas Waldmann f479850652 add comment about msgpack >= 0.6.1 features for later 2019-02-05 03:08:19 +01:00
Thomas Waldmann 940e107add cleanup msgpack requirements
we do not expect 0.5.x with x > 6 any more.

also remove outdated comments from setup.py.
2019-02-05 02:37:22 +01:00
Thomas Waldmann 0ebfaa5b61 allow msgpack 0.6.1, fixes #4308 2019-02-05 02:32:36 +01:00
Thomas Waldmann 600e798201 borg init --make-parent-dirs parent1/parent2/repo_dir, fixes #4235 2019-02-04 17:12:11 +01:00