Commit Graph

2917 Commits

Author SHA1 Message Date
Thomas Waldmann 17c77a5dc5 xattr: dynamically grow result buffer until it fits, fixes #1462
this also fixes the race condition seen in #1462 because there is only 1 call now.
either it succeeds, then we get the correct length as result and truncate the result value to that length.
or it fails with ERANGE, then we grow the buffer to double size and repeat.
or it fails with some other error, then we throw OSError.
2016-08-12 16:56:19 +02:00
enkore 6bf48bbf67 Merge pull request #1457 from ThomasWaldmann/explain-compat-typeerror
explain the confusing TypeError, fixes #1456
2016-08-12 15:16:36 +02:00
TW 32e6507366 Merge pull request #1468 from enkore/issue/1467
Print active env var override by default
2016-08-12 14:05:33 +02:00
Marian Beermann c61a9e8aa0 Print active env var override by default
Fixes #1467
2016-08-12 13:00:53 +02:00
enkore e05258f93f Merge pull request #1461 from ThomasWaldmann/improve-docs
docs: improve prune examples
2016-08-12 12:54:24 +02:00
enkore f8b2ffe999 Merge pull request #1458 from ThomasWaldmann/merge-1.0-maint
Merge 1.0 maint
2016-08-11 14:18:03 +02:00
Thomas Waldmann 6e658a5a6c docs: improve prune examples 2016-08-10 15:45:57 +02:00
enkore a951d23d27 Merge pull request #1460 from ThomasWaldmann/fix-init
add transaction_id assertion
2016-08-10 14:29:46 +02:00
Thomas Waldmann 7b5772df2d add transaction_id assertion
an acd_cli (amazon cloud drive fuse filesystem) user had "borg init" crash in the line below.

by adding the assertion we tell that we do not expect the transaction_id to be None there,
so it is easier to differentiate from a random coding error.
2016-08-10 13:56:06 +02:00
enkore e89c5b11b8 Merge pull request #1459 from ThomasWaldmann/larger-items-stream-chunks
larger item metadata stream chunks, fixes #1452
2016-08-09 23:37:17 +02:00
enkore 006d1df517 Merge pull request #1450 from enkore/f/recreatefactors
minor recreate refactor
2016-08-09 23:28:58 +02:00
Marian Beermann 7d9fba5a41 ArchiveRecreater.copy_items: progress from target instead of source
(it produces the same output, just less confusing)
2016-08-09 21:20:13 +02:00
TW ff5d9a3f12 Merge pull request #1447 from enkore/f/debug-topic
Debugging topics and some other minor stuff
2016-08-09 20:53:07 +02:00
Thomas Waldmann 2624d6f818 larger item metadata stream chunks, fixes #1452
increasing the mask (target chunk size) from 14 (16kiB) to 17 (128kiB).
this should reduce the amount of item metadata chunks an archive has to reference to 1/8.
this does not completely fix #1452, but at least enables a 8x larger item metadata stream.
2016-08-09 20:30:50 +02:00
Thomas Waldmann c955d59273 Merge branch '1.0-maint' 2016-08-09 20:09:13 +02:00
TW 316c0cd6f0 Merge pull request #1455 from ThomasWaldmann/fix-compr-buffer-size
LZ4: dynamically enlarge the (de)compression buffer, fixes #1453
2016-08-09 18:36:55 +02:00
TW 4c51bed665 Merge pull request #1454 from ThomasWaldmann/enfore-max-object-size-put
repo: do not put objects that we won't get, fixes #1451
2016-08-09 18:17:03 +02:00
Thomas Waldmann 20392f8dd9 repo: split size check into too small and too big
also add a hint if somebody needs to restore an archive that has too big objects.
2016-08-09 18:05:22 +02:00
Thomas Waldmann a360307938 repo: do not put objects that we won't get, fixes #1451
we will not get() objects that have a segment entry larger than MAX_OBJECT_SIZE.
thus we should never produce such entries.

also: introduce repository.MAX_DATA_SIZE that gives the max payload size.
2016-08-09 18:05:22 +02:00
Thomas Waldmann fc92822b6c explain the confusing TypeError, fixes #1456 2016-08-09 17:35:27 +02:00
Thomas Waldmann b0e7bb5ddc fixup: use thread-local buffer
start with 0 bytes length (saves memory in case lz4 is not used).
always grow when a bigger buffer is needed.
avoid per-call reallocation / freeing / garbage.
2016-08-09 17:05:24 +02:00
Thomas Waldmann d3000a7e5d LZ4: dynamically enlarge the (de)compression buffer, fixes #1453
the statically allocated COMPR_BUFFER was right size for chunks,
but not for the archive item which could get larger if you have
many millions of files/dirs.
2016-08-09 02:52:21 +02:00
enkore 51054da692 Merge pull request #1444 from ThomasWaldmann/merge-1.0-maint
Merge 1.0 maint
2016-08-08 20:28:35 +02:00
Marian Beermann 0a65e83df3 check --verify-data: remove incorrect help paragraph
(since d0ec7e76bb - doesn't mean this probably useful feature may not
 be reimplemented, on the contrary)
2016-08-08 16:38:31 +02:00
Marian Beermann bec5051ea5 helpers.format_timedelta: use timedelta.total_seconds() 2016-08-08 16:38:31 +02:00
Marian Beermann 389ca94407 Silence repeated "Processing files..." log entries when --no-files-cache 2016-08-08 16:38:31 +02:00
Marian Beermann 4d7a52d8e0 --debug-topic for granular debug logging
Especially: disables file-compression logging by default
2016-08-08 16:38:31 +02:00
Marian Beermann 7b70b74ad1 ArchiveRecreater.incref_partial_chunks try to use __contains__ 2016-08-08 15:54:47 +02:00
Marian Beermann 5fe6c09c34 Refactor ArchiveRecreater.try_resume 2016-08-08 15:54:41 +02:00
Thomas Waldmann 6c1c87f7ae add forgotten usage help file from build_usage 2016-08-06 01:28:02 +02:00
Thomas Waldmann dbe862f3d8 Merge branch '1.0-maint' into merge-1.0-maint 2016-08-06 01:24:22 +02:00
Thomas Waldmann a56b010960 ran build_usage 2016-08-05 20:26:09 +02:00
Thomas Waldmann 5b575f69dc CHANGES: add date to 1.0.7rc1 2016-08-05 20:23:47 +02:00
TW 58a9ccb49d Merge pull request #1441 from ThomasWaldmann/update-1.0-changes
update CHANGES
2016-08-05 20:01:49 +02:00
Thomas Waldmann b79e913244 update CHANGES 2016-08-05 19:56:10 +02:00
TW 18654eaf91 Merge pull request #1371 from ThomasWaldmann/fix-deadlock
fix deadlock
2016-08-05 19:48:51 +02:00
Thomas Waldmann 33e3348208 locking: better differentiate new vs. old clients, lock upgrade for replay
old clients use self.exclusive = None and do a read->write lock upgrade when needed.
new clients use self.exclusive = True/False and never upgrade.

replay fakes an old client by setting self.exclusive = None to get a lock upgrade if needed.
2016-08-05 19:11:16 +02:00
Thomas Waldmann 64dcbbfdd0 change RPC API, fix remote repo tests 2016-08-05 19:11:16 +02:00
Thomas Waldmann 1e739fd52d fix local repo / upgrader tests 2016-08-05 19:06:38 +02:00
Thomas Waldmann d3d51e12ea rename UpgradableLock to Lock
lock upgrading is troublesome / may deadlock, do not advertise it.
2016-08-05 19:06:09 +02:00
Thomas Waldmann 2a355e547e make sure we have a excl. lock when starting a transaction
if we don't, we try to upgrade the lock.
this is to support old clients talking to a new server and also
to avoid bad consequences from coding mistakes for new clients.
2016-08-05 18:55:32 +02:00
Thomas Waldmann 26007c0162 add Lock.got_exclusive_lock 2016-08-05 18:52:20 +02:00
TW 976925c625 Merge pull request #1436 from ThomasWaldmann/fix-fuse-tests
fix fuse tests on OS X, fixes #1433
2016-08-05 15:08:16 +02:00
TW 109c095c42 Merge pull request #1435 from ThomasWaldmann/fix-1430
fix unintended file cache eviction, fixes #1430
2016-08-05 14:19:56 +02:00
TW 3fac548b34 Merge pull request #1431 from ThomasWaldmann/osxfuse-3.4
Vagrantfile: use FUSE for macOS 3.4.1
2016-08-05 14:08:04 +02:00
TW d050efbe34 Merge pull request #1438 from ThomasWaldmann/debug-dump-repo-objs
borg debug-dump-repo-objs
2016-08-05 14:06:20 +02:00
Thomas Waldmann b96bc155ac fix unintended file cache eviction, fixes #1430
thanks much to e477 for diagnosing this and finding the right fix.
2016-08-05 13:59:21 +02:00
Thomas Waldmann 4fa420ef29 borg debug-dump-repo-objs
dump all objects stored in the repository (decrypted and decompressed)
2016-08-05 13:48:48 +02:00
TW 33ee2b0897 Merge pull request #1437 from robmv/patch-2
Add backup using stable filesystem names recommendation
2016-08-04 09:56:38 +02:00
Robert Marcano bc6050bc3c Add backup using stable filesystem names recommendation 2016-08-03 22:54:51 -04:00