Commit Graph

868 Commits

Author SHA1 Message Date
Thomas Waldmann b2f460d591 fix filenames used for locking, update docs about locking 2015-07-13 23:20:46 +02:00
Thomas Waldmann b539169ec1 update CHANGES 2015-07-13 23:03:09 +02:00
TW b3f135c642 Merge pull request #94 from ThomasWaldmann/locking
New Locking code
2015-07-13 19:27:24 +02:00
Thomas Waldmann 43bc1d5b6a improved tox configuration, documented there how to invoke it
the usage of some py.test features in the previous locking changesets somehow broke the test environment.

it seems to help to point py.test to some specific package or module (and not just invoke "py.test" without arguments).
2015-07-13 18:45:49 +02:00
Thomas Waldmann 2deb520e67 locking code: extract timeout/sleep code into reusable TimeoutTimer class 2015-07-13 16:45:18 +02:00
Thomas Waldmann e4c519b1e9 new locking code
exclusive locking by atomic mkdir fs operation
on top of that, shared (read) locks and exclusive (write) locks using a json roster.
2015-07-13 13:55:28 +02:00
Thomas Waldmann 434dac0e48 move locking code to own module, same for locking tests
fix imports, no other changes.
2015-07-12 23:41:52 +02:00
TW dda78d1abd Merge pull request #93 from ThomasWaldmann/mockfix
Mockfix
2015-07-12 23:38:54 +02:00
Thomas Waldmann d8e9a9bf96 skip test_crash_before_compact test for RemoteRepository
it was silently failing until recently. and it can't work the way it is on RemoteRepository.
it's still active (and now even really working) for the (local) Repository tests.
2015-07-12 23:29:34 +02:00
Thomas Waldmann 414dba3de7 remove usage of evil / broken unittest.mock, use mock from pypi
see testsuite.mock docstring for more details.

one test shows brokenness right now that was hidden / silent until now.
2015-07-12 23:08:44 +02:00
TW 6ed086498d Merge pull request #90 from RonnyPfannschmidt/entrypoints
use entrypoints instead of scripts, for better use of the wheel format and modern installs
2015-07-12 15:28:40 +02:00
Ronny Pfannschmidt 922a1f6bb7 never fall back to distutils, any sane install method uses setuptools 2015-07-12 15:07:24 +02:00
Ronny Pfannschmidt 45e89add52 switch to entrypoints 2015-07-12 15:00:47 +02:00
TW f6ba79d801 Merge pull request #89 from ThomasWaldmann/RepoCache
create a RepositoryCache implementation that can cope with any amount…
2015-07-12 00:28:12 +02:00
Thomas Waldmann bd354d7bb4 create a RepositoryCache implementation that can cope with any amount of data, fixes attic #326
the old code blows up with an integer OverflowError when the cache file goes beyond 2GiB size.
the new code just reuses the Repository implementation as a local temporary key/value store.

still an issue: if the place where the temporary RepositoryCache is stored (usually /tmp) can't
cope with the cache size and runs full.

if you copy data from a fuse mount, the cache size is the copied deduplicated data size.
so, if you have lots of data to extract (more than your /tmp can hold), rather do not use fuse!

besides fuse mounts, this also affects attic check and cache sync (in these cases, only the
metadata size counts, but even that can go beyond 2GiB for some people).
2015-07-12 00:18:49 +02:00
Thomas Waldmann afae720112 document environment variables 2015-07-11 20:08:22 +02:00
Thomas Waldmann d1bb892c3d fix placement of return codes docs 2015-07-11 19:45:37 +02:00
Thomas Waldmann 05b21e3c0d document return codes 2015-07-11 19:27:00 +02:00
TW 4b81f380f8 Merge pull request #88 from ThomasWaldmann/py3style
style and cosmetic fixes, no semantic changes
2015-07-11 18:39:42 +02:00
Thomas Waldmann 0580f2b4eb style and cosmetic fixes, no semantic changes
use simpler super() syntax of python 3.x

remove fixed errors/warnings' codes from setup.cfg flake8 configuration

fix file exclusion list for flake8
2015-07-11 18:31:49 +02:00
TW 13f4006c03 Merge pull request #87 from ThomasWaldmann/tmpfile
use borg-tmp as prefix for temporary files / directories
2015-07-11 17:33:50 +02:00
Thomas Waldmann a59211f295 use borg-tmp as prefix for temporary files / directories
also: remove some unused temp dir. code
2015-07-11 17:22:12 +02:00
TW bd5e1ff015 Merge pull request #75 from ThomasWaldmann/master
update AUTHORS
2015-07-04 13:50:28 +02:00
TW 729a48fa5e Merge pull request #76 from JaCoB1123/internals-compression
Update docs/internals.rst to mention compression changes
2015-07-03 00:04:59 +02:00
Jan Bader 8de8073345 Improve wording 2015-07-02 23:23:45 +02:00
Jan Bader 028e12473d Update internals to reflect new --compression argument 2015-07-02 23:21:28 +02:00
Thomas Waldmann 95828c576d update AUTHORS
I merged some pull requests of other developers into borg and now added
them to the developers / contributors list, with permission.

In general, feel free to add yourself to the list if you contributed something
to borg - just make a pull request including your addition to AUTHORS.

That's way easier for me than having to ask whether you want to be in there
and how precisely... - just write your entry yourself!
2015-07-02 23:05:14 +02:00
Thomas Waldmann 77577b7417 add related projects, fix web site url 2015-06-29 23:30:36 +02:00
Thomas Waldmann 9ead4097cf we are now on #borgbackup on chat.freenode.net
it seems like there is currently no bureaucracy required, freenode web site says group registration is suspended.
i also asked on the freenode channel, they said just make sure you are right here and use it. so we do that now.
2015-06-29 23:07:07 +02:00
Thomas Waldmann 4068fc1e31 clarify help text, fixes #73 2015-06-28 14:02:38 +02:00
TW 562f3c7c33 Merge pull request #72 from ThomasWaldmann/loggedio-exceptions
Loggedio exceptions
2015-06-27 22:17:02 +02:00
Thomas Waldmann 08688fbc13 Merge branch 'master' into loggedio-exceptions
Conflicts:
	borg/repository.py
2015-06-27 22:02:26 +02:00
TW 3303619b5f Merge pull request #69 from ThomasWaldmann/fix-prune-options
the short prune options without "keep-" are deprecated, so do not sug…
2015-06-26 01:19:29 +02:00
TW 6709ec522d Merge pull request #68 from ThomasWaldmann/improve-exit-logging
improve at-end error logging
2015-06-26 00:30:56 +02:00
Thomas Waldmann b92dd1bab2 the short prune options without "keep-" are deprecated, so do not suggest them 2015-06-26 00:04:35 +02:00
Thomas Waldmann 89db9b8b9e improve at-end error logging
always use archiver.print_error, so it goes to sys.stderr

always say "Error: ..." for errors

for rc != 0 always say "Exiting with failure status ..."

catch all exceptions subclassing Exception, so we can log them in same way and set exit_code=1
2015-06-25 23:57:38 +02:00
TW e4c4d458a6 Merge pull request #67 from ThomasWaldmann/zlib-compression-level
borg create --compression 0..9 for variable compression
2015-06-25 22:41:43 +02:00
Thomas Waldmann 6964799d13 borg create --compression 0..9 for variable compression 2015-06-25 22:16:23 +02:00
Thomas Waldmann 2944bbe6c6 update CHANGES 2015-06-21 02:36:22 +02:00
TW a487e16c16 Merge pull request #62 from ThomasWaldmann/chunker-params
Chunker params, fixes #16
2015-06-21 02:11:54 +02:00
Thomas Waldmann 41a37e77db add a misc docs directory, add a usecase for --chunker-params 2015-06-21 02:11:02 +02:00
Thomas Waldmann 54e8dd8419 misc chunker parameter changes
- use power-of-2 sizes / n bit hash mask so one can give them more easily
- chunker api: give seed first, so we can give *chunker_params after it
- fix some tests that aren't possible with 2^N
- make sparse file extraction zero detection flexible for variable chunk max size
2015-06-21 01:46:41 +02:00
TW 44ec86460b Merge pull request #60 from ThomasWaldmann/chunk-counts
determine and report chunk counts in chunks index
2015-06-20 14:32:19 +02:00
Thomas Waldmann 3b9b976f2a borg create --chunker-params=... 2015-06-20 01:20:46 +02:00
Thomas Waldmann 6d0a00496a determine and report chunk counts in chunks index
borg info repo::archive now reports unique chunks count, total chunks count

also: use index->key_size instead of hardcoded value
2015-06-19 23:53:23 +02:00
TW b41b7d06c5 Merge pull request #59 from ThomasWaldmann/fix-no-remote-borg
better Exception msg if there is no Borg installed on the remote repo…
2015-06-18 23:35:56 +02:00
Thomas Waldmann 2743ab1593 better Exception msg if there is no Borg installed on the remote repository server
(still a bit ugly to get even 2 tracebacks)
2015-06-18 23:18:05 +02:00
TW 9eaaaed95b Merge pull request #55 from pguth/patch-1
Update installation.rst
2015-06-15 15:07:54 +02:00
Per Guth e168b41406 Update installation.rst 2015-06-15 14:40:11 +02:00
Per Guth 041357e48a Update installation.rst
Replicated Ubuntus dependency list.
2015-06-15 14:39:04 +02:00