Commit Graph

753 Commits

Author SHA1 Message Date
Dmitry Astapov 6be2734b7e Do not create items_buffer and chunker unless we are creating new archive
Listing repositories with lots of archives on low-memory systems would cause attic to run out of memory due to items_buffer and chunker being created for each visited archive.

See https://github.com/jborg/attic/issues/163
2015-03-18 12:42:28 +00:00
Thomas Waldmann 3daaceb6bf PR #238 - Merge branch 'fix_fuse_error_msg' into merge 2015-03-18 03:22:38 +01:00
Thomas Waldmann ee1dc89b52 fix misleading hint the fuse ImportError handler gave, fixes #237 2015-03-18 02:33:34 +01:00
Thomas Waldmann b7cefacb69 Merge branch 'python_cleanup' into merge
Conflicts:
	attic/archiver.py
	attic/remote.py
2015-03-18 00:23:41 +01:00
Thomas Waldmann e9aacad9a3 remove unused imports 2015-03-17 23:54:05 +01:00
Thomas Waldmann 4ef6125f45 cleanup pep8 issues found by pycharm 2015-03-17 23:47:21 +01:00
Thomas Waldmann 3a33d57c7d triple-double-quotes for docstrings 2015-03-17 23:23:56 +01:00
Thomas Waldmann a21da5d2e0 no bare except 2015-03-17 23:16:12 +01:00
Thomas Waldmann dee9869a03 use "with" with open() 2015-03-17 23:09:14 +01:00
Thomas Waldmann fec5572836 no Class(object) in py3 2015-03-17 23:03:36 +01:00
Thomas Waldmann 2f5e5e01ea PR #235 - Merge branch 'fix_xattr' into merge 2015-03-15 00:22:01 +01:00
Thomas Waldmann 6116fa72b2 give specific path to xattr.is_enabled(), disable symlink setattr call that always fails 2015-03-15 00:20:50 +01:00
Thomas Waldmann 89ffe98ba2 PR #234 - Merge branch 'fix_pipe' into merge 2015-03-14 20:53:06 +01:00
Thomas Waldmann cbbe0fcc9b serve: use os.write(stdout_fd, ...), fixes #233
this way, serve() is more consistent with the other code, which always uses os.read/write (not sys.std*.buffer.read/write).

also: reduce code duplication a bit.
2015-03-14 19:45:01 +01:00
Thomas Waldmann 615ca87c17 Merge branch 'msgpack_corruption' into merge 2015-03-12 20:07:46 +01:00
Thomas Waldmann 4c7d0762a9 as msgpack-python 0.4.6 is released now, just use that. 2015-03-12 20:06:53 +01:00
Thomas Waldmann 60ac104a8f PR #231 - Merge branch 'check_last_N' of https://github.com/ThomasWaldmann/attic into merge 2015-03-11 03:12:37 +01:00
Thomas Waldmann 90c50e3171 implement check --last N
Note: of course it can only check for orphaned objects, if it has processed all archives in the repo.
Thus this check is skipped as soon as you give --last N option.

The numbers shown in progress indicator are (N,T).
N is the number of the currently checked archive (starts at T as it first checks latest archive).
T is the total number of archives.
2015-03-11 03:04:12 +01:00
Thomas Waldmann 97b5154fc5 check: sort archives in reverse time order 2015-03-10 01:11:18 +01:00
Thomas Waldmann f1fe5be4c1 PR #230 - Merge branch 'msgpack_corruption' of https://github.com/ThomasWaldmann/attic into merge 2015-03-09 23:38:19 +01:00
Thomas Waldmann a67d4219c3 avoid defect python-msgpack releases, fixes #171, fixes #185 2015-03-09 23:35:56 +01:00
Thomas Waldmann 8aebdd6c5b PR #229 - Merge branch 'fix_127' of https://github.com/ThomasWaldmann/attic into merge 2015-03-09 22:08:15 +01:00
Thomas Waldmann becae42618 check unpacked data from RPC for tuple type and correct length, fixes #127 2015-03-09 21:59:10 +01:00
Thomas Waldmann 3b744d2ee8 fix Repository._active_txn state when lock upgrade fails 2015-03-09 20:45:31 +01:00
Thomas Waldmann 954b26f64c RPCError: include the exception args we get from remote
Without this, you just got "RCPError: AttributeError", now you get (e.g.):

RPCError: AttributeError(b"'Repository' object has no attribute 'segments'",)
2015-03-09 17:01:29 +01:00
Thomas Waldmann 67eaa9d654 PR #225 - Merge branch 'fix_cache_close' of https://github.com/ThomasWaldmann/attic into merge 2015-03-09 16:21:15 +01:00
Thomas Waldmann 6e6819e626 attic delete: add repository + local cache deletion 2015-03-09 16:02:06 +01:00
Thomas Waldmann d3fe74d4c0 Cache: do not try to release the lock twice
If Cache was already closed and __del__ was called, it called close() again
and crashed when trying to release the lock again.
2015-03-09 15:17:56 +01:00
Thomas Waldmann 9841af5542 better attic create -v output
Added a indicator character to the left for (A)dded, (M)odified, (U)nchanged status
of regular files. Lowercase indicators are for special files.

You may or may not want to use grep to filter out U and d.
2015-03-08 19:18:21 +01:00
Thomas Waldmann d9cfca8989 PR #223 - Merge branch 'fix_45' of https://github.com/ThomasWaldmann/attic into merge 2015-03-08 15:09:01 +01:00
Thomas Waldmann 4633931413 add global option --no-cache-files to lower memory consumption
When given, attic does not use the "files" cache. Saves about 240B RAM per file
(that sounds only a little, but consider that backups nowadays are often millions of files).

So try this if attic eats more memory than you have as RAM (usually means paging or
MemoryErrors). Of course, saving memory is not for free. In my one experiment, run time
increased from 3.5 to 23 minutes (my system has enough RAM).
2015-03-08 15:01:24 +01:00
Thomas Waldmann 084f6e1602 PR #222 - Merge branch 'issue_189' of https://github.com/ThomasWaldmann/attic into merge 2015-03-08 04:20:13 +01:00
Thomas Waldmann be29e5f6f4 fix traceback when trying to do unsupported passphrase change, fixes #189 2015-03-08 04:19:25 +01:00
Thomas Waldmann c8b23b507d PR #221 - Merge branch 'issue_139' of https://github.com/ThomasWaldmann/attic into merge 2015-03-08 02:37:43 +01:00
Thomas Waldmann e425545c10 datetime does not like the year 10.000, fixes issue #139 2015-03-08 02:32:33 +01:00
Thomas Waldmann 70e907f4ed PR #184 - Merge branch 'fuse-filesizes' of https://github.com/dnnr/attic into merge 2015-03-07 00:09:51 +01:00
Thomas Waldmann 0d2cd28c19 PR #214 - Merge branch 'crypto_cleanup' of https://github.com/thomaswaldmann/attic into merge 2015-03-06 23:50:15 +01:00
Thomas Waldmann 29449e40bc PR #208 - Merge branch 'stdin_stdout_support' of https://github.com/thomaswaldmann/attic into merge 2015-03-06 23:28:00 +01:00
Thomas Waldmann fbaa505191 PR #206 - Merge branch 'fixes' of https://github.com/thomaswaldmann/attic into merge 2015-03-06 23:18:08 +01:00
Thomas Waldmann 8c4d290a93 PR #173 - Merge branch 'rpc-whitelist' of https://github.com/dnnr/attic into merge 2015-03-06 22:55:53 +01:00
Thomas Waldmann a5fe6160ba PR #150 - Merge branch 'normalize_paths' of https://github.com/wavexx/attic into merge 2015-03-06 22:35:16 +01:00
Thomas Waldmann edb8f21411 PR #198 - Merge https://github.com/evanhempel/attic into merge 2015-03-06 21:57:06 +01:00
Thomas Waldmann 7555792b4f PR #191 - Merge https://github.com/rpodgorny/attic into merge 2015-03-06 21:50:29 +01:00
Thomas Waldmann 2fbef266f9 PR #156 - Merge branch 'doc-multi' of https://github.com/anarcat/attic into merge 2015-03-06 21:46:02 +01:00
Thomas Waldmann 9e77251d8c PR #155 - Merge branch 'doc-encryption' of https://github.com/anarcat/attic into merge 2015-03-06 21:44:47 +01:00
Thomas Waldmann c191ab189f PR #153 - Merge branch 'dependencies' of https://github.com/anarcat/attic into merge 2015-03-06 21:36:44 +01:00
Thomas Waldmann dd641ddc5b PR #152 - Merge branch 'patch-1' of https://github.com/anarcat/attic into merge 2015-03-06 21:32:31 +01:00
anarcat effab97d11 clarify that simultaneous backups may be a problem 2015-03-05 09:06:20 -05:00
anarcat ecee5a0b51 PDKF is a key derivation function
do not use the word "encryption", as it is actually closer to "hashing" anyways.
2015-03-05 09:00:06 -05:00
anarcat 5f882e976d clarify the index memory usage analysis
it seems I extracted that data from [this mailing list post][] which
in turn takes it from [this github comment][].

 [this mailing list post]: http://librelist.com/browser/attic/2014/5/6/questions-and-suggestions-about-inner-working-of-attic/
 [this github comment]: https://github.com/jborg/attic/issues/26#issuecomment-35439254
2015-03-05 08:57:52 -05:00