Commit Graph

192 Commits

Author SHA1 Message Date
Jonas Borgström 9ae3d10a99 cache: Properly clean up txn.tmp after a failed commit 2014-03-30 22:46:57 +02:00
Jonas Borgström 37cc63fc46 Fix one more -Werror=declaration-after-statement issue 2014-03-30 22:35:13 +02:00
Tung Dao 6d77808bec Fix for ISO C90 compliance 2014-03-30 22:43:31 +07:00
Jonas Borgström 1db4ba52f9 Make sure key files end with a new line 2014-03-28 22:56:40 +01:00
Jonas Borgström 8080f183d6 mount: Improve fuse filesystem memory efficiency
Closes #59
2014-03-28 22:51:09 +01:00
Jonas Borgström 3b80688959 mount: Load archive metadata on demand 2014-03-27 23:01:14 +01:00
Jonas Borgström af262482de mount: Use RepositoryCache
This makes a full repository mount a bit faster but it's still
too slow and memory hungry.
2014-03-26 22:42:20 +01:00
Jonas Borgström 8e078b5f91 Initial (naive) full repository fuse mount support 2014-03-26 22:06:53 +01:00
Jonas Borgström a9fc62cc9a Added option to restrict remote repository access to specific path(s)
With this option remote repository access can be restricted to a
specific path for a specific ssh key using the following line
in ~/.ssh/authorized_keys::

command="attic serve --restrict-to-path /data/clientA" ssh-rsa clientA's key
command="attic serve --restrict-to-path /data/clientB" ssh-rsa clientB's key

Closes #51.
2014-03-24 21:28:59 +01:00
Johann Klähn f47353e7ef accept names of other commands in help command 2014-03-21 22:12:15 +01:00
Jonas Borgström 8a1ebe0112 Added '--stats' option to attic prune and attic delete 2014-03-19 22:32:07 +01:00
Jonas Borgström 839dd94a59 prune: Delete all archives in a single transaction
This will improve performance and make the whole operation atomic
2014-03-19 21:58:25 +01:00
Jonas Borgström 80e7e637cc Make --stats output 78 characters wide 2014-03-19 21:52:49 +01:00
Jonas Borgström 92c333c071 Add a method to detect out of date binary extension modules 2014-03-18 22:04:08 +01:00
Jonas Borgström e398d5f632 Include "all archives" size information in "--stats" output.
Closes #54
2014-03-18 21:42:03 +01:00
Jonas Borgström c3ca45315b cache: Speed up cache rebuild by caching archive metadata requests 2014-03-13 22:29:47 +01:00
Jonas Borgström 0e39acffd3 crypto: convert attic.crypto into a cython module
This will give a slight performance boost and make the openssl
detection/linking more robust.
2014-03-10 22:50:38 +01:00
Jonas Borgström 90fe318809 Attempt to make the archive metadata resync more robust 2014-03-06 20:05:13 +01:00
Jonas Borgström 66a84c0c12 check: Fix a check --repair issue and added more tests 2014-03-06 12:05:22 +01:00
Jonas Borgström 9dce2362cc archive checkpoint: Reset timer after the checkpoint has been committed 2014-03-05 20:00:27 +01:00
Jonas Borgström 3193af5482 Remove explicit flush calls since line_buffering is enabled. 2014-03-05 19:58:29 +01:00
Jonas Borgström 11c08f412a check: No progreess output to stderr during archives check 2014-03-04 22:02:35 +01:00
Jonas Borgström 55a26e553f check: replace --phase with --repository/archives-only 2014-03-04 21:56:37 +01:00
Jonas Borgström 477e1a39dd check: Make sure all non-critical output is sent to stdout 2014-03-04 21:21:58 +01:00
Jonas Borgström a68e48bc30 check: Make item stream resync a bit more robust 2014-03-04 21:15:52 +01:00
Jonas Borgström 7be0ad609d Make the manifest rebuild code more robust
Try to make sure we've actually found msgpack data before feeding
it into msgpack.
2014-03-01 15:00:21 +01:00
Jonas Borgström 3982c34e6c Improve attic check documentation
Closes #24.
2014-02-26 23:13:48 +01:00
Jonas Borgström 678fb2c845 testsuite: Fix test test_prune_repository 2014-02-26 20:27:08 +01:00
Jonas Borgström 1b42da1fa5 Merge remote-tracking branch 'jdchristensen/prune-docs-info' 2014-02-26 20:26:37 +01:00
Jonas Borgström fd2f3ea634 Fix UnicodeEncodeError on verbose output of unicode paths
We can't assume that the stdout encoding is not less restrictive
than the filesystem encoding.
2014-02-25 12:33:23 +01:00
Dan Christensen 78c2ef6a13 When -v is specified with prune, also show the timestamp of each
archive, which is helpful for debugging.  Unify this with the
formatting done by "attic list".  The spacing is chosen so that
the attic prune output is 79 characters wide when the archive
names are at most 36 characters.
2014-02-24 19:46:56 -05:00
Dan Christensen 4c4018e1d7 Update prune documentation to use new command line options,
and also to say that the time that matters is the time each
backup completes.
2014-02-24 19:32:18 -05:00
Jonas Borgström a229db0dce check: More code cleanup 2014-02-24 23:37:21 +01:00
Jonas Borgström 9e8a944a2a check: archive metadata recovery improvements 2014-02-24 22:44:43 +01:00
Jonas Borgström c1e1d6e9dd repository: Fixed a potential infinite loop in recover_segment 2014-02-24 12:59:38 +01:00
Jonas Borgström 88ff981eee Ignore xattr errors during extract if not supported by the filesystem
Closes #46.
2014-02-23 13:15:57 +01:00
Jonas Borgström 6425d16aa8 repository: Fix potential race condition
If we crash between compact_segments() and write_index() and the
transaction deletes objects that are newer than the current index
might become undeleted.
2014-02-21 20:20:17 +01:00
Jonas Borgström bd22bc8cb2 Automatically replay segments to rebuild missing repository index 2014-02-20 23:49:35 +01:00
Dan Christensen 5d2d3f1f93 Add --dry-run option to prune. 2014-02-19 21:33:05 -05:00
Jonas Borgström e458f859d3 Added command line deprecation functionality
The following deprecations are currently handled:

- attic verify -> attic extract --dry-run
- attic prune --hourly|daily|... -> attic prune --keep-X
2014-02-19 22:53:44 +01:00
Jonas Borgström d551b13a23 Use StableDict in a few more places 2014-02-18 23:09:12 +01:00
Jonas Borgström 7b31f23722 Replace verify command with "extract --dry-run"
closes #25
2014-02-18 21:33:06 +01:00
Jonas Borgström 30daa23e42 Minor cleanup 2014-02-18 21:16:36 +01:00
Jonas Borgström 7bcb0f97d6 Slightly lower the number of file descriptors kept open
This is needed to support old version of sftp-server that only
support 100 open file descriptors. Closes #40.
2014-02-17 21:16:04 +01:00
Jonas Borgström 893120e609 attic check performance improvement and minor progress reporting changes 2014-02-17 18:25:25 +01:00
Jonas Borgström c394a31d62 Improved archive metadata deduplication. 2014-02-16 23:36:48 +01:00
Jonas Borgström d44fa850cd Fix exception during "attic create" with repeated files
Closes #39
2014-02-16 22:37:12 +01:00
Jonas Borgström 32e773c15d Implemented archive metadata checking and repair 2014-02-16 22:21:39 +01:00
Jonas Borgström 0d248192e5 Implemented Repository.list() 2014-02-10 21:55:22 +01:00
Jonas Borgström bbf490bf8b More repository check/repair improvements 2014-02-10 21:32:18 +01:00
Jonas Borgström 22548841a2 Merge pull request #38 from jdchristensen/quickstart
Quickstart and --weekly doc fixes
2014-02-10 12:50:57 +01:00
Dan Christensen a79acdd10c Correct --weekly help to say weekly, not daily. 2014-02-09 16:15:49 -05:00
Jonas Borgström ab1cf32071 Implemented hashindex.iteritems(marker=X) 2014-02-09 22:05:33 +01:00
Jonas Borgström 1e4fd4e18a PyBuffer_FromMemory should be a static function 2014-02-09 21:25:05 +01:00
Jonas Borgström 66fe7e5377 Removed redundant is_a_terminal function and minor check fixes. 2014-02-09 20:35:00 +01:00
Jonas Borgström 14dd0414eb Merge remote-tracking branch 'jdchristensen/help' 2014-02-09 20:30:57 +01:00
Jonas Borgström 1809ea2f3e More attic check --repair improvements 2014-02-09 16:05:04 +01:00
Jonas Borgström 33b58eac82 Initial check --repair support 2014-02-09 16:05:04 +01:00
Jonas Borgström 5898b3b935 Merge remote-tracking branch 'jdchristensen/prune-within'
Conflicts:
	attic/archiver.py
2014-02-09 16:04:32 +01:00
Dan Christensen b5483b79a4 Add a --within option to the prune command. 2014-02-08 15:37:27 -05:00
Dan Christensen 9d19550c1a Add support for --exclude-from. 2014-02-08 12:44:48 -05:00
Dan Christensen 020c4f802f return self.exit_code from do_help 2014-02-08 11:51:44 -05:00
Dan Christensen 989a86463c Escape the backslash in the help text. 2014-02-08 11:45:36 -05:00
Dan Christensen c0cb26613d On help on patterns. 2014-02-08 09:44:31 -05:00
Jonas Borgström 2252616f9e Repository code cleanup 2014-02-08 13:32:47 +01:00
Dan Christensen 4da8466b3b Fix a doc typo. 2014-02-07 18:10:52 -05:00
Dan Christensen b76a40c987 Simplify the ExcludePattern logic further, and explain. 2014-02-07 18:04:21 -05:00
Dan Christensen 8cc74be62e Simplify IncludePattern and add a few more tests to PatternTestCase. 2014-02-07 16:27:18 -05:00
Dan Christensen db9632532c Add one more test to PatternTestCase. 2014-02-07 16:04:06 -05:00
Dan Christensen 9c5e3d4ddc Fix minor bugs in IncludePattern and ExcludePattern, and speed up
ExcludePattern by storing a compiled regular expression.
2014-02-07 15:54:24 -05:00
Dan Christensen 59c8008436 Add more tests to PatternTestCase. The first two fail with the
current code.
2014-02-07 15:51:27 -05:00
Jonas Borgström 4271ffa25f Only cleanup partial transactions if an existing transaction is found 2014-02-05 22:49:12 +01:00
Jonas Borgström e4a28f288f Remove flush=True argument from print() since it is Python3.3+ 2014-02-05 20:30:53 +01:00
Jonas Borgström 5abd254a11 Fixed repository check progress reporting. 2014-02-05 20:18:31 +01:00
Jonas Borgström 7a7cc2bc88 Removed some dead code. 2014-02-05 00:07:03 +01:00
Jonas Borgström faedaf8160 Basic repository consistency check functionality.
Still no archive metadata validation or repair functionality.
2014-02-05 00:04:30 +01:00
Dan Christensen 6b639e4c22 Update prune docs and a few other things in usage.rst. 2014-02-03 20:11:47 -05:00
Dan Christensen d3ff6ac25d Clean-up helpers.prune_split 2014-02-02 23:45:53 -05:00
Dan Christensen 13863e64f8 Correct a theoretical bug in helpers.prune_split 2014-02-02 23:44:04 -05:00
Dan Christensen c45aedf480 Add tests for helpers.prune_split, one of which fails. 2014-02-02 23:42:10 -05:00
Jonas Borgström 50cabd53b1 Fix deadlock when extracting 0 sized files from remote repositories 2014-01-30 22:16:21 +01:00
Jonas Borgström 2068e7cf34 Properly apply --exclude wildcards to the full path.
Closes #5
2014-01-30 20:33:29 +01:00
Jonas Borgström 0a7c51ef7c Improve endianness code portability 2014-01-29 21:34:21 +01:00
Jonas Borgström 5bde1c8987 Make source code endianness agnostic
Closes #1
2014-01-28 22:40:07 +01:00
Jonas Borgström d023f2aa64 Remove peek concept and fix fuse issue 2014-01-23 22:13:08 +01:00
Jonas Borgström c26d63b5fe Removed some unused imports 2014-01-23 21:46:53 +01:00
Jonas Borgström 1ee159a939 Improved remote repository exception handling. 2014-01-23 21:43:20 +01:00
Jonas Borgström b41914aa8f Verify archive metadata version before syncing cache. 2014-01-22 22:10:36 +01:00
Jonas Borgström bd5b72a646 Improved remote repository performance and reliability 2014-01-22 20:59:12 +01:00
Jonas Borgström c973040bad Improve libcrypto detection
Closes #23.
2014-01-09 22:10:21 +01:00
Jonas Borgström 3a064aba56 Fix sorting of segment names to ignore NFS left over files.
Closes #17.
2014-01-03 14:47:44 +01:00
Petros Moisiadis c93363946b Consider segment incomplete if segment file is empty or too small
This fixes an IOError that could be raised when trying to seek() to
a negative file offset, because (for any reason) a segment file was
empty (or too small).
2013-12-30 17:21:47 +02:00
Petros Moisiadis 4e68f98dde Added missing repository path/url when raising DoesNotExist and AlreadyExists exceptions 2013-12-17 16:53:10 +02:00
Jonas Borgström 87749e413e Fix timezone issue on python 3.2 2013-12-15 21:36:20 +01:00
Jonas Borgström f3e9fa28bc Fix incorrect display of time
Closes #13
2013-12-15 21:25:32 +01:00
Jonas Borgström 7bacfa2d6e Improved error handling/reporting.
Closes #12.
2013-12-15 20:35:29 +01:00
Jonas Borgström a56652f5c1 Use lockf() instead of flock() when locking repository/cache
This is a reworked version of Petros Moisiadis' original pull request
since some extra effort is needed to support access to read-only
repositories.
2013-12-14 20:44:20 +01:00
Brian Johnson 29d184dfcb Let ssh figure out port/user if not specified so we don't override .ssh/config
Modified by Jonas Borgström:
- Added CHANGES entry
- Fixed broken unit test
2013-11-27 22:19:21 +01:00
Jonas Borgström a579b7917b Remove unused imports 2013-10-18 21:00:20 +02:00
Jonas Borgström b5b0b7b322 Add two additional tests. 2013-10-10 22:24:06 +02:00