Commit Graph

51 Commits

Author SHA1 Message Date
Thomas Waldmann 118fb1a55c PR #226 - Merge branch 'delete_repo' into merge 2015-04-15 19:05:58 +02:00
Thomas Waldmann a6f8d51b37 updated PR #279 - Merge branch 'fadvise' into merge 2015-04-11 17:08:17 +02:00
Thomas Waldmann 57071ce6fd repo writes: call posix_fadvise between fsync and close
less calls to posix_fadvise (which seem to force a write-cache sync-to-disk and
wait for that to complete) - if we call it after we synced anyway, we don't lose time.

also: fixed a bug in the os.fsync call, it needs the fileno.
2015-04-11 17:04:10 +02:00
Thomas Waldmann 3653acd259 PR #279 - Merge branch 'fadvise' into merge 2015-04-11 02:51:33 +02:00
Thomas Waldmann 874f5c491b use posix_fadvise for repo writes also 2015-04-11 02:40:22 +02:00
Thomas Waldmann dd8ff50a5c PR #245 - Merge branch 'fix_exceptions' into merge
Conflicts:
	attic/archive.py
2015-03-20 02:34:10 +01:00
Thomas Waldmann 2da70c7d7a fix some exception handlers
don't catch "Exception" when OSError was meant (otherwise e.errno is not there anyway)
don't use bare "except:" if one can avoid (copied code fragment from similar handler)
2015-03-20 02:31:39 +01:00
Thomas Waldmann b7cefacb69 Merge branch 'python_cleanup' into merge
Conflicts:
	attic/archiver.py
	attic/remote.py
2015-03-18 00:23:41 +01:00
Thomas Waldmann e9aacad9a3 remove unused imports 2015-03-17 23:54:05 +01:00
Thomas Waldmann 4ef6125f45 cleanup pep8 issues found by pycharm 2015-03-17 23:47:21 +01:00
Thomas Waldmann 3a33d57c7d triple-double-quotes for docstrings 2015-03-17 23:23:56 +01:00
Thomas Waldmann fec5572836 no Class(object) in py3 2015-03-17 23:03:36 +01:00
Thomas Waldmann 6e6819e626 attic delete: add repository + local cache deletion 2015-03-09 16:02:06 +01:00
Thomas Waldmann db7ce095d2 fix typos / spelling 2015-02-28 03:43:08 +01:00
Thomas Waldmann 8ffd4c4926 convert docstrings to triple-double-quoted 2015-02-28 03:24:30 +01:00
Thomas Waldmann a8ce0b8b8a remove unused imports 2015-02-28 03:09:01 +01:00
Thomas Waldmann 74768511e1 do os.fsync like recommended in the python docs (gets a fileno, not file object, also do a flush first) 2015-02-28 02:45:21 +01:00
Jonas Borgström 547fa0a590 Speed up initial repository access
Use os.listdir() instead of os.walk() to avoid unnecessary
stat() calls.

Closes #119
2014-10-10 21:01:55 +02:00
Jonas Borgström 5a69f58823 Introduce a separate exception for missing repository objects 2014-10-09 20:41:47 +02:00
Jonas Borgström 2f72b9f960 HashIndex: Switch to a non-mmap based implementation
This eliminates unnecessary IO on low memory systems
2014-07-10 15:32:29 +02:00
Jonas Borgström 64cd6632a1 Avoid deadlock problems
Explicitly use write locks, instead of read locks (that are later
upgraded) unless we know we will never modify the repository
2014-05-31 15:39:51 +02:00
Jonas Borgström 66a84c0c12 check: Fix a check --repair issue and added more tests 2014-03-06 12:05:22 +01:00
Jonas Borgström 3193af5482 Remove explicit flush calls since line_buffering is enabled. 2014-03-05 19:58:29 +01:00
Jonas Borgström 477e1a39dd check: Make sure all non-critical output is sent to stdout 2014-03-04 21:21:58 +01:00
Jonas Borgström 3982c34e6c Improve attic check documentation
Closes #24.
2014-02-26 23:13:48 +01:00
Jonas Borgström c1e1d6e9dd repository: Fixed a potential infinite loop in recover_segment 2014-02-24 12:59:38 +01:00
Jonas Borgström 6425d16aa8 repository: Fix potential race condition
If we crash between compact_segments() and write_index() and the
transaction deletes objects that are newer than the current index
might become undeleted.
2014-02-21 20:20:17 +01:00
Jonas Borgström bd22bc8cb2 Automatically replay segments to rebuild missing repository index 2014-02-20 23:49:35 +01:00
Jonas Borgström 30daa23e42 Minor cleanup 2014-02-18 21:16:36 +01:00
Jonas Borgström 7bcb0f97d6 Slightly lower the number of file descriptors kept open
This is needed to support old version of sftp-server that only
support 100 open file descriptors. Closes #40.
2014-02-17 21:16:04 +01:00
Jonas Borgström 893120e609 attic check performance improvement and minor progress reporting changes 2014-02-17 18:25:25 +01:00
Jonas Borgström 32e773c15d Implemented archive metadata checking and repair 2014-02-16 22:21:39 +01:00
Jonas Borgström 0d248192e5 Implemented Repository.list() 2014-02-10 21:55:22 +01:00
Jonas Borgström bbf490bf8b More repository check/repair improvements 2014-02-10 21:32:18 +01:00
Jonas Borgström 66fe7e5377 Removed redundant is_a_terminal function and minor check fixes. 2014-02-09 20:35:00 +01:00
Jonas Borgström 1809ea2f3e More attic check --repair improvements 2014-02-09 16:05:04 +01:00
Jonas Borgström 33b58eac82 Initial check --repair support 2014-02-09 16:05:04 +01:00
Jonas Borgström 2252616f9e Repository code cleanup 2014-02-08 13:32:47 +01:00
Jonas Borgström 4271ffa25f Only cleanup partial transactions if an existing transaction is found 2014-02-05 22:49:12 +01:00
Jonas Borgström e4a28f288f Remove flush=True argument from print() since it is Python3.3+ 2014-02-05 20:30:53 +01:00
Jonas Borgström 5abd254a11 Fixed repository check progress reporting. 2014-02-05 20:18:31 +01:00
Jonas Borgström 7a7cc2bc88 Removed some dead code. 2014-02-05 00:07:03 +01:00
Jonas Borgström faedaf8160 Basic repository consistency check functionality.
Still no archive metadata validation or repair functionality.
2014-02-05 00:04:30 +01:00
Jonas Borgström bd5b72a646 Improved remote repository performance and reliability 2014-01-22 20:59:12 +01:00
Jonas Borgström 3a064aba56 Fix sorting of segment names to ignore NFS left over files.
Closes #17.
2014-01-03 14:47:44 +01:00
Petros Moisiadis c93363946b Consider segment incomplete if segment file is empty or too small
This fixes an IOError that could be raised when trying to seek() to
a negative file offset, because (for any reason) a segment file was
empty (or too small).
2013-12-30 17:21:47 +02:00
Petros Moisiadis 4e68f98dde Added missing repository path/url when raising DoesNotExist and AlreadyExists exceptions 2013-12-17 16:53:10 +02:00
Jonas Borgström 7bacfa2d6e Improved error handling/reporting.
Closes #12.
2013-12-15 20:35:29 +01:00
Jonas Borgström a56652f5c1 Use lockf() instead of flock() when locking repository/cache
This is a reworked version of Petros Moisiadis' original pull request
since some extra effort is needed to support access to read-only
repositories.
2013-12-14 20:44:20 +01:00
Jonas Borgström 16d9e55f84 Support access of read only repositories 2013-08-11 22:19:38 +02:00