mirror
/
borg
mirror of https://github.com/borgbackup/borg.git
1
0
Fork 0
Code Issues Releases Wiki Activity
582 Commits 9 Branches 137 Tags 57 MiB
Commit Graph

87 Commits

Author SHA1 Message Date
Thomas Waldmann ee80db4eb2 add commandline argument --sparse 
default is to not create sparse files.

if you give --sparse, it will create a hole in the sparse output file when a all-zero chunk is extracted.
 2015-04-17 22:28:40 +02:00
Thomas Waldmann a2bf2aea22 simple sparse file support, made chunk buffer size flexible 
Implemented sparse file support to remove this blocker for people backing up lots of
huge sparse files (like VM images). Attic could not support this use case yet as it would
have restored all files to their fully expanded size, possibly running out of disk space if
the total expanded size would be bigger than the available space.

Please note that this is a very simple implementation of sparse file support - at backup time,
it does not do anything special (it just reads all these zero bytes, chunks, compresses and
encrypts them as usual). At restore time, it detects chunks that are completely filled with zeros
and does a seek on the output file rather than a normal data write, so it creates a hole in
a sparse file. The chunk size for these all-zero chunks is currently 10MiB, so it'll create holes
of multiples of that size (depends also a bit on fs block size, alignment, previously written data).

Special cases like sparse files starting and/or ending with a hole are supported.

Please note that it will currently always create sparse files at restore time if it detects all-zero
chunks.

Also improved:
I needed a constant for the max. chunk size, so I introduced CHUNK_MAX (see also
existing CHUNK_MIN) for the maximum chunk size (which is the same as the chunk
buffer size).

Attic still always uses 10MiB chunk buffer size now, but it could be changed now more easily.
 2015-04-15 16:29:18 +02:00
Jonas Borgström 20026e58aa Fix incorrect behaviour with two character directory names. 
Independetely found and fixed by both Thomas Waldemann and
Cam Hutchison. Closes #265 and #268.
 2015-04-13 22:44:14 +02:00
Jonas Borgström 7198929bae cache: Warn user before accessing relocated repositories 
This also closes #225
 2015-04-13 22:42:32 +02:00
Jonas Borgström 78f9ad1fab Require approval before accessing previously unknown unencrypted repositories 
Closes #271
 2015-04-06 23:07:10 +02:00
Jonas Borgström 18641ae687 xattr: Fix issue with empty (0 bytes) xattr values. 
Closes #106
 2014-12-14 14:28:33 +01:00
Jonas Borgström 678fdd1d42 Fix "Number of files" output for attic info. 
Closes #124
 2014-10-27 22:00:56 +01:00
Jonas Borgström 5a69f58823 Introduce a separate exception for missing repository objects 2014-10-09 20:41:47 +02:00
Jonas Borgström 21e03af56b Fix minor location parser bug. 
Make sure the entire input is parsed.
 2014-08-08 23:34:27 +02:00
Jonas Borgström 9f64e39d9f Reuse chunker buffer between files. 2014-08-03 15:04:41 +02:00
Jonas Borgström 193fb1fcd5 Added support for stripping leading path segments 
closes #95
 2014-08-02 22:15:21 +02:00
Jonas Borgström 8f4e0f7506 acl: Added workaround for old Linux systems 
Really old Linux systems do not have extended_file_no_follow()
 2014-08-01 15:50:18 +02:00
Jonas Borgström 2f72b9f960 HashIndex: Switch to a non-mmap based implementation 
This eliminates unnecessary IO on low memory systems
 2014-07-10 15:32:29 +02:00
Jonas Borgström 95162ce1f8 buzhash: One more test case 
Test with more than 31 bytes to make sure our BARREL_SHIFT macro
works correctly.
 2014-06-05 15:11:46 +02:00
Jonas Borgström 64cd6632a1 Avoid deadlock problems 
Explicitly use write locks, instead of read locks (that are later
upgraded) unless we know we will never modify the repository
 2014-05-31 15:39:51 +02:00
Jonas Borgström 3ab53b776d Fix crash on extreme mtime timestamps (year 2400+) 
Closes #81
 2014-05-18 18:28:26 +02:00
Jonas Borgström dd5c0aa0e3 acl: make darwin acl code numeric_owner aware 2014-05-03 23:22:55 +02:00
Jonas Borgström 47c6141aab Experimental OS X ACL support (#66) 2014-05-03 15:10:11 +02:00
Jeremy Maitin-Shepard 1d5410a3cf Add --exclude-caches option, for excluding CACHEDIR.TAG directories 
See http://www.brynosaurus.com/cachedir/spec.html
 2014-04-30 14:57:21 -07:00
Jeremy Maitin-Shepard 1955484e1b testsuite/archiver.py: Fix misspelling regual -> regular 2014-04-30 14:43:32 -07:00
Jonas Borgström 1b779c896d Linux ACL support improvements 2014-04-29 22:29:26 +02:00
Jonas Borgström 4bcc1b8ff9 Fix Cython issue on Linux 2014-04-27 14:29:03 +02:00
Jonas Borgström 29f95bba68 Use a dedicated file for bsdflags tests 2014-04-13 21:52:49 +02:00
Jonas Borgström 0ad5253d84 Experimental Linux ACL support (#66) 2014-04-13 20:26:46 +02:00
Jonas Borgström 8bf3bb1ca3 Added support for backup and restore of bsdflags (OS X and FreeBSD) 
This improves our Backup-Bouncer score (#56)
 2014-04-08 21:52:26 +02:00
Jonas Borgström 4bbde41466 Fix bug where xattrs on symlinks were not correctly restored 2014-04-08 21:22:57 +02:00
Jonas Borgström 8e078b5f91 Initial (naive) full repository fuse mount support 2014-03-26 22:06:53 +01:00
Jonas Borgström a9fc62cc9a Added option to restrict remote repository access to specific path(s) 
With this option remote repository access can be restricted to a
specific path for a specific ssh key using the following line
in ~/.ssh/authorized_keys::

command="attic serve --restrict-to-path /data/clientA" ssh-rsa clientA's key
command="attic serve --restrict-to-path /data/clientB" ssh-rsa clientB's key

Closes #51.
 2014-03-24 21:28:59 +01:00
Jonas Borgström 0e39acffd3 crypto: convert attic.crypto into a cython module 
This will give a slight performance boost and make the openssl
detection/linking more robust.
 2014-03-10 22:50:38 +01:00
Jonas Borgström 90fe318809 Attempt to make the archive metadata resync more robust 2014-03-06 20:05:13 +01:00
Jonas Borgström 66a84c0c12 check: Fix a check --repair issue and added more tests 2014-03-06 12:05:22 +01:00
Jonas Borgström 55a26e553f check: replace --phase with --repository/archives-only 2014-03-04 21:56:37 +01:00
Jonas Borgström a68e48bc30 check: Make item stream resync a bit more robust 2014-03-04 21:15:52 +01:00
Jonas Borgström 7be0ad609d Make the manifest rebuild code more robust 
Try to make sure we've actually found msgpack data before feeding
it into msgpack.
 2014-03-01 15:00:21 +01:00
Jonas Borgström 3982c34e6c Improve attic check documentation 
Closes #24.
 2014-02-26 23:13:48 +01:00
Jonas Borgström 678fb2c845 testsuite: Fix test test_prune_repository 2014-02-26 20:27:08 +01:00
Jonas Borgström 9e8a944a2a check: archive metadata recovery improvements 2014-02-24 22:44:43 +01:00
Jonas Borgström 6425d16aa8 repository: Fix potential race condition 
If we crash between compact_segments() and write_index() and the
transaction deletes objects that are newer than the current index
might become undeleted.
 2014-02-21 20:20:17 +01:00
Jonas Borgström bd22bc8cb2 Automatically replay segments to rebuild missing repository index 2014-02-20 23:49:35 +01:00
Dan Christensen 5d2d3f1f93 Add --dry-run option to prune. 2014-02-19 21:33:05 -05:00
Jonas Borgström e458f859d3 Added command line deprecation functionality 
The following deprecations are currently handled:

- attic verify -> attic extract --dry-run
- attic prune --hourly|daily|... -> attic prune --keep-X
 2014-02-19 22:53:44 +01:00
Jonas Borgström d551b13a23 Use StableDict in a few more places 2014-02-18 23:09:12 +01:00
Jonas Borgström 7b31f23722 Replace verify command with "extract --dry-run" 
closes #25
 2014-02-18 21:33:06 +01:00
Jonas Borgström 893120e609 attic check performance improvement and minor progress reporting changes 2014-02-17 18:25:25 +01:00
Jonas Borgström c394a31d62 Improved archive metadata deduplication. 2014-02-16 23:36:48 +01:00
Jonas Borgström d44fa850cd Fix exception during "attic create" with repeated files 
Closes #39
 2014-02-16 22:37:12 +01:00
Jonas Borgström 32e773c15d Implemented archive metadata checking and repair 2014-02-16 22:21:39 +01:00
Jonas Borgström 0d248192e5 Implemented Repository.list() 2014-02-10 21:55:22 +01:00
Jonas Borgström bbf490bf8b More repository check/repair improvements 2014-02-10 21:32:18 +01:00
Jonas Borgström ab1cf32071 Implemented hashindex.iteritems(marker=X) 2014-02-09 22:05:33 +01:00