1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2024-12-26 09:47:58 +00:00
Commit graph

1220 commits

Author SHA1 Message Date
Thomas Waldmann
b3f5231bac BORG_REPO env var support
sets the default repository to use, e.g. like:

export BORG_REPO=/mnt/backup/repo
borg init
borg create ::archive
borg list
borg mount :: /mnt
fusermount -u /mnt
borg delete ::archive
2015-09-06 18:18:24 +02:00
Thomas Waldmann
a912c02757 detect inconsistency / corruption / hash collision, closes #170
added a check that compares the size of the new chunk with the stored size of the
already existing chunk in storage that has the same id_hash value.
raise an exception if there is a size mismatch.

this could happen if:

- the stored size is somehow incorrect (corruption or software bug)
- we found a hash collision for the id_hash (for sha256, this is very unlikely)
2015-09-06 01:10:43 +02:00
Thomas Waldmann
0b1035746e read special files as if they were regular files, update docs, closes #79
do not use the files cache for such special files
2015-09-06 00:29:46 +02:00
Thomas Waldmann
54ccbc5ae2 chunks index resync: do all in one pass
if we do not have a cached archive index: fetch and build and merge it
if we have one: merge it
2015-08-30 15:15:15 +02:00
Thomas Waldmann
22dd925986 chunks index archive: remove all tar and compression related stuff and just use separate files in a directory
the compression was quite cpu intensive and didn't work that great anyway.
now the disk space usage is a bit higher, but it is much faster and less hard on the cpu.

disk space needs grow linearly with the amount and size of the archives, this
is a problem esp. if one has many and/or big archives (but this problem existed
before also because compression was not as effective as I believed).

the tar archive always needed a complete rebuild (and thus: decompression
and recompression) because deleting outdated archive indexes was not
possible in the tar file.

now we just have a directory chunks.archive.d and keep archive index files
there for all archives we already know.
if an archive does not exist any more in the repo, we just delete its index file.
if an archive is unknown still, we fetch the infos and build a new index file.

when merging, we avoid growing the hash table from zero, but just start
with the first archive's index as basis for merging.
2015-08-30 03:03:48 +02:00
Thomas Waldmann
f7210c749f remove cpu intensive compression methods for the chunks.archive
also remove the comment about how good xz compresses - while that was true for smaller index files,
it seems to be less effective with bigger ones. maybe just an issue with compression dict size.
2015-08-29 23:42:28 +02:00
TW
17c4394896 Merge pull request #161 from RonnyPfannschmidt/setuptools-scm
replace versioneer with setuptools_scm
2015-08-29 16:46:41 +02:00
Thomas Waldmann
ab545ae45d CHANGES: release 0.25.0 2015-08-29 14:10:31 +02:00
Thomas Waldmann
2332ee433e remove docs/misc/create_compression.txt
outdated - it just showed different levels of zlib compression,
but not we additionally have "lzma", "lz4" and "none" compression.
the "usage" and "internals" docs give some hints about them, too.
2015-08-29 13:35:58 +02:00
Thomas Waldmann
31e97d568b remove x bits from repository.py 2015-08-29 12:52:18 +02:00
Thomas Waldmann
d2e5ce90f2 add FAQ entry about OS X commandline vs. HFS+ encoding issue, see #143.
a real fix for this is in development, but needs some time.
2015-08-29 05:41:41 +02:00
Thomas Waldmann
273c04329f update CHANGES 2015-08-29 05:02:05 +02:00
Thomas Waldmann
d779057b79 fix issue with negative "all archives" size, fixes #165
This fixes a infrequent problem when (refcount * chunksize) overflowed a int32_t.
chunksize is always <= 8MiB and usually rather ~64KiB (with default chunker params).
Thus, this happened only for high refcounts and/or unusually big chunks.
2015-08-29 04:46:13 +02:00
Thomas Waldmann
1a949eab3e docs: internals: improve structure and formatting 2015-08-29 04:23:57 +02:00
Thomas Waldmann
ddd0e2700b docs: improve faq and support section 2015-08-29 04:16:03 +02:00
Thomas Waldmann
c823554b6b docs: usage: improved formatting, cosmetic changes 2015-08-29 04:00:22 +02:00
Thomas Waldmann
e4dc482598 docs: quickstart: key modes, cosmetic changes 2015-08-29 03:39:53 +02:00
Thomas Waldmann
8a483511cb docs: theme: use a lighter green 2015-08-29 03:01:39 +02:00
Thomas Waldmann
cebb61d8dd docs: add pypi, wheel, dist package based install docs 2015-08-29 02:55:09 +02:00
Thomas Waldmann
79d7f49a44 docs: split install into system-specific preparations and generic instructions 2015-08-29 01:17:19 +02:00
Thomas Waldmann
ee58d4f074 docs: be more specific about Debian/Ubuntu release 2015-08-29 00:45:54 +02:00
TW
acf5ddc68d Merge pull request #166 from ThomasWaldmann/fix-restore-xattrs
Fix restore xattrs
2015-08-29 00:26:38 +02:00
Thomas Waldmann
9ebc53ad77 restore_xattrs: ignore if setxattr fails with EACCES, fixes #162
e.g.:
- setting any security.* key is expected to fail with EACCES if one is not root.
- issue #162 on our issue tracker: user was root, but due to some specific scenario
  involving docker and selinux, setting security.selinux key fails even when running as root

not sure if it is the best solution to silently ignore this, but some lines below this change
failure to do a chown is also silently ignored (happens e.g. when restoring a file not owned
by the current user as a non-root user).
2015-08-29 00:11:04 +02:00
Thomas Waldmann
ea8f3bd7e7 restore_xattrs: minor cleanup / simplification
if we use {} as default for item.get(), we do not need the "if" as iteration over an empty dict won't do anything.
also fixes too deep indentation the original code had.
2015-08-28 23:22:26 +02:00
Ronny Pfannschmidt
8b6ca0d912 propperly handle borg._version using setuptools_scm 2015-08-22 15:54:40 +02:00
Ronny Pfannschmidt
c7023ed12a replace versioneer with setuptools_scm 2015-08-22 14:58:06 +02:00
Thomas Waldmann
034e3477bd travis: use requirements file 2015-08-21 01:10:34 +02:00
Thomas Waldmann
fdc18eb0a0 support docs: write some words about BountySource 2015-08-21 00:25:24 +02:00
Thomas Waldmann
2402a2269e update docs about fakeroot 2015-08-20 23:39:40 +02:00
Thomas Waldmann
3d974d28b3 update CHANGES 2015-08-20 23:27:24 +02:00
Thomas Waldmann
1e11e24fc4 document locking system 2015-08-20 22:37:38 +02:00
Alan Jenkins
59a44296e4 chunker - cast from size_t to off_t can now be removed
Sorry, this should really have been part of the previous commit -
it's why I noticed a problem.
2015-08-20 17:48:59 +01:00
Thomas Waldmann
0a2bd8dad5 lock roster: catch file not found in remove() method and ignore it 2015-08-20 18:40:24 +02:00
Alan Jenkins
ce3e67cb96 chunker - fix 4GB files on 32-bit systems
From code inspection - effect not actually tested.
2015-08-20 17:23:50 +01:00
Alan Jenkins
7c6f3ece66 Initialize chunker fd to -1, so it's not equal to STDIN_FILENO (0) 2015-08-20 17:23:41 +01:00
Thomas Waldmann
d3d78f7ae3 call fadvise DONTNEED for the byterange we actually have read, fixes #158
avoid throwing away potential readahead data the OS might have read into the cache.
2015-08-20 05:33:51 +02:00
Per Guth
0926c2821f docs: installation: korora/fedora -> lz4-devel 2015-08-17 17:10:37 +02:00
Per Guth
8cf0ead693 docs: added favicon.ico 2015-08-17 12:58:49 +02:00
Thomas Waldmann
7f0fe83483 Merge branch 'master' of github.com:borgbackup/borg 2015-08-17 11:52:05 +02:00
Thomas Waldmann
93a89d97fa ChunkerParams: fix parameter order
the parser for the --chunker-params argument had a wrong parameter order.
fixed the order so it conforms to the help text and the docs.
also added some tests for it and a text for the ValueError exception.
2015-08-17 11:50:47 +02:00
Per Guth
1c7b5b23f4 minor change in copyright notice 2015-08-17 11:31:42 +02:00
Thomas Waldmann
b180158876 generalize hashindex code for any key length
currently, we only use sha256 hashes as key, so key length is always 32.
but instead of hardcoding 32 everywhere, using key_length is just better
readable and also more flexible for the future.
2015-08-16 14:51:15 +02:00
Thomas Waldmann
608c0935e0 borg list --short, remove requirement for fakeroot, xfail a test
borg list --short just spills out the list of files / dirs - better for some tests
and also useful on the commandline for interactive use.

the tests previously needed fakeroot because in the test setup it always
made calls to mknod and chown, which require (fake)root.
now, the tests adapt to whether it detects (fake)root or not - to run the
the tests completely, you still need fakeroot, but it won't fail all the archiver
tests just due to failing test setup.

also, a test not working correctly due to fakeroot was found:
it should detect whether a read-only repo is usable, but it failed to do that
because with (fake)root, there is no "read only" (at least not via taking away
 the w permission bits).
2015-08-15 20:52:14 +02:00
Thomas Waldmann
738ed5d91b 2 small archiver testsuite fixes
environment context manager: if a env var was not present before, it should not be present afterwards

teardown: cd out of the tmpdir before deleting it
2015-08-15 17:07:09 +02:00
Thomas Waldmann
e5b647fbd1 minor lrucache test fix 2015-08-15 16:15:10 +02:00
Thomas Waldmann
986b70c189 Merge branch 'lrucache' of https://github.com/sourcejedi/borg 2015-08-15 16:06:09 +02:00
TW
fffe509268 Merge pull request #129 from ThomasWaldmann/compression
compression flexibility, new none, lz4 and lzma compression
2015-08-15 16:03:34 +02:00
Thomas Waldmann
e1de3dce7b integrate compression branch changes into change history for 0.25 2015-08-15 15:49:11 +02:00
Thomas Waldmann
1d16e7a37c compression: update / refine docs 2015-08-15 15:45:15 +02:00
Thomas Waldmann
bf757738f7 Merge branch 'master' into compression 2015-08-14 23:24:04 +02:00