Commit Graph

2130 Commits

Author SHA1 Message Date
TW 91f92d65df Merge pull request #1173 from ThomasWaldmann/fix-sparse-test
fix sparse file test, fixes #1170
2016-06-19 22:48:33 +02:00
TW 195a6b9093 Merge pull request #1186 from ThomasWaldmann/ignore-F405
flake8: ignore F405, fixes #1185
2016-06-19 15:09:26 +02:00
Thomas Waldmann 278cf0b9ad helpers: fix "undefined name 'e'" in exception handler 2016-06-19 14:55:57 +02:00
Thomas Waldmann 745007bd12 flake8: ignore F405, fixes #1185
We know that star imports have their issues and we only have one of them,
for good reasons. Thus, we switch off that new F405 test/message.
2016-06-19 14:40:11 +02:00
Thomas Waldmann 9be32e97b1 fix sparse file test, fixes #1170
found out that xfs is doing stuff behind the scenes: it is pre-allocating 16MB
to prevent fragmentation (in my case, value depends on misc factors).

fixed the test so it just checks that the extracted sparse file uses less (not
necessary much less) space than a non-sparse file would use.

another problem showed up when i tried to verify the holes in the sparse file
via SEEK_HOLE, SEEK_DATA:
after the few bytes of real data in the file, there was another 16MB
preallocated space.
So I ended up checking just the hole at the start of the file.

tested on: ext4, xfs, zfs, btrfs
2016-06-15 12:22:29 +02:00
enkore 22e14a0a75 Merge pull request #1160 from ThomasWaldmann/fix-1159
Fix 1159
2016-06-12 18:24:15 +02:00
Thomas Waldmann 69c3b5e196 rebuild_manifest: refactor archive metadata dict validation
this was already done in a similar way for item metadata dict validation.

also: check for some more required keys - the old code would crash if 'name' or 'time' key were missing.
2016-06-12 16:06:16 +02:00
Thomas Waldmann 03f6282eab make rebuild_manifest more future-proof 2016-06-12 15:31:31 +02:00
Thomas Waldmann 866417853d rename valid_msgpacked_item to valid_msgpacked_dict
the code is generic, it can also be used for other msgpacked dictionaries.
2016-06-12 15:31:31 +02:00
enkore 90d621ce35 Merge pull request #1149 from ThomasWaldmann/validkeys-in-manifest
item_keys in manifest, validate item keys
2016-06-12 14:44:27 +02:00
Thomas Waldmann a7b5165149 better validation of item metadata dicts, fixes #1130
the previous check only checked that we got a dict, but did not validate the dict keys.
this triggered issues with e.g. (invalid) integer keys.

now it validates the keys:
- some required keys must be present
- the set of keys is a subset of all valid keys
2016-06-12 00:11:31 +02:00
Thomas Waldmann 78121a8d04 store item_keys into manifest, fixes #1147
we need a list of valid item metadata keys. using a list stored in the repo manifest
is more future-proof than the hardcoded ITEM_KEYS in the source code.

keys that are in union(item_keys_from_repo, item_keys_from_source) are considered valid.
2016-06-12 00:11:10 +02:00
TW 4c98b91e12 Merge pull request #1153 from witten/patch-2
Update resources.rst to rename atticmatic to borgmatic
2016-06-11 18:28:00 +02:00
Dan Helfman 7642632359 Update resources.rst to rename atticmatic to borgmatic
atticmatic wrapper script has been renamed to borgmatic!
2016-06-10 19:27:19 -07:00
TW d9b77d57c8 Merge pull request #1148 from ThomasWaldmann/fix-1135
fix resync and msgpacked item qualifier, fixes #1135
2016-06-09 22:08:55 +02:00
Thomas Waldmann 918e0b2a52 fix resync and msgpacked item qualifier, fixes #1135
when trying to resync and skip invalid data, borg tries to qualify a byte sequence as
valid-looking msgpacked item metadata dict (or not) before even invoking msgpack's unpack.

besides previously hard to understand code, there were 2 issues:

- a missing check for map16 - this type is what msgpack uses if the dict has more than
15 items (could happen in future, not for 1.0.x).

- missing checks for str8/16/32 - str16 is what msgpack uses if the bytestring has more than 31 bytes
(borg does not have that long key names, thus this wasn't causing any harm)

this misqualification (valid data considered invalid) could lead to a wrong resync, skipping valid items.

added more comments and tests.
2016-06-09 22:08:06 +02:00
TW b5b4a72fa6 Merge pull request #1137 from ThomasWaldmann/no-manifest
better error handling for missing repo manifest, fixes #1043
2016-06-09 21:41:02 +02:00
Thomas Waldmann e10d543ef4 delete a repo without manifest 2016-06-09 20:43:13 +02:00
Thomas Waldmann 6a70d9968c make borg check work for empty repo 2016-06-09 20:43:13 +02:00
Thomas Waldmann 7501c3b530 better error handling for missing repo manifest, fixes #1043
can happen for not correctly initialized repos or corrupted repos.

here: borg list failing more pretty
2016-06-09 20:43:13 +02:00
TW a8c4c9ffab Merge pull request #1144 from ThomasWaldmann/fix-prune-docs
fix / clarify prune help, fixes #1143
2016-06-06 16:12:46 +02:00
Thomas Waldmann fb89152392 fix / clarify prune help, fixes #1143 2016-06-06 16:11:22 +02:00
TW ee5e89b575 Merge pull request #1136 from ThomasWaldmann/env-security-cleanup
remove passphrase from subprocess environment, fixes #1105
2016-06-05 13:29:52 +02:00
Thomas Waldmann ca35d9f255 remove passphrase from subprocess environment, fixes #1105 2016-06-05 00:25:30 +02:00
TW 53937a2b60 Merge pull request #1131 from ThomasWaldmann/freebsd-llfuse-requirements
llfuse: update version requirement for freebsd
2016-06-04 17:18:20 +02:00
Thomas Waldmann 8e7ed8ebea llfuse: update version requirement for freebsd
as the requirement string is a AND operation, I used the rough "<2.0" requirement
and then excluded the non-working versions.
2016-06-04 15:53:30 +02:00
Thomas Waldmann 21a30269c2 freebsd: use a older llfuse release that builds
llfuse >0.41.1 <=1.0.0 does not build on freebsd.
the issue is already fixed, but no new release yet.
2016-05-20 21:50:26 +02:00
Thomas Waldmann 5be71b506e update CHANGES 2016-05-20 21:49:01 +02:00
Thomas Waldmann 8d8374d23c ran build_api + build_usage 2016-05-20 14:34:48 +02:00
TW 938628f4e1 Merge pull request #1062 from ThomasWaldmann/update-1.0-docs
update CHANGES
2016-05-20 14:06:27 +02:00
Thomas Waldmann 1e061a2fa4 update CHANGES 2016-05-20 14:05:05 +02:00
TW f974770db5 Merge pull request #1066 from enkore/issue/1063
RepositoryCache: don't cache large objects
2016-05-20 13:55:12 +02:00
Marian Beermann f7f95ae731
RepositoryCache: don't cache large objects
avoids excessive cache repository disk usage
2016-05-20 02:01:17 +02:00
enkore d043794757 Merge pull request #1052 from enkore/fix/commitcheck
Repository: fix commit tags being seen in data
2016-05-19 17:12:52 +02:00
Marian Beermann 714e93bf64
Repository: fix commit tags being seen in data 2016-05-19 01:12:06 +02:00
Thomas Waldmann 596a276a26 setup.py: add check for platform_darwin.c
this was just forgotten...
2016-05-18 18:07:10 +02:00
TW 5c6cfc6329 Merge pull request #1058 from enkore/issue/1056
When probing key files, do binary reads
2016-05-18 17:56:26 +02:00
Marian Beermann 24cd303ca9
When probing key files, do binary reads 2016-05-18 17:37:48 +02:00
TW eaf0f62022 Merge pull request #1049 from enkore/feature/sigterm
Handle SIGTERM
2016-05-17 19:28:14 +02:00
Marian Beermann 6cf61024d9
Handle SIGTERM and make a clean exit 2016-05-17 18:57:55 +02:00
TW f9638645d1 Merge pull request #996 from ThomasWaldmann/update-1.0-docs
update CHANGES in 1.0-maint
2016-05-05 13:51:59 +02:00
TW 6ae1999ff3 Merge pull request #1010 from ThomasWaldmann/doc-tags
borg create help: document format tags, fixes #894
2016-05-05 13:49:21 +02:00
Thomas Waldmann 7457c0f5da update CHANGES in 1.0-maint 2016-05-05 13:43:40 +02:00
Thomas Waldmann 9e09786b33 borg create help: document format tags, fixes #894
docs: rephrased, more useful examples
2016-05-05 13:38:08 +02:00
TW 21f753816d Merge pull request #1000 from ThomasWaldmann/prune-ignore-checkpoints
borg prune: ignore checkpoints, fixes #997
2016-05-02 01:26:06 +02:00
Thomas Waldmann b743fd09ab borg prune: ignore checkpoints, fixes #997
also:
- add a test for this
- add some words to borg create help about the archive name
2016-05-02 01:12:15 +02:00
Thomas Waldmann 700e715b23 add missing docs for borg break-lock, fixes #992 2016-04-30 00:32:25 +02:00
TW 7718d73133 Merge pull request #995 from ThomasWaldmann/update-site-urls
update readthedocs urls, fixes #991
2016-04-30 00:15:01 +02:00
Thomas Waldmann 4b73ebc613 update readthedocs urls, fixes #991 2016-04-29 03:19:27 +02:00
TW 81a1952c9a Merge pull request #978 from ThomasWaldmann/fix-osx
vagrant: OS X update osxfuse / fix lzma, fixes #933
2016-04-24 02:35:34 +02:00