Commit Graph

3924 Commits

Author SHA1 Message Date
Marian Beermann d5515b6952 add msgid to progress output 2017-02-28 01:19:20 +01:00
Marian Beermann 6288c9f751 enhance JSON progress information
separate output types, extra information
2017-02-27 20:30:20 +01:00
enkore 7c9c4b61d7 Merge pull request #2157 from ThomasWaldmann/add-filesize
archived file items: add size metadata
2017-02-27 18:05:43 +01:00
enkore 0e3560b6cd Merge pull request #2219 from enkore/f/frontend-docs
document JSON
2017-02-27 17:03:37 +01:00
enkore b7e158ce89 Merge pull request #2227 from ThomasWaldmann/update-files-cache-inodes
files cache: update inode number, fixes #2226
2017-02-27 16:52:29 +01:00
Thomas Waldmann 0721cb1ede files cache: update inode number, fixes #2226 2017-02-27 15:30:55 +01:00
TW 1a67b82726 Merge pull request #2220 from ThomasWaldmann/compression-default-lz4
use lz4 compression by default, fixes #2179
2017-02-27 00:31:43 +01:00
Marian Beermann 332f7898bc update docs for create -C default change 2017-02-26 23:07:31 +01:00
enkore 46787a90a0 Merge pull request #2204 from ThomasWaldmann/test-delete-force
Test delete --force
2017-02-26 21:23:23 +01:00
enkore 56f99a7719 Merge pull request #2222 from kmq/master
Document relative path usage #1868
2017-02-26 20:57:21 +01:00
kmq 757921dbdc
Document relative path usage #1868 2017-02-26 20:33:15 +02:00
Thomas Waldmann 2ad5f903fe add test for borg delete --force 2017-02-26 18:49:10 +01:00
TW 9bc825a27a Merge pull request #2184 from ThomasWaldmann/zap
borg delete --force --force to delete severely corrupted archives, fixes #1975
2017-02-26 18:44:31 +01:00
Thomas Waldmann 92e5db0c2e use lz4 compression by default, fixes #2179
not for create_src_archive() though as this triggers a bug that has to get fixed outside this PR first.
2017-02-26 18:38:29 +01:00
enkore ecd13f11b0 Merge pull request #2218 from kmq/master
document snapshot usage #2178
2017-02-26 17:34:15 +01:00
Marian Beermann 9ed8e4b7a9 document JSON 2017-02-26 17:01:06 +01:00
enkore 5f802f3942 Merge pull request #2215 from enkore/f/logjson
Add --log-json option for structured logging output
2017-02-26 16:49:48 +01:00
Marian Beermann 70c11976bc Add --log-json option for structured logging output 2017-02-26 16:25:58 +01:00
kmq f7d28e76a0
document snapshot usage #2178 2017-02-26 13:47:26 +02:00
enkore 0f9b7a270d Merge pull request #2203 from enkore/f/jsono
JSON output for major commands
2017-02-25 23:05:35 +01:00
enkore 5c46136433 Merge pull request #2210 from enkore/f/rxtx
RemoteRepository: account rx/tx bytes
2017-02-25 22:41:27 +01:00
enkore e59d313666 Merge pull request #2206 from ThomasWaldmann/fix-location-regex-smb
Location: accept //servername/share/path
2017-02-25 16:11:44 +01:00
enkore 43bb43a271 Merge pull request #2192 from enkore/f/doctoc
Edited + and corrected some stuff in internals+security
2017-02-24 11:55:13 +01:00
Thomas Waldmann c8ec698d73 Location: accept //servername/share/path 2017-02-24 04:22:12 +01:00
Thomas Waldmann 4d81b186ec borg delete --force --force to delete severely corrupted archives, fixes #1975 2017-02-24 01:28:42 +01:00
Abogical e487b8404c Add testsuite to test handling of too big xattr 2017-02-23 23:42:56 +02:00
Abogical 4c9bc96fb7 Print a warning for too big extended attributes 2017-02-23 23:42:56 +02:00
Thomas Waldmann 7da0a9c982 borg extract: check file size consistency 2017-02-23 21:46:15 +01:00
Thomas Waldmann adc4da280d borg check: check file size consistency 2017-02-23 21:46:15 +01:00
Marian Beermann 4f1db82f6d info <archive>: use Archive.info() for both JSON and human display 2017-02-23 21:39:56 +01:00
Thomas Waldmann 50068c596d rename Item.file_size -> get_size
file_size is misleading here because one thinks of on-disk file size,
but for compressed=True, there is no such on-disk file.
2017-02-23 21:27:05 +01:00
Thomas Waldmann 97bb1b7d9a deduplicate / refactor item (c)size code 2017-02-23 21:27:05 +01:00
Thomas Waldmann 0021052dbd reduce code duplication 2017-02-23 21:24:37 +01:00
Thomas Waldmann ae6742fb34 fuse: use precomputed size from Item 2017-02-23 21:24:37 +01:00
Thomas Waldmann fe8e14cb2c fuse: get rid of chunk accounting
the chunk accounting code tried to reflect repo space usage via the st_blocks of the files.
so, a specific chunk that was shared between multiple files [inodes] was only accounted for one specific file.
thus, the overall "du" of everything in the fuse mounted repo was maybe correctly reflecting the repo space usage,
but the decision which file has the chunk (the space) was kind of arbitrary and not really useful.

otoh, a simple fuse getattr() was rather expensive due to this as it needed to iterate over the chunks list
to compute the st_blocks value. also it needed quite some memory for the accounting.

thus, st_blocks is now just ceil(size / blocksize).

also: fixed bug that st_blocks was a floating point value previously.

also: preparing for further optimization of size computation (see next cs)
2017-02-23 21:24:37 +01:00
Thomas Waldmann a52b54dc3c archived file items: add size metadata
if an item has a chunk list, pre-compute the total size and store it into "size" metadata entry.

this speeds up access to item size (e.g. for regular files) and could also be used to verify the validity of the chunks list.

note about hardlinks: size is only stored for hardlink masters (only they have an own chunk list)
2017-02-23 21:24:37 +01:00
Marian Beermann f3c7e7cd36 RemoteRepository: account rx/tx bytes 2017-02-23 16:50:52 +01:00
Marian Beermann 8cdf192511 list: add "name" key for consistency with info cmd 2017-02-23 15:23:27 +01:00
Marian Beermann 56563a4392 move JSON generation and utilities to helpers 2017-02-23 15:23:27 +01:00
enkore 4862efe718 Merge pull request #2194 from jdchristensen/master
In api decorator, pass wait argument to RemoteRepository.call; fixes …
2017-02-23 14:50:33 +01:00
Marian Beermann 1f8c0929bf list: --json for archive contents listing 2017-02-23 14:28:30 +01:00
Marian Beermann 25781f53d4 list: --json for archive listing 2017-02-23 14:28:15 +01:00
Marian Beermann 6180f5055c info: --json for archives 2017-02-23 14:28:15 +01:00
Marian Beermann 2ab5d0f213 use custom JSON encoder for repr'ing Borg objects consistently 2017-02-23 14:27:52 +01:00
Marian Beermann 7cbade2f8c create: add --json option 2017-02-23 12:00:21 +01:00
Marian Beermann cc26bdf810 info: add --json option 2017-02-23 11:55:14 +01:00
enkore e3a8ccdf3b Merge pull request #2197 from enkore/f/info-limits
info: show utilization of maximum archive size
2017-02-23 09:18:04 +01:00
Marian Beermann b0e4f13fba set MAX_DATA_SIZE = 20971479 bytes in solid stone 2017-02-23 00:34:40 +01:00
Marian Beermann 69f7810658 info: show utilization of maximum archive size
See #1452

This is 100 % accurate.

Also increases maximum data size by ~41 bytes. Not 100 % side-effect free;
if you manage to exactly land in that area then older Borg would not read
it. OTOH it gives us a nice round number there.
2017-02-22 23:47:21 +01:00
Dan Christensen 20a5282a4c In api decorator, pass wait argument to RemoteRepository.call; fixes #2185 2017-02-22 16:26:08 -05:00