Thomas Waldmann
6fa5bb4630
add tests for blake2b_128
2021-01-28 17:59:46 +01:00
Ryan Polley
83116e58f2
add examples for --paths-from-stdin, --paths-from-command, --paths-separator ( #5644 )
...
docs: add examples for --paths-from-stdin, --paths-from-command, --paths-separator
2021-01-27 00:15:14 +01:00
SanskritFritz
4346c4e85b
Tab completion support for additional archives for 'borg delete'
...
Bash and Fish tab completions now too support more than just one
archive provided for 'borg delete'.
2021-01-26 22:43:05 +01:00
TW
b20c7229e6
Merge pull request #5642 from rpolley/fix-grammar-in-docs-5632
...
fix grammar in faq
2021-01-23 01:44:28 +01:00
TW
2d8c9f1acf
Merge pull request #5643 from rpolley/update-docs-on-dev-env-setup-#5504
...
Update docs for dev environment installation instructions
2021-01-23 01:40:26 +01:00
Ryan Polley
1c5b57f1b3
recomend running tests only on installed versions for setup
2021-01-22 16:21:13 -06:00
Ryan Polley
974fd9eecb
add link back to git-installation
2021-01-22 16:12:52 -06:00
Ryan Polley
921cac0177
fix grammar in faq
2021-01-22 14:35:45 -06:00
TW
699256edbd
Merge pull request #5620 from ThomasWaldmann/sparse-file-integr2
...
Sparse file support (integration)
2021-01-17 17:45:53 +01:00
Thomas Waldmann
6dc334422e
fixup: improve comment about assumptions in the item metadata stream chunker
2021-01-15 21:51:15 +01:00
Thomas Waldmann
2391d160a8
add all-zero detection to buzhash chunk data processing
2021-01-15 21:27:29 +01:00
Thomas Waldmann
2d76365214
cosmetic: directly set allocation instead going via is_zero
2021-01-15 21:10:07 +01:00
Thomas Waldmann
8162e2e67b
cached_hash is only used in archive, move it there
2021-01-14 20:50:12 +01:00
Thomas Waldmann
e41dc6e96f
use zeros for benchmarks
2021-01-14 20:19:10 +01:00
Thomas Waldmann
be257728ca
move zeros to constants module
2021-01-14 20:02:18 +01:00
Thomas Waldmann
3b9798cffc
remove max_chunk_size (unused)
2021-01-14 19:56:39 +01:00
TW
4041bdf169
Merge pull request #5606 from ThomasWaldmann/fix-5603-master
...
do not recurse into duplicate roots, fixes #5603 (master)
2021-01-11 16:51:49 +01:00
TW
abb87ce68b
Merge pull request #5623 from infectormp/patch-4
...
Add badge with current status of package
2021-01-10 13:54:36 +01:00
infectormp
ab97bc20ed
Add badge with current status of package
2021-01-09 17:58:47 +03:00
Thomas Waldmann
4e3be1db5e
reuse zeros also in fixed-size chunker for all-zero chunk detection
...
also: zeros.startswith() is faster
2021-01-08 23:39:53 +01:00
Thomas Waldmann
ef19d937ed
use cached_hash also to generate all-zero replacement chunks
...
at least for major amounts of fixed-size replacement hashes,
this will be much faster. also less memory management overhead.
2021-01-08 23:39:53 +01:00
Thomas Waldmann
f3088a9893
rename chunk_to_id_data to cached_hash
2021-01-08 23:39:53 +01:00
Thomas Waldmann
92f221075a
refactor recreate to use chunk_to_id_data
2021-01-08 23:39:53 +01:00
Thomas Waldmann
b3659e0b8c
reuse chunker.zeros for sparse extraction
2021-01-08 23:39:53 +01:00
Thomas Waldmann
9fd284ce1a
refactor new zero chunk handling to be reusable
2021-01-08 23:39:53 +01:00
Thomas Waldmann
6d0f9a52eb
detect all-zero chunks, avoid hashing them
...
comparing zeros is quicker than hashing them.
the comparison should fail quickly inside non-zero data.
2021-01-08 17:40:06 +01:00
Thomas Waldmann
52bd55b29a
integrate Chunk type, avoid hashing holes
2021-01-08 17:39:51 +01:00
Thomas Waldmann
7319f85b54
adapt the existing chunker tests
2021-01-08 17:33:25 +01:00
Thomas Waldmann
8c299696aa
Chunker: yield Chunk namedtuple instead of bytes/memoryview
2021-01-08 01:10:44 +01:00
TW
37d4aee122
Merge pull request #5597 from Gelma/typos_dec
...
Fix typos
2021-01-07 19:21:03 +01:00
Thomas Waldmann
7a3a49e99b
reverted changes to 3rd party code
...
all algorithms/* stuff needs to be fixed upstream.
we just copy the files from there now and then.
https://github.com/lz4/lz4
https://github.com/facebook/zstd
https://github.com/Cyan4973/xxHash
2021-01-07 18:18:15 +01:00
Thomas Waldmann
0984e00d31
revert changes to autogenerated files
...
this stuff will be auto-generated, regularly done at release time.
2021-01-07 18:08:49 +01:00
Andrea Gelmini
72e7c46fa7
Fix typos
2021-01-07 17:54:33 +01:00
TW
41d282bc14
Merge pull request #5615 from jedie/patch-1
...
Change docs/install instruction: "python-virtualenv" -> "python3-virtualenv"
2021-01-07 17:13:31 +01:00
Jens Diemer
5a585c5c2e
"python-virtualenv" -> "python3-virtualenv"
2021-01-07 10:31:26 +01:00
TW
268eb2e598
Merge pull request #5610 from ThomasWaldmann/update-changes-master
...
update CHANGES
2021-01-04 19:50:56 +01:00
Thomas Waldmann
4394fa45ba
update CHANGES
2021-01-03 21:40:31 +01:00
TW
9088b1e85a
Merge pull request #5608 from ThomasWaldmann/improve-borg-check-logging-master
...
check: debug log segment filename
2021-01-03 19:04:40 +01:00
Thomas Waldmann
f2cb17d66c
check: debug log segment filename
2021-01-03 18:23:52 +01:00
Thomas Waldmann
73c04398f3
move requires_hardlinks upwards
2021-01-03 17:55:45 +01:00
axapaxa
b291b91962
Add remote upload buffer (--remote-buffer) ( #5574 )
...
add remote upload buffer (--remote-buffer)
- added new option --remote-buffer
- allow to_send to grow to selected size
- don't grow if wait is specified
- fill pipe on any command (including 'async_response')
- add new option to docs
- create EfficientBytesQueue to prevent recreation of buffer each time we send something
- add tests for EfficientBytesQueue
2021-01-03 17:37:16 +01:00
Thomas Waldmann
806dc5084d
do not recurse into duplicate roots, fixes #5603
2021-01-02 23:01:31 +01:00
Thomas Waldmann
257791274f
add a test whether a duplicate root is skipped, see #5603
2021-01-02 23:00:52 +01:00
Thomas Waldmann
58c0a0186f
add a test for hardlink extraction issue, see #5603
2021-01-02 23:00:38 +01:00
TW
2851a84003
Merge pull request #5561 from ThomasWaldmann/sparse-file-support
...
sparse map / file map support for fixed size chunker
2020-12-28 20:21:50 +01:00
TW
27651f6f28
Merge pull request #5600 from ThomasWaldmann/fix-mp
...
fix cythonize crash on macOS/py39, fixes #5599
2020-12-28 20:20:33 +01:00
Thomas Waldmann
d2fe303967
fix cythonize crash on macOS/py39, fixes #5599
2020-12-28 19:56:08 +01:00
Thomas Waldmann
37a7436ff9
detect sparse support by fs
2020-12-28 19:53:52 +01:00
Thomas Waldmann
c0c0da9c76
skip sparse tests if has_seek_hole is False
...
also: do the os.SEEK_(HOLE|DATA) check only once
2020-12-27 22:06:08 +01:00
Thomas Waldmann
b8bb0494f6
create --sparse, file map support for the "fixed" chunker, see #14
...
a file map can be:
- created internally inside chunkify by calling sparsemap, which uses
SEEK_DATA / SEEK_HOLE to determine data and hole ranges inside a
seekable sparse file.
Usage: borg create --sparse --chunker-params=fixed,BLOCKSIZE ...
BLOCKSIZE is the chunker blocksize here, not the filesystem blocksize!
- made by some other means and given to the chunkify function.
this is not used yet, but in future this could be used to only read
the changed parts and seek over the (known) unchanged parts of a file.
sparsemap: the generate range sizes are multiples of the fs block size.
the tests assume 4kiB fs block size.
2020-12-27 22:06:08 +01:00