mirror
/
borg
mirror of https://github.com/borgbackup/borg.git
1
0
Fork 0
Code Issues Releases Wiki Activity
6,125 Commits 9 Branches 137 Tags 57 MiB
Commit Graph

6125 Commits

Author SHA1 Message Date
Ryan Polley 921cac0177 fix grammar in faq 2021-01-22 14:35:45 -06:00
TW 699256edbd
Merge pull request #5620 from ThomasWaldmann/sparse-file-integr2 
Sparse file support (integration)
 2021-01-17 17:45:53 +01:00
Thomas Waldmann 6dc334422e fixup: improve comment about assumptions in the item metadata stream chunker 2021-01-15 21:51:15 +01:00
Thomas Waldmann 2391d160a8 add all-zero detection to buzhash chunk data processing 2021-01-15 21:27:29 +01:00
Thomas Waldmann 2d76365214 cosmetic: directly set allocation instead going via is_zero 2021-01-15 21:10:07 +01:00
Thomas Waldmann 8162e2e67b cached_hash is only used in archive, move it there 2021-01-14 20:50:12 +01:00
Thomas Waldmann e41dc6e96f use zeros for benchmarks 2021-01-14 20:19:10 +01:00
Thomas Waldmann be257728ca move zeros to constants module 2021-01-14 20:02:18 +01:00
Thomas Waldmann 3b9798cffc remove max_chunk_size (unused) 2021-01-14 19:56:39 +01:00
TW 4041bdf169
Merge pull request #5606 from ThomasWaldmann/fix-5603-master 
do not recurse into duplicate roots, fixes #5603 (master)
 2021-01-11 16:51:49 +01:00
TW abb87ce68b
Merge pull request #5623 from infectormp/patch-4 
Add badge with current status of package
 2021-01-10 13:54:36 +01:00
infectormp ab97bc20ed
Add badge with current status of package 2021-01-09 17:58:47 +03:00
Thomas Waldmann 4e3be1db5e reuse zeros also in fixed-size chunker for all-zero chunk detection 
also: zeros.startswith() is faster
 2021-01-08 23:39:53 +01:00
Thomas Waldmann ef19d937ed use cached_hash also to generate all-zero replacement chunks 
at least for major amounts of fixed-size replacement hashes,
this will be much faster. also less memory management overhead.
 2021-01-08 23:39:53 +01:00
Thomas Waldmann f3088a9893 rename chunk_to_id_data to cached_hash 2021-01-08 23:39:53 +01:00
Thomas Waldmann 92f221075a refactor recreate to use chunk_to_id_data 2021-01-08 23:39:53 +01:00
Thomas Waldmann b3659e0b8c reuse chunker.zeros for sparse extraction 2021-01-08 23:39:53 +01:00
Thomas Waldmann 9fd284ce1a refactor new zero chunk handling to be reusable 2021-01-08 23:39:53 +01:00
Thomas Waldmann 6d0f9a52eb detect all-zero chunks, avoid hashing them 
comparing zeros is quicker than hashing them.
the comparison should fail quickly inside non-zero data.
 2021-01-08 17:40:06 +01:00
Thomas Waldmann 52bd55b29a integrate Chunk type, avoid hashing holes 2021-01-08 17:39:51 +01:00
Thomas Waldmann 7319f85b54 adapt the existing chunker tests 2021-01-08 17:33:25 +01:00
Thomas Waldmann 8c299696aa Chunker: yield Chunk namedtuple instead of bytes/memoryview 2021-01-08 01:10:44 +01:00
TW 37d4aee122
Merge pull request #5597 from Gelma/typos_dec 
Fix typos
 2021-01-07 19:21:03 +01:00
Thomas Waldmann 7a3a49e99b reverted changes to 3rd party code 
all algorithms/* stuff needs to be fixed upstream.

we just copy the files from there now and then.

https://github.com/lz4/lz4
https://github.com/facebook/zstd
https://github.com/Cyan4973/xxHash
 2021-01-07 18:18:15 +01:00
Thomas Waldmann 0984e00d31 revert changes to autogenerated files 
this stuff will be auto-generated, regularly done at release time.
 2021-01-07 18:08:49 +01:00
Andrea Gelmini 72e7c46fa7 Fix typos 2021-01-07 17:54:33 +01:00
TW 41d282bc14
Merge pull request #5615 from jedie/patch-1 
Change docs/install instruction: "python-virtualenv" -> "python3-virtualenv"
 2021-01-07 17:13:31 +01:00
Jens Diemer 5a585c5c2e
"python-virtualenv" -> "python3-virtualenv" 2021-01-07 10:31:26 +01:00
TW 268eb2e598
Merge pull request #5610 from ThomasWaldmann/update-changes-master 
update CHANGES
 2021-01-04 19:50:56 +01:00
Thomas Waldmann 4394fa45ba update CHANGES 2021-01-03 21:40:31 +01:00
TW 9088b1e85a
Merge pull request #5608 from ThomasWaldmann/improve-borg-check-logging-master 
check: debug log segment filename
 2021-01-03 19:04:40 +01:00
Thomas Waldmann f2cb17d66c check: debug log segment filename 2021-01-03 18:23:52 +01:00
Thomas Waldmann 73c04398f3 move requires_hardlinks upwards 2021-01-03 17:55:45 +01:00
axapaxa b291b91962
Add remote upload buffer (--remote-buffer) (#5574) 
add remote upload buffer (--remote-buffer)

- added new option --remote-buffer
- allow to_send to grow to selected size
- don't grow if wait is specified
- fill pipe on any command (including 'async_response')
- add new option to docs
- create EfficientBytesQueue to prevent recreation of buffer each time we send something
- add tests for EfficientBytesQueue
 2021-01-03 17:37:16 +01:00
Thomas Waldmann 806dc5084d do not recurse into duplicate roots, fixes #5603 2021-01-02 23:01:31 +01:00
Thomas Waldmann 257791274f add a test whether a duplicate root is skipped, see #5603 2021-01-02 23:00:52 +01:00
Thomas Waldmann 58c0a0186f add a test for hardlink extraction issue, see #5603 2021-01-02 23:00:38 +01:00
TW 2851a84003
Merge pull request #5561 from ThomasWaldmann/sparse-file-support 
sparse map / file map support for fixed size chunker
 2020-12-28 20:21:50 +01:00
TW 27651f6f28
Merge pull request #5600 from ThomasWaldmann/fix-mp 
fix cythonize crash on macOS/py39, fixes #5599
 2020-12-28 20:20:33 +01:00
Thomas Waldmann d2fe303967 fix cythonize crash on macOS/py39, fixes #5599 2020-12-28 19:56:08 +01:00
Thomas Waldmann 37a7436ff9 detect sparse support by fs 2020-12-28 19:53:52 +01:00
Thomas Waldmann c0c0da9c76 skip sparse tests if has_seek_hole is False 
also: do the os.SEEK_(HOLE|DATA) check only once
 2020-12-27 22:06:08 +01:00
Thomas Waldmann b8bb0494f6 create --sparse, file map support for the "fixed" chunker, see #14 
a file map can be:

- created internally inside chunkify by calling sparsemap, which uses
  SEEK_DATA / SEEK_HOLE to determine data and hole ranges inside a
  seekable sparse file.
  Usage: borg create --sparse --chunker-params=fixed,BLOCKSIZE ...
  BLOCKSIZE is the chunker blocksize here, not the filesystem blocksize!

- made by some other means and given to the chunkify function.
  this is not used yet, but in future this could be used to only read
  the changed parts and seek over the (known) unchanged parts of a file.

sparsemap: the generate range sizes are multiples of the fs block size.
           the tests assume 4kiB fs block size.
 2020-12-27 22:06:08 +01:00
TW 5d46395ed0
Merge pull request #5598 from ThomasWaldmann/pyi41-master 
vagrant: use py 3.9.1 pyi 4.1 to build binary
 2020-12-27 22:02:07 +01:00
Thomas Waldmann 7b060ea792 vagrant: use py 3.9.1 pyi 4.1 to build binary 2020-12-26 00:50:05 +01:00
TW 5abc4eb5d4
Merge pull request #5595 from ThomasWaldmann/docs-hardlinks-master 
Docs hardlinks master (fwd port)
 2020-12-26 00:31:15 +01:00
Thomas Waldmann f75c561e07 docs: add another case of attempted hardlink usage 2020-12-26 00:04:24 +01:00
Thomas Waldmann 3a828bea0b docs: fix description of borg upgrade hardlink usage, fixes #5518 2020-12-26 00:03:34 +01:00
TW 34c38ee052
Merge pull request #5594 from ThomasWaldmann/vagrant-no-duplicate-fuse-tests 
do not run the -fuse2 AND -none tests, fixes #5556
 2020-12-25 21:30:44 +01:00
Thomas Waldmann 30e0afb113 do not run the -fuse2 AND -none tests, fixes #5556 2020-12-25 21:23:12 +01:00