822422ef03
retry key loading on hash mismatch
2022-09-25 11:35:35 +02:00
78d2312ee9
Merge pull request #3854 from MichaelEischer/sparsefiles
...
restore: Add support for sparse files
2022-09-24 22:04:02 +02:00
19afad8a09
restore: support sparse restores also on windows
2022-09-24 21:39:39 +02:00
0f89f443c7
update sparse restore changelog
2022-09-24 21:39:39 +02:00
c147422ba5
repository: special case SaveBlob for all zero chunks
...
Sparse files contain large regions containing only zero bytes. Checking
that a blob only contains zeros is possible with over 100GB/s for modern
x86 CPUs. Calculating sha256 hashes is only possible with 500MB/s (or
2GB/s using hardware acceleration). Thus we can speed up the hash
calculation for all zero blobs (which always have length
chunker.MinSize) by checking for zero bytes and then using the
precomputed hash.
The all zeros check is only performed for blobs with the minimal chunk
size, and thus should add no overhead most of the time. For chunks which
are not all zero but have the minimal chunks size, the overhead will be
below 2% based on the above performance numbers.
This allows reading sparse sections of files as fast as the kernel can
return data to us. On my system using BTRFS this resulted in about
4GB/s.
2022-09-24 21:39:39 +02:00
34fe1362da
restorer: move zeroPrefixLen to restic package
2022-09-24 21:39:39 +02:00
a5ebd5de4b
restorer: Fix race condition in partialFile.WriteAt
...
The restorer can issue multiple calls to WriteAt in parallel. This can
result in unexpected orderings of the Truncate and WriteAt calls and
sometimes too short restored files.
2022-09-24 21:39:39 +02:00
5b6a77058a
Enable sparseness only conditionally
...
We can either preallocate storage for a file or sparsify it. This
detects a pack file as sparse if it contains an all zero block or
consists of only one block. As the file sparsification is just an
approximation, hide it behind a `--sparse` parameter.
2022-09-24 21:20:00 +02:00
3047bf611c
Changelog entry for sparse file restoring
2022-09-24 21:18:48 +02:00
5d4568d393
Write sparse files in restorer
...
This writes files by using (*os.File).Truncate, which resolves to the
truncate system call on Unix.
Compared to the naive loop,
for _, b := range p {
if b != 0 {
return false
}
}
the optimized allZero is about 10× faster:
name old time/op new time/op delta
AllZero-8 1.09ms ± 1% 0.09ms ± 1% -92.10% (p=0.000 n=10+10)
name old speed new speed delta
AllZero-8 3.84GB/s ± 1% 48.59GB/s ± 1% +1166.51% (p=0.000 n=10+10)
2022-09-24 21:18:48 +02:00
eb83402d39
Merge pull request #3935 from miles170/master
...
Only display the message if there were locks to be removed
2022-09-24 20:53:13 +02:00
ef58ddd7b1
Merge pull request #3923 from MichaelEischer/fix-flaky-cache-test
...
cache: fix flaky TestFileSaveConcurrent on windows
2022-09-24 20:52:55 +02:00
7fc178aaf4
internal/cache: extend description of cache sharing test failure
2022-09-24 13:07:01 +02:00
1acbda18f8
Only display the message if there were locks to be removed
...
`restic unlock` now only shows `successfully removed locks` if there were locks to be removed.
In addition, it also reports the number of the removed lock files.
2022-09-24 19:02:24 +08:00
da1a359c8b
Merge pull request #3927 from MichaelEischer/faster-index-each
...
Speed up MasterIndex.Each
2022-09-24 12:35:23 +02:00
041a51512a
Merge pull request #3780 from jkmw/fix/2578
...
Remove existing path before restoring a symlink
2022-09-24 12:34:42 +02:00
1ebd57247a
repository: optimize MasterIndex.Each
...
Sending data through a channel at very high frequency is extremely
inefficient. Thus use simple callbacks instead of channels.
> name old time/op new time/op delta
> MasterIndexEach-16 6.68s ±24% 0.96s ± 2% -85.64% (p=0.008 n=5+5)
2022-09-24 12:21:59 +02:00
825b95e313
repository: add benchmark for MasterIndex.Each
2022-09-24 12:21:59 +02:00
1220fe9650
internal/cache: Concurrent use of cache not working on Windows
2022-09-17 19:49:44 +02:00
ef618bdd3f
use os.Remove if path already exists on symlink restore
2022-09-14 08:14:31 +02:00
b48766d7b8
Merge pull request #3928 from restic/rawtaz-doc-b2-s3
...
doc: Clarify S3 recommendation for B2 slightly
2022-09-13 20:59:50 +02:00
20f1913ef7
doc: Clarify S3 recommendation for B2 slightly
...
This gives slightly more background to the recommendation for using restic's S3 backend with Backblaze B2.
2022-09-12 17:48:59 +02:00
d79e61ce5d
Merge pull request #3925 from hgraeber/add-powershell-completion
...
Add powershell completion
2022-09-11 01:04:57 +02:00
988b386e8b
Add powershell completion
...
- Add code for powersehll complition available in cobra
- Add documentation for powershell completion
- Add changelog for pr3925
2022-09-11 00:44:12 +02:00
14d09a6081
Merge pull request #3912 from MichaelEischer/cleanup-snapshot-filter-options
...
Clean up snapshot filter options
2022-09-11 00:18:42 +02:00
381da0443a
tweak snapshot filter descriptions
2022-09-10 23:50:20 +02:00
8b9778d537
Merge pull request #3900 from MichaelEischer/b2-init-timeout
...
Add timeout for the initial connection to B2
2022-09-10 23:28:59 +02:00
17c27400f8
Merge pull request #3921 from MichaelEischer/filter-cleanup-error-handling
...
filter: deduplicate error handling for pattern validation
2022-09-10 23:24:50 +02:00
f76643bd2e
Merge pull request #3894 from MichaelEischer/filter-mount-exit-code
...
Mount should return exit code 0 after pressing Ctrl-C
2022-09-10 23:22:01 +02:00
be9ccc186e
Merge pull request #3875 from MichaelEischer/fix-fuse-context-cancel
...
mount: Fix input/output errors for canceled syscalls
2022-09-10 23:20:29 +02:00
2363e5c083
Merge pull request #3913 from MichaelEischer/better-migrate-error-message
...
migrate: Report why an migration cannot be applied
2022-09-09 23:37:25 +02:00
8e0ca80547
filter: deduplicate error handling for pattern validation
2022-09-09 23:12:41 +02:00
d6309961c5
deduplicate the snapshot filter cli option setup
2022-09-04 10:27:33 +02:00
8b4dd70013
migrate: Report why an migration cannot be applied
...
Just returning that `Migration upgrade cannot be applied: check failed`
is not too useful when running `migrate upgrade_repo_v2`.
2022-09-03 11:49:31 +02:00
7689d6c679
normalize help text for host, tag and path options
2022-09-03 00:06:38 +02:00
6c69f08a7b
Merge pull request #3905 from DRON-666/haspaths-linear
...
Reduce quadratic time complexity of `Snapshot.HasPaths`
2022-08-30 20:35:56 +02:00
3e70bac56e
Merge pull request #3898 from MichaelEischer/fix-copy-hang
...
don't hang when `copy` uses a single connection
2022-08-30 20:23:39 +02:00
2a630c51c1
Add changelog
2022-08-30 20:22:07 +02:00
d0f1060df7
Fix quadratic time complexity of Snapshot.HasPaths
2022-08-30 04:38:17 +03:00
f481ad64c8
Merge pull request #3904 from lbausch/add-newline
...
Add newline to keep prompt intact
2022-08-29 21:43:18 +02:00
7ddd803e46
Add newline to keep prompt intact
2022-08-29 17:37:49 +02:00
e5b2c4d571
b2: sniff the error that caused init retry loops
2022-08-28 17:46:03 +02:00
dc2db2de5e
b2: cancel connection setup after a minute
...
If the connection to B2 fails, the library enters an endless loop.
2022-08-28 14:56:17 +02:00
7682149c9d
repository: cleanup copy connection count check
2022-08-28 11:40:56 +02:00
b03277ead5
repository: don't hang when copying using a single connection
2022-08-28 11:40:31 +02:00
1b233c4e2e
Merge pull request #2661 from creativeprojects/issue-1734
...
"Everything is ok" message after retrying
2022-08-28 11:04:59 +02:00
4042db5169
Add changelog
2022-08-27 22:36:19 +02:00
be6baaec12
Add success callback to the backend
2022-08-27 22:27:15 +02:00
baf58fbaa8
Add unit tests
2022-08-27 22:21:06 +02:00
d629333efe
Add function to notify of success after retrying
2022-08-27 22:21:06 +02:00
Michael Eischer