Commit Graph

8 Commits

Author SHA1 Message Date
Michael Eischer 9aa2eff384 Add plumbing to calculate backend specific file hash for upload
This enables the backends to request the calculation of a
backend-specific hash. For the currently supported backends this will
always be MD5. The hash calculation happens as early as possible, for
pack files this is during assembly of the pack file. That way the hash
would even capture corruptions of the temporary pack file on disk.
2021-08-04 22:17:46 +02:00
aawsome 0fed6a8dfc
Use "pack file" instead of "data file" (#2885)
- changed variable names, especially changed DataFile into PackFile
- changed in some comments
- always use "pack file" in docu
2020-08-16 11:16:38 +02:00
greatroar f4cd2a7120 Make backend benchmarks fairer by removing checks
Checking whether the right data is returned takes up half the time in
some benchmarks. Results for local backend benchmarks on linux/amd64:

name                                      old time/op    new time/op    delta
Backend/BenchmarkLoadFile-8                 4.89ms ± 0%    2.72ms ± 1%   -44.26%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8           936µs ± 6%     439µs ±15%   -53.07%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8     940µs ± 1%     456µs ±10%   -51.50%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                     23.9ms ±14%    24.8ms ±41%      ~     (p=0.690 n=5+5)

name                                      old speed      new speed      delta
Backend/BenchmarkLoadFile-8               3.43GB/s ± 0%  6.16GB/s ± 1%   +79.40%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8        4.48GB/s ± 6%  9.63GB/s ±14%  +114.78%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8  4.46GB/s ± 1%  9.22GB/s ±10%  +106.74%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                    706MB/s ±13%   698MB/s ±31%      ~     (p=0.690 n=5+5)
2020-06-17 13:11:45 +02:00
Alexander Neumann 99f7fd74e3 backend: Improve Save()
As mentioned in issue [#1560](https://github.com/restic/restic/pull/1560#issuecomment-364689346)
this changes the signature for `backend.Save()`. It now takes a
parameter of interface type `RewindReader`, so that the backend
implementations or our `RetryBackend` middleware can reset the reader to
the beginning and then retry an upload operation.

The `RewindReader` interface also provides a `Length()` method, which is
used in the backend to get the size of the data to be saved. This
removes several ugly hacks we had to do to pull the size back out of the
`io.Reader` passed to `Save()` before. In the `s3` and `rest` backend
this is actively used.
2018-03-03 15:49:44 +01:00
Igor Fedorenko d58ae43317 Reworked Backend.Load API to retry errors during ongoing download
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:12:14 -05:00
Alexander Neumann 23c903074c Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
Alexander Neumann 6caeff2408 Run goimports 2017-07-23 14:21:03 +02:00
Alexander Neumann 83d1a46526 Moves files 2017-07-23 14:19:13 +02:00
Renamed from src/restic/backend/test/benchmarks.go (Browse further)