restic/internal/repository
Matthew Dawson df2c03a6a4
repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
..
testdata Moves files 2017-07-23 14:19:13 +02:00
doc.go Moves files 2017-07-23 14:19:13 +02:00
index.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
index_test.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
key.go backend: Only return top-level files for most dirs 2017-12-14 19:14:16 +01:00
master_index.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
master_index_test.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
packer_manager.go replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00
packer_manager_test.go repo: Split packers for tree and data 2017-09-22 15:36:47 +02:00
parallel.go Ignore files with invalid name in the repo 2017-09-10 11:00:07 +02:00
parallel_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
pool.go Moves files 2017-07-23 14:19:13 +02:00
repack.go replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00
repack_test.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
repository.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
repository_test.go replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00
testing.go Rename KDFParams -> Params 2017-10-28 10:28:29 +02:00