restic/internal/restic
Matthew Dawson df2c03a6a4
repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
..
testdata Update golden files 2017-09-11 17:52:22 +02:00
backend.go backend: Retry deletes 2017-12-22 22:41:28 +01:00
backend_find.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
backend_find_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
blob.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
blob_set.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
blob_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
buffer.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
cache.go Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
config.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
config_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
doc.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
file.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
file_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
find.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
find_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
hardlinks_index.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
hardlinks_index_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
id.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
id_int_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
id_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
ids.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
ids_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
idset.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
idset_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
lock.go small cleanup: 2017-10-25 12:03:55 -04:00
lock_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
lock_unix.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
lock_windows.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
node_darwin.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_freebsd.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_linux.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_openbsd.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
node_unix.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_unix_test.go Add argument to Skipf() 2018-01-17 23:14:37 +01:00
node_windows.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
node_xattr.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
progress.go Control progress rate with RESTIC_PROGRESS_FPS env 2017-10-26 14:46:56 +08:00
progress_unix.go Handle SIGINFO on all supported platforms 2017-09-02 22:06:31 +03:00
progress_unix_with_siginfo.go Handle SIGINFO on all supported platforms 2017-09-02 22:06:31 +03:00
rand_reader.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
readerat.go backend: Improve ReadAt 2017-09-24 23:11:23 +02:00
repository.go repository/master_index: Optimize Index.Lookup() 2018-01-23 22:25:56 -05:00
restorer.go restorer: Set directory mode as last step 2018-01-07 15:13:24 +01:00
restorer_test.go restorer: Set directory mode as last step 2018-01-07 15:13:24 +01:00
snapshot.go Correct debug message 2017-09-25 14:35:37 +02:00
snapshot_find.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
snapshot_policy.go removed unnacessary line 2017-09-10 10:41:07 +02:00
snapshot_policy_test.go added test cases 2017-09-10 12:23:28 +02:00
snapshot_test.go Remove all dot-imports 2017-10-02 15:06:39 +02:00
tag_list.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
testing.go replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00
testing_test.go Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
tree.go Allow sorting nodes in trees 2017-09-22 12:37:05 +02:00
tree_test.go replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 2017-12-03 07:22:14 -05:00