Commit Graph

31 Commits

Author SHA1 Message Date
Michael Eischer 921e328b56 restore: Fix linting error 2021-09-19 14:41:07 +02:00
Michael Eischer 2cdc0719af restorer: Sanitize verify errors 2021-09-19 14:01:26 +02:00
greatroar bdcdfaf6b4 restore --verify: buffer reuse consistency and comment 2021-09-19 13:11:27 +02:00
greatroar 2b94742ca5 Replace no-op closures in restorer by nil check 2021-09-19 13:11:16 +02:00
greatroar d357744104 Handle canceled context in restore --verify properly 2021-09-19 13:11:05 +02:00
greatroar d4225ec803 Simplify buffer growing in Restorer.verifyFile
Suggested-by: Igor Fedorenko <igor@ifedorenko.com>
2021-09-19 13:10:55 +02:00
greatroar de8521ae56 Report number of successes from Restorer.VerifyFiles
Previously, the number of attempts was reported.
2021-09-19 13:10:44 +02:00
greatroar bb066cf7d3 Concurrent Restorer.VerifyFiles
Time to verify a 2GB snapshot down from 9.726s to 4.645s (-52%).
2021-09-19 13:10:41 +02:00
greatroar 556424d61b Reuse buffer in Restorer.VerifyFiles
Time to verify a 2GB snapshot down from 11.568s to 9.726s (-16%).
2021-09-19 12:55:31 +02:00
greatroar 973fa921cb Test and document Restorer.VerifyFiles 2021-09-19 12:52:11 +02:00
Alexander Weiss 5e22ae10f1 Add error handling for fileRestorer 2021-01-31 14:22:57 +01:00
Alexander Neumann 36c5d39c2c Fix issues reported by semgrep 2020-12-11 09:41:59 +01:00
Alexander Neumann f0d49ca600
Merge pull request #2933 from MichaelEischer/less-context-todo
Replace most usages of context.TODO()
2020-10-10 15:03:44 +02:00
kitone 6099f81692 Fix #1212 restore code produces inconsistent timestamps/permissions.
Keep track of restored child status so parent and root directory not selected by filter will also restore metadata when traversing tree.
2020-10-10 13:46:44 +02:00
kitone 9d7f616190 Improve restorer debug log information 2020-10-10 13:46:43 +02:00
Michael Eischer 603bb0e309 restore: Use proper context while loading snapshot 2020-10-09 22:37:56 +02:00
Michael Eischer 8cc9514879 restorer: pre-allocate files before loading chunks 2020-09-07 21:41:47 +02:00
Alexander Weiss 9d1fb94c6c make Lookup() return all blobs
+ simplify syntax
2020-07-25 21:18:34 +02:00
Igor Fedorenko f17ffa0283 restorer: Allow writing target file blobs out of order
Much simpler implementation that guarantees each required pack
is downloaded only once (and hence does not need to manage
pack cache). Also improves large file restore performance.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2020-02-26 16:14:45 +01:00
Alexander Neumann c0fca3f50a restore: Close files tested with --verify
Before, the target files were opened once per blob and never closed,
this commit fixes that.
2018-10-14 21:00:14 +02:00
Igor Fedorenko bda8d7722e restorer: Optimize empty file restore
don't create fileInfo structs for empty files. this saves memory.
this also avoids extra serial scan of all fileInfo, which should
make restore faster and more consistent.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko c2bcb764cd restorer: Add a note on hardlink metadata
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko 9e24154ec9 restore: significantly reduce memory footprint
reworked restore error callback to use file location
path instead of much heavier Node. this reduced restore
memory usage by as much as 50% in some of my tests.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko 9f3ca97ee8 restore: Chang fileInfo to use snapshot location instead of target path
* uses less memory as common prefix is only stored once
* stepping stone for simpler error callback api, which
  will allow further memory footprint reduction

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko da57302fca restore: Removed legacy restore implementation
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Igor Fedorenko 1869930d95 restore: New optimized multithreaded implementation
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Alexander Neumann 44924ba043 restorer: Fix traverseTree
traverseTree() was meant to call enterDir() whenever a directory is
selected for restore, either explicitly or implicitly (=contains a file
which is to be restored). After restoring a file, leaveDir() is called
in reverse order for all intermediate directories so that the metadata
can be restored.

When a directory is selected implicitly, the metadata for it is
restored. This is different from the previous restorer behavior, which
created implicitly selected intermediate directories with permissions
0700 (only user can read/write it).

This commit changes the behavior back to the old one. Only a directory
is explicitly selected for restore, enterDir()/leaveDir() are called for
it. Otherwise, only visitNode() is called, so visitNode() needs to make
sure the parent directory exists. If the directory is explicitly
included, leaveDir() will then restore the metadata correctly.

When we decide to change the behavior (restore metadata for all
intermediate directories, even if selected implicitly), we should do
that in the selection functions, not here.

This finally resolves #1870
2018-07-21 23:24:40 +02:00
Alexander Neumann 92421ec47f restore: Make sure target directory exists 2018-07-15 16:02:04 +02:00
Igor Fedorenko e206680947 restore: New --verify flag to verify restored files content
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko 5fa6dc53cb Refactor: introduced restorer tree visitor
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko 26be094f28 Refactor: moved restorer to separate package
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Renamed from internal/restic/restorer.go (Browse further)