1
0
Fork 0
mirror of https://github.com/restic/restic.git synced 2024-12-27 18:28:30 +00:00
restic/internal/walker
Michael Eischer 6f53ecc1ae adapt workers based on whether an operation is CPU or IO-bound
Use runtime.GOMAXPROCS(0) as worker count for CPU-bound tasks,
repo.Connections() for IO-bound task and a combination if a task can be
both. Streaming packs is treated as IO-bound as adding more worker
cannot provide a speedup.

Typical IO-bound tasks are download / uploading / deleting files.
Decoding / Encoding / Verifying are usually CPU-bound. Several tasks are
a combination of both, e.g. for combined download and decode functions.
In the latter case add both limits together. As the backends have their
own concurrency limits restic still won't download more than
repo.Connections() files in parallel, but the additional workers can
decode already downloaded data in parallel.
2022-07-03 12:19:26 +02:00
..
testing.go Add walker for trees in the repo 2018-06-09 23:35:20 +02:00
walker.go Limit number of large tree blobs loaded in parallel by StreamTrees 2022-02-19 12:26:09 +01:00
walker_test.go adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00