Commit Graph

7 Commits

Author SHA1 Message Date
Michael Eischer d05f6211d1 lock: Do not limit backend concurrency for lock files
restic must be able to refresh lock files in time. However, large
uploads over slow connections can cause the lock refresh to be stuck
behind the large uploads and thus time out.
2023-05-08 19:04:46 +02:00
Michael Eischer 831f593b87 backend/sema: Add tests 2023-04-22 13:20:20 +02:00
Michael Eischer ebba233a3a backend/sema: rename constructor to NewBackend 2023-04-22 12:32:57 +02:00
Michael Eischer f27750e270 backend/sema: rename type to connectionLimitedBackend 2023-04-21 22:53:09 +02:00
Michael Eischer 6042df075f migrations: Fix S3 backend detection 2023-04-14 22:32:16 +02:00
Michael Eischer 8e1e3844aa backend: factor out connection limiting and parameter validation
The SemaphoreBackend now uniformly enforces the limit of concurrent
backend operations. In addition, it unifies the parameter validation.

The List() methods no longer uses a semaphore. Restic already never runs
multiple list operations in parallel.

By managing the semaphore in a wrapper backend, the sections that hold a
semaphore token grow slightly. However, the main bottleneck is IO, so
this shouldn't make much of a difference.

The key insight that enables the SemaphoreBackend is that all of the
complex semaphore handling in `openReader()` still happens within the
original call to `Load()`. Thus, getting and releasing the semaphore
tokens can be refactored to happen directly in `Load()`. This eliminates
the need for wrapping the reader in `openReader()` to release the token.
2023-04-14 22:32:15 +02:00
greatroar 910d917b71 backend: Move semaphores to a dedicated package
... called backend/sema. I resisted the temptation to call the main
type sema.Phore. Also, semaphores are now passed by value to skip a
level of indirection when using them.
2022-06-18 10:01:58 +02:00