935327d480
sftp: slightly increase write concurrency
...
This should increase upload throughput for high latency links a bit.
2024-04-28 11:50:09 +02:00
669a669603
sftp: Fix upload performance issue
...
Since pkg/sftp 1.13.0 files were uploaded sequentially using 32kb chunks
instead of sending 64 chunks in parallel.
2024-04-28 11:48:26 +02:00
b15d867414
Merge pull request #4763 from MichaelEischer/refactor-prune
...
Refactor repair index / prune into the repository package
2024-04-22 22:24:53 +02:00
35277b7797
backend/mem: cleanup not found error message
2024-04-14 13:45:06 +02:00
55d56db31b
Merge pull request #4743 from MichaelEischer/deprecate-s3legacy-layout
...
Deprecate s3legacy layout
2024-04-11 22:09:34 +02:00
591b421c4a
Deprecate s3legacy layout
2024-04-10 21:27:56 +02:00
6e775d3787
Enhancement: option to send HTTP over unix socket
...
add tests for unix socket connection
switch HTTP rest-server test to use any free port
allow rest-server test graceful shutdown opportunity
2024-03-28 17:41:41 +01:00
c0514dd8ba
Fix linter errors (except for tests)
2024-02-10 22:58:10 +01:00
a763a5c67d
s3: minor cleanups for archive storage class handling
2024-01-20 11:25:28 +01:00
8ca58b487c
S3: do not set storage class for metadata when using archive storage
2024-01-20 11:04:15 +01:00
bd883caae1
CI: enable bodyclose linter
2024-01-19 21:17:18 +01:00
b1a8fd1d03
rest: fix and cleanup closing of http response body
...
If client.Do returns an error, then there's no body that has to be
closed. For requests for which we are not interested in the response
body, immediately drain and close the body to make sure it isn't
forgotten later on.
This change in particular adds the missing `Close()` call for the
`List()` command.
2024-01-19 21:17:17 +01:00
7b2de84763
Merge pull request #4618 from MichaelEischer/workaround-rclone-list-errors
...
rclone: Workaround for incorrect "not found" errors while listing files
2024-01-09 18:28:31 +01:00
2e8de9edfd
rclone: Workaround for incorrect "not found" errors while listing files
...
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:20:16 +01:00
02bc73f5eb
s3: minor code cleanups
2024-01-06 21:44:53 +01:00
20cf4777cb
s3: check for EnvAWS credentials before Static credentials
...
EnvAWS considers more environment variables, including AWS_SESSION_TOKEN
and thus should be checked first.
2024-01-06 21:43:47 +01:00
5ffb536aae
feat: support AWS assume role
2024-01-06 21:19:58 +01:00
241916d55b
Fix typos
2023-12-06 13:11:55 +01:00
c7b770eb1f
convert MemorizeList to be repository based
...
Ideally, code that uses a repository shouldn't directly interact with
the underlying backend. Thus, move MemorizeList one layer up.
2023-10-25 23:01:35 +02:00
1b8a67fe76
move Backend interface to backend package
2023-10-25 23:00:18 +02:00
ceb0774af1
backend: make LoadAll independent of restic package
2023-10-25 22:58:39 +02:00
b6d79bdf6f
restic: decouple restic.Handle
2023-10-25 22:54:07 +02:00
7881309d63
backend: move backend implementation helpers to util package
...
This removes code that is only used within a backend implementation from
the backend package. The latter now only contains code that also has
external users.
2023-10-25 22:54:07 +02:00
8e6fdf5edf
Merge pull request #4520 from awannabeengineer/load-retry-nonexistent
...
retry: Do not retry Load() if file does not exist
2023-10-25 20:42:05 +00:00
d15ffd9c92
retry: Do not retry Load() if file does not exist
2023-10-22 13:25:32 -05:00
7f05af02b9
Merge pull request #4480 from AgathaSorceress/add-rest-auth-env
...
Support reading basic auth credentials for REST server from environment variables
2023-10-21 17:41:08 +00:00
ab6defbace
Merge pull request #4519 from adamantike/backend/sftp/add-args-option
...
backend/sftp: Add sftp.args option
2023-10-21 17:37:46 +00:00
6a4d6d5da4
sftp: test that Args and Command options cannot be set at the same time
2023-10-21 19:26:39 +02:00
41f70f1f4f
backend/sftp: Add sftp.args option
...
Allow setting custom arguments for the `sftp` backend, by using the
`sftp.args` option. This is similar to the approach already implemented
in the `rclone` backend, to support new arguments without requiring
future code changes for each different SSH argument.
Closes #4241
2023-10-21 19:25:44 +02:00
6c7560e537
backend/b2: Switch to official library `Backblaze/blazer`
...
According to https://github.com/kurin/blazer Readme, the repository is
no longer maintained, and the official one at
https://github.com/Backblaze/blazer took over.
Closes #4148
2023-10-21 19:18:13 +02:00
f1877e721e
feat: support reading REST credentials from env
2023-10-20 13:04:58 +02:00
f750aa8dfb
backend/limiter: replace juju/ratelimit with x/time/rate
2023-10-17 22:49:25 +02:00
4a33370072
backend: Fix unreliable TestListCancel/Timeout test
...
The test uses `WithTimeout` to create a context that cancels the List
operation after a given delay. Several backends internally use a derived
child context created using WithCancel.
The cancellation of a context first closes the done channel of the
context (here: the `WithTimeout` context) and _afterwards_ propagates
the cancellation to child contexts (here: the `WithCancel` context).
Therefor if the List implementation uses a child context, then it may
take a moment until that context is also cancelled. Thus give the
context cancellation a moment to propagate.
2023-07-23 11:47:56 +02:00
96eada3d5f
backend/sema: add test for freeze functionality
2023-07-22 23:23:17 +02:00
f490288738
lock: freeze backend operations while refreshing stale lock
...
Freeze new backend operations while trying to refresh a stale lock.
2023-07-22 19:55:44 +02:00
978ebaac49
rest: use http status code constants
2023-07-14 22:39:38 +02:00
307aeb6849
rest: Don't return error if listing non-existent directory
...
When transferring a repository from S3 to, for example, a local disk
then all empty folders will be missing.
When saving files, the missing intermediate folders are created
automatically. Therefore, missing directories can be ignored by the
`List()` operation.
2023-07-14 22:39:38 +02:00
e990d3d483
azure: Support authentication using managed / workload identity
...
See https://github.com/Azure/azure-sdk-for-go/tree/sdk/azidentity/v1.3.0/sdk/azidentity
2023-07-09 14:47:03 +02:00
e36d17a6f8
azure: tweak documentation for endpoint suffix
2023-07-07 23:10:26 +02:00
068b115abc
added azure domain parameter
2023-07-07 23:10:26 +02:00
93038ed8f4
s3: restore retries for minio tests
2023-06-17 15:25:08 +02:00
b5511e8e4c
Fix linter warnings
2023-06-17 15:15:58 +02:00
50e0d5e6b5
backend: Hardcode backend scheme in Factory
...
Our ParseConfig implementations always expect a specific scheme, thus no
other scheme would work.
2023-06-17 15:15:58 +02:00
705ad51bcc
backend: check that StripPassword can be called
2023-06-17 15:15:58 +02:00
13a8b5822f
backend: Adjust tests to use the Factory to instantiate the backend
...
This drastically reduces the amount of duplicated test code.
2023-06-17 15:15:58 +02:00
3d3bb88745
b2: remove duplicate check for config file during repository creation
...
No other backend implements that check. The check that a repository is
not yet initialized is handled by the Repository later on.
2023-06-17 15:15:58 +02:00
3a3cf608f5
b2/s3: Move config validation from ApplyEnvironment to Open/Create
...
Conceptually the backend configuration should be validated when creating
or opening the backend, but not when filling in information from
environment variables into the configuration.
2023-06-17 15:15:58 +02:00
19ac12d95b
location: make genericBackendFactory private
2023-06-17 15:15:58 +02:00
3325a7c862
location: extract backend specific part of StripPassword
...
The tests for the rest backend now reside there.
2023-06-17 15:15:58 +02:00
555be49a79
location: Make ParseConfig-test backend agnostic
...
The backend specific parts of the test are now directly handled by the
respective backend. Duplicate tests were removed.
2023-06-17 15:15:58 +02:00
Michael Eischer