Commit Graph

1752 Commits

Author SHA1 Message Date
Michael Eischer 4a33370072 backend: Fix unreliable TestListCancel/Timeout test
The test uses `WithTimeout` to create a context that cancels the List
operation after a given delay. Several backends internally use a derived
child context created using WithCancel.

The cancellation of a context first closes the done channel of the
context (here: the `WithTimeout` context) and _afterwards_ propagates
the cancellation to child contexts (here: the `WithCancel` context).
Therefor if the List implementation uses a child context, then it may
take a moment until that context is also cancelled. Thus give the
context cancellation a moment to propagate.
2023-07-23 11:47:56 +02:00
Michael Eischer f12bbd9229 restic: check that Node.LinkTarget can handle non-utf8 characters 2023-07-23 00:05:14 +02:00
Michael Eischer 6adb629608 Add support for non-utf8 symlink targets 2023-07-23 00:05:14 +02:00
Michael Eischer 25ff9fa893
Merge pull request #4334 from MichaelEischer/snapshot-subtree-syntax
Add support for snapshot:path syntax
2023-07-22 23:59:26 +02:00
Michael Eischer 3f63b53090 lock: Shrink critical section in RefreshStaleLock
A cleaning up after a failed lock refresh attempt does not require a
mutex.
2023-07-22 23:29:14 +02:00
Michael Eischer 96eada3d5f backend/sema: add test for freeze functionality 2023-07-22 23:23:17 +02:00
Michael Eischer 8154f6a77a Ensure consistent naming for `<snapshot>:<subfolder>` syntax 2023-07-22 19:55:57 +02:00
Michael Eischer 090f9d6237 restic: Cleanup and simplify TestCreateSnapshot 2023-07-22 19:55:57 +02:00
Michael Eischer 321cc35cde restic: add test for FindTreeDirectory 2023-07-22 19:55:57 +02:00
Michael Eischer a97915642c restic: Ensure snapshots created by TestCreateSnapshot are valid
All nodes in a tree must be sorted by node name. Otherwise functionality
like searching for a specific tree node will break.
2023-07-22 19:55:57 +02:00
Michael Eischer c64d81063e test snapshot filtering with snapshot:path syntax 2023-07-22 19:55:57 +02:00
Michael Eischer 85860e6e97 Add support for snapshot:subpath syntax
This snapshot specification syntax is supported by the cat, diff, dump,
ls and restore command.
2023-07-22 19:55:57 +02:00
Michael Eischer 5d9b0d894e lock: add unit test for RefreshStaleLock 2023-07-22 19:55:44 +02:00
Michael Eischer d4bf7a3cb1 restic: cleanup lock test 2023-07-22 19:55:44 +02:00
Michael Eischer 24c8a33da9 restic: reduce sleeps in lock tests 2023-07-22 19:55:44 +02:00
Michael Eischer f490288738 lock: freeze backend operations while refreshing stale lock
Freeze new backend operations while trying to refresh a stale lock.
2023-07-22 19:55:44 +02:00
Michael Eischer 51718ec561 restic: generic implemention of AsBackend 2023-07-22 19:55:44 +02:00
Michael Eischer bee3231ed4 lock: try refreshing of stale locks
A stale lock may be refreshed if it continues to exist until after a
replacement lock has been created. This ensures that a repository was
not unlocked in the meantime.
2023-07-22 19:55:44 +02:00
Michael Eischer 978ebaac49 rest: use http status code constants 2023-07-14 22:39:38 +02:00
Michael Eischer 307aeb6849 rest: Don't return error if listing non-existent directory
When transferring a repository from S3 to, for example, a local disk
then all empty folders will be missing.

When saving files, the missing intermediate folders are created
automatically. Therefore, missing directories can be ignored by the
`List()` operation.
2023-07-14 22:39:38 +02:00
Michael Eischer e990d3d483 azure: Support authentication using managed / workload identity
See https://github.com/Azure/azure-sdk-for-go/tree/sdk/azidentity/v1.3.0/sdk/azidentity
2023-07-09 14:47:03 +02:00
Michael Eischer 9d44682e3e fuse: fix windows build 2023-07-08 22:40:22 +02:00
Michael Eischer 4d43509423
Merge pull request #4379 from chenxiaolong/symlink_xattrs
Add support for extended attributes on symlinks
2023-07-08 16:56:51 +00:00
Michael Eischer ea9ad77e05 restic: refactor node test 2023-07-08 18:54:56 +02:00
Michael Eischer cc84884d2e restic: basic xattr test for files/dirs 2023-07-08 18:49:21 +02:00
Michael Eischer 4a5ae2ba84 restic: test NodeFromFileInfo for symlinks 2023-07-08 18:18:13 +02:00
Michael Eischer 1f1e50f49e fuse: add test for symlink xattr 2023-07-08 18:02:17 +02:00
Michael Eischer f3c3b0f377 fuse: deduplicate xattr code 2023-07-08 17:41:45 +02:00
Michael Eischer bbac74b172 add program version to snapshot 2023-07-07 23:27:10 +02:00
Michael Eischer a280b7364e
Merge pull request #4395 from greatroar/parsebytes
cmd, ui: Move size parsing code and make it more robust
2023-07-07 21:24:47 +00:00
Michael Eischer e36d17a6f8 azure: tweak documentation for endpoint suffix 2023-07-07 23:10:26 +02:00
arjunajesh 068b115abc added azure domain parameter 2023-07-07 23:10:26 +02:00
greatroar 41a5bf357f cmd, ui: Move size parsing code and make it more robust 2023-07-02 21:44:50 +02:00
Andrew Gunnerson 8c02ebb029
Add support for extended attributes on symlinks
Linux allows the use of non-`user.` extended attributes on symlinks. One
of the main users of this functionality is SELinux's `security.selinux`
xattr for storing a path's label. By storing symlink xattrs, restic is
now suitable for backing up the root filesystem on Linux distributions
that use SELinux.

This commit adds support for symlink xattrs when backing up data,
restoring data, and mounting snapshots via a fuse mount. All calls to
the xattr library have been updated to the use `L` variants of the
various functions, which always operate on the path given, without
following symlinks.

Fixes: #4375

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-06-19 14:37:31 -04:00
Michael Eischer 74ca82a6f8
Merge pull request #4362 from restic/unified-backend-factory
Unified backend initialization
2023-06-17 15:52:37 +02:00
Michael Eischer 93038ed8f4 s3: restore retries for minio tests 2023-06-17 15:25:08 +02:00
Gautam Menghani 8da5a6649b Preserve oldest snapshot when keep-within* does not collect enough 2023-06-17 15:17:20 +02:00
Michael Eischer 1257c2c075 forget: Add comments to snapshot policy 2023-06-17 15:16:58 +02:00
Gautam Menghani 182b9796e4 Issue #3624: Preserve oldest snapshot when keep-* values are not satisfied 2023-06-17 15:16:58 +02:00
Michael Eischer b5511e8e4c Fix linter warnings 2023-06-17 15:15:58 +02:00
Michael Eischer 50e0d5e6b5 backend: Hardcode backend scheme in Factory
Our ParseConfig implementations always expect a specific scheme, thus no
other scheme would work.
2023-06-17 15:15:58 +02:00
Michael Eischer 705ad51bcc backend: check that StripPassword can be called 2023-06-17 15:15:58 +02:00
Michael Eischer 13a8b5822f backend: Adjust tests to use the Factory to instantiate the backend
This drastically reduces the amount of duplicated test code.
2023-06-17 15:15:58 +02:00
Michael Eischer 3d3bb88745 b2: remove duplicate check for config file during repository creation
No other backend implements that check. The check that a repository is
not yet initialized is handled by the Repository later on.
2023-06-17 15:15:58 +02:00
Michael Eischer 3a3cf608f5 b2/s3: Move config validation from ApplyEnvironment to Open/Create
Conceptually the backend configuration should be validated when creating
or opening the backend, but not when filling in information from
environment variables into the configuration.
2023-06-17 15:15:58 +02:00
Michael Eischer 19ac12d95b location: make genericBackendFactory private 2023-06-17 15:15:58 +02:00
Michael Eischer 3325a7c862 location: extract backend specific part of StripPassword
The tests for the rest backend now reside there.
2023-06-17 15:15:58 +02:00
Michael Eischer 555be49a79 location: Make ParseConfig-test backend agnostic
The backend specific parts of the test are now directly handled by the
respective backend. Duplicate tests were removed.
2023-06-17 15:15:58 +02:00
Michael Eischer 9aa9e0d1ec local/sftp: move limiter setup into backend 2023-06-17 15:15:58 +02:00
Michael Eischer 7d12c29286 backend: Unify backend construction using factory and registry
This unified construction removes most backend-specific code from
global.go. The backend registry will also enable integration tests to
use custom backends if necessary.
2023-06-17 15:15:57 +02:00