mirror
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Releases Wiki Activity
7,991 Commits 7 Branches 44 Tags 80 MiB
Commit Graph

7991 Commits

Author SHA1 Message Date
Michael Eischer 80132e71d8
Merge pull request #4808 from MichaelEischer/insecure-no-password 
Implement `--insecure-no-password` option.
 2024-05-24 22:58:25 +02:00
Michael Eischer 130506250f document insecure-no-password 2024-05-24 22:38:20 +02:00
Michael Eischer 1d2277b4c3 Add --insecure-no-password option 
This also includes two derived options `--from-insecure-no-password`
used for commands that require specifying a source repository. And
`--new-insecure-no-password` for the `key add` and `key passwd`
commands.

Specifying `--insecure-no-password` disabled the password prompt and
immediately uses an empty password. Passing a password via CLI option or
environment variable at the same time is an error.
 2024-05-24 22:38:20 +02:00
Michael Eischer d4b0d21199 key add/passwd: deduplicate options setup and remove globals 
The current pattern of using a global options variable is problematic.
 2024-05-24 22:38:20 +02:00
Michael Eischer 55cb8d174a
Merge pull request #4354 from MichaelEischer/associated-data 
Reduce prune memory usage
 2024-05-24 22:26:22 +02:00
Michael Eischer 436afbff23 add changelog for memory efficient prune 2024-05-24 22:18:14 +02:00
Michael Eischer 3c7b7efdc9 repository: remove prune plan parts once they are no longer necessary 2024-05-24 22:18:14 +02:00
Michael Eischer 462b82a060 index: reduce size of compressed indexes 
use the same index size for compressed and uncompressed indexes.
Otherwise, decoding the index of a compressed repository requires
significantly more memory.
 2024-05-24 22:18:14 +02:00
Michael Eischer 77873f5a9d repository: let prune control data structure of usedBlobs set 2024-05-24 22:18:14 +02:00
Michael Eischer 2033c02b09 index: replace CountedBlobSet with AssociatedSet 2024-05-24 22:18:14 +02:00
Michael Eischer 93098e9265 prune: hide implementation details of counted blob set 2024-05-24 21:42:56 +02:00
Michael Eischer ff4775a15f
Merge pull request #4812 from MichaelEischer/streaming-index-rewrite 
Resumable prune & memory-efficient index rewrite
 2024-05-24 21:41:30 +02:00
Michael Eischer 860b595a8b backend: increase watchdog test timeout for deflaking 2024-05-24 21:33:17 +02:00
Michael Eischer f680a2331d add changelog for streaming index rewrite 2024-05-24 21:33:17 +02:00
Michael Eischer 027cc64737 repository: fix prune heuristic to allow resuming interrupted runs 
Pack files created by interrupted prune runs, appear to consist only of
duplicate blobs on the next run. This caused the previous heuristic to
ignore those pack files. Now, a duplicate blob in a specific pack file
is also selected if that pack file only contains duplicate blobs. This
allows prune to select the already rewritten pack files.
 2024-05-24 21:33:17 +02:00
Michael Eischer e52033a8bd index: slightly reduce Rewrite concurrency 
The index operations are likely CPU-bounded. Thus, reduce the
concurrency accordingly.
 2024-05-24 21:33:17 +02:00
Michael Eischer 57d69aa640 index: cleanup SaveIndex method 2024-05-24 21:33:17 +02:00
Michael Eischer 2ca1c37721 index: additional tests for new index save methods 2024-05-24 21:33:17 +02:00
Michael Eischer 5f7b48e65f index: replace Save() method with Rewrite and SaveFallback 
Rewrite implements a streaming rewrite of the index that excludes the
given packs. For this it loads all index files from the repository and
only modifies those that require changes. This will reduce the index
churn when running prune. Rewrite does not require the in-memory index
and thus can drop it to significantly reduce the memory usage.

However, `prune --unsafe-recovery` cannot use this strategy and requires
a separate method to save the whole in-memory index. This is now handled
using SaveFallback.
 2024-05-24 21:33:17 +02:00
Michael Eischer ad98fbf7dd restic: add IDSet.Clone() method 2024-05-24 21:33:17 +02:00
Michael Eischer 72482ce5bd index: misc cleanups 2024-05-24 21:33:17 +02:00
Michael Eischer 68fa0e0305 prune: no longer disable automatic index updates 
this allows prune to resume an interrupted prune run.
 2024-05-24 21:33:17 +02:00
Michael Eischer 9aa0c90fb2 index: remove supersedes field 
Using the field with its current semantics is nearly impossible to get
right. Remove it as it will be replaced anyways in repository format 3.
 2024-05-24 21:33:17 +02:00
Michael Eischer 76e6719f2e repository: make CreateIndexFromPacks method private 2024-05-24 21:33:17 +02:00
Michael Eischer 04ad9f0c0c repository: remove Packer and SavePacker from public interface 2024-05-24 21:33:17 +02:00
Michael Eischer 550d1eeac3 repository: remove SaveIndex from interface 
The method is now only indirectly accessible via Prune or RepairIndex.
 2024-05-24 21:33:17 +02:00
Michael Eischer fb59e00614 index: rewrite MasterIndex load/save test to be independent of repository 2024-05-24 21:33:17 +02:00
Michael Eischer 447b486c20 index: deduplicate index loading of check and repository 2024-05-24 21:33:17 +02:00
Michael Eischer 6ca12c1b4a archiver: replace most uses of restic.Repository 2024-05-24 21:33:17 +02:00
Michael Eischer 864995271e repository: unwrap BlobHandle parameters of LookupBlob 
The method now uses the same parameters as LookupBlobSize.
 2024-05-24 21:33:17 +02:00
Michael Eischer 1266a4932f repository: fix parameter order of LookupBlobSize 
All methods should use blobType followed by ID.
 2024-05-24 21:33:17 +02:00
Michael Eischer e848ad651a restic: name parameters in restic interface 2024-05-24 21:33:17 +02:00
Michael Eischer 0bb0720348 test cleanups 2024-05-24 21:33:17 +02:00
Michael Eischer c01bcb1001 archiver: remove unused masterIndex from test 2024-05-24 21:33:17 +02:00
Michael Eischer 0aa5c53842 repository: replace HasBlob with LookupBlobSize 2024-05-24 21:33:17 +02:00
Michael Eischer 8f1e70cd9b repository: remove clearIndex and packSize from public interface 2024-05-24 21:33:17 +02:00
Michael Eischer 4df887406f repository: inline MasterIndex interface into Repository interface 2024-05-24 21:33:17 +02:00
Michael Eischer 3eeb6723cd
Merge pull request #4764 from MichaelEischer/safe-keep-tag 
Prevent unsafe uses of `forget --keep-tag`
 2024-05-24 20:51:20 +02:00
Michael Eischer 3cc6827f09 forget: add feature flag for safe --keep-tags behavior 2024-05-24 20:45:33 +02:00
Michael Eischer 7948912b31 document --unsafe-allow-remove-all option 2024-05-24 20:45:33 +02:00
Michael Eischer 3f46808898 add forget safety net changelogs 2024-05-24 20:45:33 +02:00
Michael Eischer 82ca0030b7 forget: test --unsafe-allow-remove-all and --keep-tags safety check 2024-05-24 20:45:33 +02:00
Michael Eischer 57f9739573 forget: Add --unsafe-allow-remove-all option 
To prevent accidentally wiping all snapshots from a repository, that
option can only be used if either a snapshot filter or a keep policy is
specified.

Essentially, the option allows `forget --tag something
--unsafe-allow-remove-all` calls to remove all snapshots with a specific
tag.
 2024-05-24 20:45:33 +02:00
Michael Eischer 5b7952e426 forget: return error if no policy was specified 2024-05-24 20:45:33 +02:00
Michael Eischer c0e1f36830 forget: refuse deleting the last snapshot in a snapshot group 
`--keep-tag invalid-tag` was previously able to wipe all snapshots in a
repository. As a user specified a `--keep-*` option this is likely
unintentional. This forbid deleting all snapshot if a `--keep-*` option
was specified to prevent data loss. (Not specifying such an option
currently also causes the command to abort)
 2024-05-24 20:45:33 +02:00
Michael Eischer d106ad6921 restic: regenerate snapshot keep policy golden test files 2024-05-24 20:45:33 +02:00
Michael Eischer 16ef4d515b
Merge pull request #4784 from MichaelEischer/rework-backend-retries 
Rework backend retries
 2024-05-24 20:29:54 +02:00
Michael Eischer e4a48085ae backend/retry: feature flag new retry behavior 2024-05-24 20:24:02 +02:00
Michael Eischer 723247c8e5 add changelog for longer retries 2024-05-24 20:24:02 +02:00
Michael Eischer b1266867d2 repository: wait max 1 minutes for lock removal if context is canceled 
The toplevel context in restic only canceled if the user interrupts a
restic operation. If the network connection has failed this can require
waiting the full retry duration of 15 minutes which is a bad user
experience for interactive usage. Thus limit the delay to one minute in
this case.
 2024-05-24 20:24:02 +02:00