Commit Graph

6381 Commits

Author SHA1 Message Date
MichaelEischer 98dcd0a887
Merge pull request #3874 from MichaelEischer/disk-wear-note
Add note that larger packs increase disk wear
2022-08-21 22:18:08 +02:00
Michael Eischer e0d6bf525c doc: fix typo 2022-08-21 19:12:19 +02:00
Michael Eischer 5d0523e2f1 Add note that larger packs increase disk wear 2022-08-21 19:12:19 +02:00
Michael Eischer c586a5e20f doc: recommend usage of B2's S3 API 2022-08-21 11:39:03 +02:00
Michael Eischer 623556bab6 b2: Increase list size to maximum
Just request as many files as possible in one call to reduce the number
of network roundtrips.
2022-08-21 11:20:03 +02:00
Michael Eischer de0162ea76 backend/retry: Overwrite failed uploads instead of deleting them
For backends which are able to atomically replace files, we just can
overwrite the old copy, if it is necessary to retry an upload. This has
the benefit of issuing one operation less and might be beneficial if a
backend storage, due to bugs or similar, could mix up the order of the
upload and delete calls.
2022-08-21 11:14:53 +02:00
Michael Eischer fc506f8538 b2: Repeat deleting until all file versions are removed
When hard deleting the latest file version on B2, this uncovers earlier
versions. If an upload required retries, multiple version might exist
for a file. Thus to reliably delete a file, we have to remove all
versions of it.
2022-08-21 11:11:00 +02:00
Michael Eischer cc4728d287 repository: Do not report ignored packs in EachByPack
Ignored packs were reported as an empty pack by EachByPack. The most
immediate effect of this is that the progress bar for rebuilding the
index reports processing more packs than actually exist.
2022-08-21 10:38:40 +02:00
Michael Eischer c3374b3ea5 helper: download modules as first step
There's no use in running that step in parallel.
2022-08-20 12:11:54 +02:00
Michael Eischer 7f0929e519 helper: Reduce number of parallel builds a bit
The go compiler is already parallelized. The high concurrency caused my
podman container to hit a resource limit.
2022-08-20 12:10:48 +02:00
Michael Eischer ed94678820 helper: cleanups 2022-08-20 12:10:29 +02:00
Michael Eischer e530d422a0 helper: don't setup cmd paths twice 2022-08-20 12:09:42 +02:00
Michael Eischer 7a992fc794 repository: Reduce buffer reallocations in ForAllIndexes
Previously the buffer was grown incrementally inside `repo.LoadUnpacked`.
But we can do better as we already know how large the index will be.
Allocate a bit more memory to increase the chance that the buffer can be
reused in the future.
2022-08-19 21:13:40 +02:00
Michael Eischer 77b1980d8e repository: MasterIndex.Packs: reduce allocations 2022-08-19 21:10:43 +02:00
Michael Eischer 6ff9517e45 repository: MasterIndex.ListPacks / Index.EachByPack allow earlier GC
Allow earlier garbage collection of some of the intermediate data
structures.
2022-08-19 21:06:33 +02:00
Michael Eischer ce902aac67 cache: Just try to open cache entry without calling stat first
Instead of first checking whether a file is in the repository cache and
then opening it, we just can open the file. This saves one stat call. If
the file is in the cache, everything is fine and otherwise the code
follows its normal fallback path.
2022-08-19 20:59:06 +02:00
Michael Eischer b6c86ababe doc: document aws session token 2022-08-19 20:41:15 +02:00
MichaelEischer 0d9ac78437
Merge pull request #3873 from MichaelEischer/gofmt-comments
gofmt comments
2022-08-19 19:54:30 +02:00
MichaelEischer 7e96a5af62
Merge pull request #3872 from MichaelEischer/fuse-fix
mount: Only remember successful snapshot refreshes
2022-08-19 19:21:29 +02:00
Michael Eischer f414db987d gofmt all files
Apparently the rules for comment formatting have changed with go 1.19.
2022-08-19 19:12:26 +02:00
Michael Eischer 522406b4f0 mount: Only remember successful snapshot refreshes
If the context provided by the fuse library is canceled before the index
was loaded this could lead to missing snapshots.
2022-08-19 19:07:07 +02:00
MichaelEischer dbca93da28
Merge pull request #3742 from MichaelEischer/from-repo
copy: replace --repo2 with --from-repo
2022-08-19 19:01:04 +02:00
MichaelEischer b4dfab002a
Merge pull request #3691 from greatroar/tag-filenames
Sanitize tags when used as filenames by restic mount
2022-08-19 18:33:44 +02:00
Michael Eischer 2758d76b77 copy: replace --repo2 with --from-repo
`init` and `copy` use `--repo2` with two different meaning which has
proven to be confusing for users. `--from-repo` now consistently marks a
source repository from which data is read. `--repo` is now always the
target/destination repository.
2022-08-19 18:33:26 +02:00
Michael Eischer af50fe9ac0 mount: Map slashes in tags to underscores
Suggested-by: greatroar <>
2022-08-19 18:17:57 +02:00
MichaelEischer 4cccffab58
Merge pull request #3862 from restic/3861-forget-invalid-policy
forget: Error when invalid unit is given in duration policy
2022-08-18 20:45:26 +02:00
Michael Eischer 2ea6c82cf6 comment cleanup
gofmt reformatted the comment
2022-08-18 20:15:38 +02:00
Michael Eischer bb27f7408c forget: Fail test if duration parsing error is missing 2022-08-18 20:14:09 +02:00
MichaelEischer c4b3a154ba
Merge pull request #3865 from restic/rawtaz-doc-comp-typo
doc: Fix typo in compression section
2022-08-18 19:37:44 +02:00
rawtaz 2cb2aa31cd
doc: Fix typo in compression section 2022-08-12 22:17:38 +02:00
Leo R. Lundgren 6f517858e8 forget: Error when invalid unit is given in duration policy 2022-08-10 13:37:26 +02:00
rawtaz f0bb4f8708
Merge pull request #3857 from restic/rawtaz-gcs-account
doc: Update links to GCS documentation
2022-08-08 22:52:30 +02:00
Michael Eischer 40c8755b13 doc: Update more links to GCS documentation 2022-08-08 20:00:44 +02:00
rawtaz f673068dbb
doc: Update link to GCS documentation
Updates the link to Google Cloud Storage documentation about creating a service account key.
2022-08-08 12:51:43 +02:00
rawtaz f26231c9e6
Merge pull request #3852 from MichaelEischer/snapshots-processed
stats: Add snapshots count to json output
2022-08-08 00:05:15 +02:00
MichaelEischer 80e3efffef
Merge pull request #3855 from mattxtaz/master
Fix typo with double percentage in help text
2022-08-07 22:44:36 +02:00
mattxtaz 01ab36336f Fix typo with double percentage in help text 2022-08-07 20:21:05 +01:00
Michael Eischer 6e92d852a8 stats: Add snapshots count to json output 2022-08-07 15:44:09 +02:00
MichaelEischer 9ad3ad5972
Merge pull request #3850 from lbausch/go1.19
Update tests to Go 1.19
2022-08-07 14:56:17 +02:00
MichaelEischer 2930a102de
Merge pull request #3731 from metalsp0rk/feature/min-packsize-flag
Feature: min packsize flag
2022-08-07 14:54:45 +02:00
MichaelEischer 8fa64a8f99
Merge pull request #3036 from greatroar/refactor-fuse
Clean up internal/fuse
2022-08-07 14:46:30 +02:00
Michael Eischer f3fdc66b32
restic: Use stable sorting in snapshot policy
sort.Sort is not guaranteed to be stable. Go 1.19 has changed the
sorting algorithm which resulted in changes of the sort order. When
comparing snapshots with identical timestamp but different paths and
tags lists, there is not meaningful order among them. So just keep their
order stable.
2022-08-07 14:10:40 +02:00
Lorenz Bausch b82f4824f0
Bump golangci-lint version 2022-08-07 14:10:39 +02:00
Lorenz Bausch 0b9b4c52ad
Update tests to Go 1.19 2022-08-07 14:10:39 +02:00
Michael Eischer 0b7291b8b2 mount: Fix parent inode used by snapshots dir 2022-08-07 13:03:32 +02:00
greatroar cfa80e2c6b mount: remove unused inode field from root node 2022-08-07 13:03:26 +02:00
MichaelEischer 74ae76036f
Merge pull request #2913 from aawsome/mount-snapshot-slashes
mount: Make snapshots dir structure customizable
2022-08-07 12:27:59 +02:00
MichaelEischer 09497aec02
Merge pull request #3826 from MichaelEischer/debug-log-for-release
Debug log for release build
2022-08-07 12:15:27 +02:00
Michael Eischer 83b4c50ee3 Mention --snapshot-template and --time-template in changelog 2022-08-07 12:13:06 +02:00
Michael Eischer caa17988a3 fuse: Redesign snapshot dirstruct
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
2022-08-07 12:13:06 +02:00