mirror
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Releases Wiki Activity
5,904 Commits 7 Branches 48 Tags 92 MiB
Commit Graph

1207 Commits

Author SHA1 Message Date
Daniel Gröber f31b4f29c1 Use config file modes to derive new dir/file modes 
Fixes #2351
 2022-04-30 15:59:51 +02:00
Michael Eischer 4b01b06f2f repository: Test compressed blobs in StreamPack 2022-04-30 11:34:10 +02:00
Michael Eischer bcab548617 pack: slightly expand testing of compressed blobs 2022-04-30 11:34:10 +02:00
Michael Eischer ec2b25565a repository: test uncompressedLength field and index example 2022-04-30 11:34:10 +02:00
Michael Eischer 9ffb8920f1 repository: run blackbox tests using old and new repo version 2022-04-30 11:34:10 +02:00
Michael Eischer abe5935693 repository: unify repository version-specific initialization 
Mark the master index as compressed also when initializing a new
repository. This is only relevant for testing.
 2022-04-30 11:34:10 +02:00
Alexander Neumann 8776031f96 Leave allocating slices to the decompress code 2022-04-30 11:34:10 +02:00
Alexander Neumann 5eb05a0afe Configure zstd encoder/decoder 2022-04-30 11:34:10 +02:00
Michael Eischer 2f36e044db Cleanup pack header check 2022-04-30 11:34:10 +02:00
Alexander Neumann 8b11b86383 Add option global --compression 2022-04-30 11:34:10 +02:00
Michael Eischer 7132df529e repository: Increase index size for repo version 2 
A compressed index is only about one third the size of an uncompressed
one. Thus increase the number of entries in an index to avoid cluttering
the repository with small indexes.
 2022-04-30 11:34:10 +02:00
Michael Eischer 66f9048bce repository: Alloc zstd encoder/decoder on demand 2022-04-30 11:34:10 +02:00
Michael Eischer fd05037e1a repository: recalibrate index batch allocation size 2022-04-30 11:34:10 +02:00
Michael Eischer 6fb408d90e repository: implement pack compression 2022-04-30 11:34:10 +02:00
Michael Eischer 362ab06023 init: Add flag to specify created repository version 2022-04-30 10:07:42 +02:00
Michael Eischer 4b957e7373 repository: Implement index/snapshot/lock compression 
The config file is not compressed as it should remain readable by older
restic versions such that these can return a proper error.

As the old format for unpacked data does not include a version header,
make use of a trick: The old data is always encoded as JSON. Thus it can
only start with '{' or '['. For any other value the first byte indicates
a versioned format. The version is set to 2 for now. Then the zstd
compressed data follows.
 2022-04-30 10:07:42 +02:00
Michael Eischer e597b99b55 repository: Reduce repack workers to prevent deadlock 
As repack streams packs these occupy one backend connection. Uploading a
new pack also requires a backend connection. To prevent a deadlock
during repack when reaching the backend connections limit, simply limit
the repackWorker count to always leave one connection for uploading.
 2022-04-23 11:28:18 +02:00
Michael Eischer ee627cd832 backend/mem: Actually enforce connection limit 
This will allow tests to detect deadlocks related to the connections
limit.
 2022-04-23 11:22:00 +02:00
Michael Eischer 4f97492d28 Backend: Expose connections parameter 2022-04-23 11:13:08 +02:00
Alexander Neumann 89ee1cf9ee
Merge pull request #3610 from MichaelEischer/windows-temp-files 
Improve handling of temporary files on windows
 2022-04-10 20:29:04 +02:00
Alexander Neumann a059ef90f8
Merge pull request #3702 from MichaelEischer/extend-config-error 
Print used key name if config fails to load
 2022-04-10 20:25:24 +02:00
Michael Eischer 4077a81b34 Add simple test for fs.TempFile on windows 2022-04-09 23:37:58 +02:00
Michael Eischer 9a3f1a9703 Simplify and comment TempFile implementation for windows 2022-04-09 23:37:58 +02:00
Michael Eischer c2aabb2686 Print used key name if config fails to load 2022-04-09 22:38:18 +02:00
MichaelEischer c60a5f00c9
Merge pull request #3675 from ItsMattL/update 
Refactor file handing for self-update.
 2022-04-09 21:55:56 +02:00
Matt LaPlante 0ba9d4ced7 Refactor file handing for self-update. 
* Write new file payload to a temp file before touching the original
binary. Minimizes the possibility of failing mid-write and corrupting
the binary.
* On Windows, move the original binary out to a temp file rather than
removing it as the running binary is locked. Fixes issue #2248.
 2022-04-09 21:40:33 +02:00
Alexander Neumann 04e054465a
Merge pull request #3475 from MichaelEischer/local-sftp-conn-limit 
Limit concurrent operations for local / sftp backend
 2022-04-09 21:33:00 +02:00
Michael Eischer 7b9ae91e04 copy: Load snapshots before indexes 2022-04-09 12:27:25 +02:00
Michael Eischer 9e12159230 Fix O(n) backend list calls in FindFilteredSnapshots 
When resolving snapshotIDs in FindFilteredSnapshots either
FindLatestSnapshot or FindSnapshot is called. Both operations issue a
list operation to the backend. When for example passing a long list of
snapshot ids to `forget` this could lead to a large number of list
operations.
 2022-04-09 12:26:31 +02:00
Michael Eischer 3d29083e60 copy/find/ls/recover/stats: Memorize snapshot listing before index 
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
 2022-04-09 12:26:30 +02:00
Michael Eischer ece06f125e sftp: Limit concurrent backend operations 2022-04-09 12:21:38 +02:00
Michael Eischer cd783358d3 local: Limit concurrent backend operations 
Use a limit of 2 similar to the filereader concurrency in the archiver.
 2022-04-09 12:21:38 +02:00
Michael Eischer 0b258cc054 backends: clean reader closing 2022-04-09 12:21:38 +02:00
Alex Duchesne 9e34c791c9 Better temp file cleanup on Windows. 2022-04-09 12:00:22 +02:00
Charlie Jiang d9c9415cfd Fix rclone (scoop shim) and sftp issue due to detached console on Windows 2022-04-03 17:53:17 +08:00
greatroar c23c0f7c14 Cast unix.Statfs_t.Type to int64 when checking for btrfs 
Fixes #3687. Uses the cast suggested by @MichaelEischer, except that the
contant isn't cast along, because it's untyped and will be converted by
the compiler as necessary.
 2022-03-31 22:30:45 +02:00
Alexander Neumann db8a958991
Merge pull request #3683 from MichaelEischer/fix-golangci-lint-warnings 
Fix golangci lint warnings
 2022-03-29 11:45:10 +02:00
Michael Eischer af31266b7d golangci-lint: replace deprecated golint with revive 2022-03-28 22:33:17 +02:00
Michael Eischer 2f81af6afa bloblru: Fix comment for New function 2022-03-28 22:25:25 +02:00
Michael Eischer 61e179ee78 switch to golang.org/x/term 2022-03-28 22:24:15 +02:00
Michael Eischer c60540b196 add go:build headers everywhere 2022-03-28 22:23:47 +02:00
Michael Eischer fefe9f5c0e pack: Hide more implementation details 2022-03-28 22:12:16 +02:00
Michael Eischer a773cb6527 pack: cleanup header size calculation 2022-03-28 22:09:49 +02:00
Michael Eischer 6408686973 repository: Simplify Blob equality check 2022-03-28 22:09:49 +02:00
Michael Eischer 243698680a crypto: Use helpers for size calculations 2022-03-28 22:09:49 +02:00
Michael Eischer d6db5a1fc2 archiver: Fix test 
The test relied on an undeocumented sideeffect of the LoadBlob implementation
 2022-03-28 22:09:49 +02:00
Michael Eischer f78bd14e28 repository: Remove pack implementation details from MasterIndex 2022-03-28 22:09:49 +02:00
Michael Eischer dc3d77dacc repository: make saveAndEncrypt private 2022-03-28 22:09:49 +02:00
Michael Eischer 6877e7edbb repository: Rename LoadAndDecrypt to LoadUnpacked 
The method is the complement for SaveUnpacked and not for
SaveAndEncrypt. The latter assembles blobs into pack files.
 2022-03-28 22:09:49 +02:00
Michael Eischer 2e1613d4c6 errors: Ensure that errors.IsFatal(errors.Fatal("err")) == true 
This fixes a few cases where restic output "Fatal: Fatal: [...]"
 2022-03-28 22:09:49 +02:00