Alexander Weiss
e3013271a6
Harmonize naming
2020-11-22 20:41:12 +01:00
Alexander Weiss
92bd448691
Make BlobHandle substruct of Blob
2020-11-22 20:41:10 +01:00
Alexander Weiss
ce5d630681
Add MasterIndex.PackSize()
2020-11-21 22:13:54 +01:00
Alexander Weiss
c3ddde9e7d
Return hdrSize in ListPack
2020-11-21 22:13:54 +01:00
Michael Eischer
1f43cac12d
check: Only track data blobs when unused blobs should be reported
...
This improves the memory usage of check a lot as it now only has to
track tree blobs when run using the default parameters.
2020-11-15 18:43:07 +01:00
Alexander Neumann
3c0c0c132b
Merge pull request #3006 from aawsome/new-rebuild-index
...
Reimplement rebuild-index and remove /internal/index
2020-11-15 17:48:43 +01:00
greatroar
ab2b7d7f9a
Decrease allocation rate in internal/pack
...
internal/repository benchmark results:
name old time/op new time/op delta
PackerManager-8 179ms ± 1% 181ms ± 1% +0.78% (p=0.009 n=10+10)
name old speed new speed delta
PackerManager-8 294MB/s ± 1% 292MB/s ± 1% -0.77% (p=0.009 n=10+10)
name old alloc/op new alloc/op delta
PackerManager-8 91.3kB ± 0% 72.2kB ± 0% -20.92% (p=0.000 n=9+7)
name old allocs/op new allocs/op delta
PackerManager-8 1.38k ± 0% 0.76k ± 0% -45.20% (p=0.000 n=10+7)
2020-11-15 16:51:47 +01:00
Alexander Weiss
30b6a0878a
Reimplement rebuild-index
2020-11-15 07:05:09 +01:00
Alexander Weiss
187c8fb259
Parallelize MasterIndex.Save()
2020-11-15 07:05:09 +01:00
Alexander Weiss
1ec628ddf5
Add extraObsolete to MasterIndex.Save
2020-11-15 07:05:09 +01:00
Alexander Weiss
5898cb341f
Use CreateIndexFromPacks() in test
2020-11-15 07:05:05 +01:00
Alexander Weiss
43732bb885
Add CreateIndexFromPacks()
2020-11-15 07:04:51 +01:00
greatroar
21b787a4d1
Stop Counters where they're constructed and started
2020-11-09 13:03:31 +01:00
greatroar
ddca699cd2
Replace restic.Progress with new progress.Counter
...
This fixes two race conditions while cleaning up the code.
2020-11-09 12:12:35 +01:00
Alexander Weiss
fd33030556
Use in-memory index to rebuild index in prune
2020-11-06 20:23:30 +01:00
Alexander Weiss
38cc4393f6
Add Masterindex.Save(); Add Index.Packs()
2020-11-06 20:23:30 +01:00
Alexander Weiss
aaf1c44362
Fix #3062
2020-11-05 17:05:42 +01:00
Alexander Neumann
ae5302c7a8
Add comment that keepBlobs is modified
2020-11-05 10:33:38 +01:00
Alexander Neumann
866a52ad4e
Remove unneeded seek
...
The file returned from DownloadAndHash() is already seeked to the start
of the file.
2020-11-05 10:31:49 +01:00
Alexander Neumann
a4507610a0
Fix typo
2020-11-05 10:31:49 +01:00
Alexander Neumann
7def2d8ea7
Use a non-constant seed
2020-11-05 10:31:49 +01:00
Alexander Neumann
ee0112ab3b
Clarify message about expected error
2020-11-05 10:31:49 +01:00
Michael Eischer
b373f164fe
prune: Parallelize repack command
2020-11-05 10:31:49 +01:00
Alexander Neumann
3ff37215df
Merge pull request #2935 from MichaelEischer/upgrade-minio
...
Upgrade minio SDK to version 7
2020-11-02 09:09:10 +01:00
Sergio Rubio
e708628cfd
Remove unused function
...
Not currently used, and it'd need to be added to the MasterIndex interface first.
2020-10-28 13:24:49 +01:00
Alexander Weiss
b44ecde8b0
Fix setting of ID in DecodeIndex
2020-10-17 09:12:58 +02:00
MichaelEischer
4ba237bb93
Merge pull request #3019 from greatroar/refactor-decodeindex
...
Refactor index decoding
2020-10-15 23:22:33 +02:00
greatroar
b27375f5ce
defer close(ch) outside repository.RunWorkers
2020-10-14 15:50:16 +02:00
greatroar
720e0ee0c7
if cond { return true }; return false => return cond
2020-10-13 20:56:43 +02:00
greatroar
27db3ec262
Refactor index decoding
...
Decoding old-format indices no longer requires loading and decrypting
twice.
2020-10-13 20:47:50 +02:00
Alexander Neumann
56883817d8
Merge pull request #2990 from MichaelEischer/fix-goreport-warnings
...
Fix some goreport warnings
2020-10-12 20:44:56 +02:00
Michael Eischer
e638b46a13
Embed context into ReaderAt
...
The io.Reader interface does not support contexts, such that it is
necessary to embed the context into the backendReaderAt struct. This has
the problem that a reader might suddenly stop working when it's
contained context is canceled. However, this is now problem here as the
reader instances never escape the calling function.
2020-10-09 22:39:07 +02:00
Michael Eischer
d6cfe857b7
pass proper context into MasterIndex.RebuildIndex
2020-10-09 22:39:07 +02:00
Michael Eischer
a449450021
init: pass proper context to master key generation
...
This is no change in behavior as a canceled context did later on cause
the config file creation to fail. Therefore this change just lets the
repository initialization fail a bit earlier.
2020-10-09 22:37:56 +02:00
Michael Eischer
c458e114d4
pass context to Find / FindSnapshot
...
This allows proper interruption of restic while it searches for
snapshots or key files.
2020-10-09 22:37:56 +02:00
Michael Eischer
45e9a55c62
Wire context into backend layout detection
2020-10-09 22:37:24 +02:00
Michael Eischer
eba5dd831f
Fix typos reported by misspell
2020-10-06 14:55:13 +02:00
Michael Eischer
f003410402
init: Add `--copy-chunker-params` option
...
This allows creating multiple repositories with identical chunker
parameters which is required for working deduplication when copying
snapshots between different repositories.
2020-09-19 16:53:05 +02:00
greatroar
23fcbb275a
Remove restic.Cache interface
...
It was used in one code path, which then asserted its concrete type as
*cache.Cache.
Privatised some of the interface methods.
2020-09-18 10:48:13 +02:00
Michael Eischer
c46edcd9d6
error strings should not end with punctuation
2020-09-05 10:07:17 +02:00
Michael Eischer
d0329cf3eb
Adjust comments to match name of exported methods
2020-09-05 10:07:16 +02:00
Michael Eischer
4784540f04
repository: Simplify worker group code
2020-09-05 10:07:16 +02:00
Michael Eischer
d9a80e07b9
repository: Simplify index age calculation
2020-09-05 10:07:16 +02:00
Michael Eischer
2f8335554c
Remove a few unused variables
2020-09-05 10:06:23 +02:00
Michael Eischer
f4b9544ab2
prune: Add test that repack aborts on wrong blob
2020-08-16 11:34:01 +02:00
Michael Eischer
367449dede
prune: Reduce memory allocations while repacking
...
The slicing operator `slice[low:high]` default to 0 for the lower bound and
len(slice) for the upper bound when either or both are not specified.
Fix the code to use `cap(slice)` to check for the slice capacity.
2020-08-16 11:34:01 +02:00
Michael Eischer
7042bafea5
prune: Abort repacking when a pack contains a wrong blob
...
If a blob in a pack file can be decrypted successfully but contains data
that results in a different hash than stated in the header pack, then
abort repacking. As both the pack header and the blob are
cryptographically verified this either means than a malicious entity
tampered with the backup or indicates hardware problems on the client.
prune should fail with an error in both cases.
2020-08-16 11:34:01 +02:00
aawsome
0fed6a8dfc
Use "pack file" instead of "data file" ( #2885 )
...
- changed variable names, especially changed DataFile into PackFile
- changed in some comments
- always use "pack file" in docu
2020-08-16 11:16:38 +02:00
MichaelEischer
eca0f0ad24
Merge pull request #2863 from aawsome/index-no-duplicates
...
Don't save exact duplicates in merged index
2020-08-08 18:24:14 +02:00
Alexander Weiss
b112533812
Don't save exact duplicates when merging indexes
2020-08-05 06:32:02 +02:00