Commit Graph

7053 Commits

Author SHA1 Message Date
Michael Eischer f903db492c backend: let ParseConfig return a Config pointer
In order to change the backend initialization in `global.go` to be able
to generically call cfg.ApplyEnvironment() for supported backends, the
`interface{}` returned by `ParseConfig` must contain a pointer to the
configuration.

An alternative would be to use reflection to convert the type from
`interface{}(Config)` to `interface{}(*Config)` (from value to pointer
type). However, this would just complicate the type mess further.
2023-06-07 22:31:15 +02:00
Michael Eischer 25a0be7f26 backend: fix linter warnings 2023-06-07 22:31:15 +02:00
Michael Eischer a27b7f1370 backend: use generic instead of any type for test suite 2023-06-07 22:31:14 +02:00
Michael Eischer fa361dbfbd backend: use generic implementation for ParseConfig tests 2023-06-07 22:30:35 +02:00
Michael Eischer 5260d38980 backend: let ParseConfig return concrete type 2023-06-07 22:30:35 +02:00
Michael Eischer 2f7b4ceae1 backend: Move environment based configuration into backend 2023-06-07 22:30:35 +02:00
Michael Eischer 49a6a4f5bf
Merge pull request #4356 from MichaelEischer/fix-unlock-hint
lock: fix missing hint to unlock command if repository is locked
2023-06-06 21:03:06 +02:00
Michael Eischer 237d00000e lock: fix missing hint to unlock command if repository is locked 2023-06-02 23:16:49 +02:00
dependabot[bot] 0c727f6ad1
Merge pull request #4355 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.56 2023-06-02 20:51:26 +00:00
Michael Eischer 4e7d3efad4
Merge pull request #4345 from DanielGibson/patch-1
restic restore docs: Mention that restic restore is fastest
2023-06-02 22:44:38 +02:00
Michael Eischer 17446da5fd
Merge pull request #4343 from greatroar/cache
cache: Restructure New to remove redundant operations
2023-06-02 22:41:17 +02:00
dependabot[bot] a3cee840d2
build(deps): bump github.com/minio/minio-go/v7 from 7.0.52 to 7.0.56
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.52 to 7.0.56.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.52...v7.0.56)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:35:42 +00:00
dependabot[bot] c76f3a1e27
Merge pull request #4348 from restic/dependabot/go_modules/github.com/spf13/cobra-1.7.0 2023-06-02 20:34:38 +00:00
dependabot[bot] 1e9714088d
Merge pull request #4347 from restic/dependabot/go_modules/golang.org/x/sync-0.2.0 2023-06-02 20:32:27 +00:00
Michael Eischer 58e3f5955c
Merge pull request #4350 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.6.0
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.4.0 to 1.6.0
2023-06-02 22:26:55 +02:00
dependabot[bot] dd8d8b1ae0
build(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:39 +00:00
dependabot[bot] e1ac0f0e0c
build(deps): bump golang.org/x/sync from 0.1.0 to 0.2.0
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.1.0 to 0.2.0.
- [Commits](https://github.com/golang/sync/compare/v0.1.0...v0.2.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:36 +00:00
Michael Eischer a88d90b8e5
Merge pull request #4346 from restic/dependabot/go_modules/golang.org/x/oauth2-0.8.0
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
2023-06-02 22:21:22 +02:00
Michael Eischer 2fcb3947df prune: trigger GC after prune planning 2023-06-02 21:57:40 +02:00
Michael Eischer eef0ee7a85 repository: trigger GC after loading the index
Loading the index requires some scratch space, thus make sure that this
memory does not factor into the targeted gc memory usage limit.
2023-06-02 21:56:14 +02:00
Michael Eischer f1b73c9301 Reduce GOGC to 50
The index used by restic consumes a major part of the total memory
usage. This leads to an unnecessarily large amount of memory that
contains ephemeral objects that are only used for a short time.
2023-06-02 21:51:50 +02:00
Michael Eischer ac1dfc99bb index: fix blocklist size 2023-06-02 19:39:12 +02:00
dependabot[bot] 379282299a
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) from 1.4.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/azcore/v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:59:16 +00:00
dependabot[bot] 0c796dbd9b
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:58:29 +00:00
Daniel Gibson f9dded83b3 doc/050_restore.rst: Mention that restic restore is fastest
Especially with lots of files (like when backing up git repos),
restic mount + cp -r can be really slow.
2023-05-31 22:44:36 +02:00
Michael Eischer 88a10a368f
Merge pull request #4342 from greatroar/errors
cmd: Don't check for errno == 0
2023-05-31 22:17:07 +02:00
Michael Eischer 9a7056a479 index: implement indexmap.grow() without random access 2023-05-30 20:13:33 +02:00
Michael Eischer fc05e35a08 index: let indexmap.Each iterate in allocation order
Iterating through the indexmap according to the bucket order has the
problem that all indexEntries are accessed in random order which is
rather cache inefficient.

As we already keep a list of all allocated blocks, just iterate through
it. This allows iterating through a batch of indexEntries without random
memory accesses. In addition, the packID will likely remain similar
across multiple blobs as all blobs of a pack file are added as a single
batch.
2023-05-30 20:12:36 +02:00
Michael Eischer f1c388c623 index: remove redundant storage of indexmap size 2023-05-30 20:11:53 +02:00
Michael Eischer 12141afbad index: Allow inlining of HAT 2023-05-30 20:11:14 +02:00
Michael Eischer fed33295c3 index: store indexEntries in hashed array tree
This data structure reduces the wasted memory to O(sqrt(n)). The
top-layer of the hashed array tree (HAT) also has a size of O(sqrt(n)),
which makes it cache efficient. The top-layer should be small enough to
easily fit into the CPU cache and thus only adds little overhead
compared to directly accessing an index entry via a pointer.
2023-05-29 00:24:15 +02:00
Michael Eischer b217f38ee7 index: Remove pointers from within indexentrys
The indexEntry objects are now allocated in a separate array. References
to an indexEntry are now stored as array indices. This has the benefit
of allowing the garbage collector to ignore the indexEntry objects as
these do not contain pointers and are part of a single large allocation.
2023-05-29 00:24:15 +02:00
Michael Eischer 0c1240360d index: add garbage collection benchmark
Allocates an index and repeatedly triggers the GC.
2023-05-29 00:23:04 +02:00
Michael Eischer ffca602315 repository: Fix panic in benchmarkLoadIndex 2023-05-28 23:55:47 +02:00
greatroar da419be43c cache: Restructure New to remove redundant operations
New and its helpers used to create the cache directories several times
over. They now only do so once. The added test ensures that the cache is
produced in a consistent state when parts are deleted.
2023-05-27 10:32:08 +02:00
greatroar 8f1ca8fabe cmd: Don't check for errno == 0
Maybe term.Restore's predecessor could return this, but the current
implementation never does.
2023-05-27 08:52:37 +02:00
Michael Eischer f74dad2afb
Merge pull request #4340 from restic/least-privilege-ci
CI: least privilege for workflows
2023-05-26 20:29:05 +02:00
Michael Eischer 3e287afdbf
Merge pull request #4226 from Sohalt/gcs-region
gs: support other regions than us
2023-05-26 20:28:52 +02:00
Michael Eischer 06894484a1 shorten changelog title 2023-05-26 20:05:49 +02:00
Michael Eischer 6e5b42d5c4 CI: least privilege for workflows 2023-05-26 19:58:02 +02:00
sohalt ed5b2c2c9b gs: support other regions than us 2023-05-26 19:54:42 +02:00
Michael Eischer 13c32b0fbe
Merge pull request #4328 from MichaelEischer/cleanup-integration-tests
Cleanup integration tests
2023-05-26 19:49:16 +02:00
Michael Eischer 9747cef338 fix linter warnings 2023-05-26 19:38:44 +02:00
Michael Eischer 8e913e6d3a repair index: always read Quiet flags from GlobalOptions passed as parameter 2023-05-26 19:38:44 +02:00
Michael Eischer b93459cbb0 repair snapshots: use local copy of globalOptions to open repository 2023-05-26 19:38:44 +02:00
Michael Eischer 692f81ede8 cleanup prune integration test 2023-05-26 19:38:44 +02:00
Michael Eischer 7a268e4aba always access stdout/stderr via globalOptions 2023-05-26 19:38:44 +02:00
Michael Eischer 4b3a0b4104 read JSON/Quiet flag from the passed in globalOptions 2023-05-26 19:38:44 +02:00
Michael Eischer cebce52c16 test: add helper to capture stdout for integration tests 2023-05-26 19:38:44 +02:00
Michael Eischer 675a49a95b Restructure integration tests
The tests are now split into individual files for each command. The
separation isn't perfect as many tests make use of multiple commands. In
particular `init`, `backup`, `check` and `list` are used by a larger
number of test cases.

Most tests now reside in files name cmd_<name>_integration_test.go. This
provides a certain indication which commands have significant test
coverage.
2023-05-26 19:38:43 +02:00