Commit Graph

5788 Commits

Author SHA1 Message Date
MichaelEischer a29777f467
Merge pull request #3501 from greatroar/printprogress
Streamline progress printing in cmd/restic
2021-09-03 23:34:36 +02:00
Alexander Weiss bce87922c0 Improve recover
- only save directories not referenced by any snapshot
- dont't write empty snapshot
- use progress bar
2021-09-03 21:36:57 +02:00
Alexander Weiss 81876d5c1b Simplify cache logic 2021-09-03 21:01:00 +02:00
greatroar 7f0aa49f45 cmd/restic: Streamline progress printing
* PrintProgress no longer does unnecessary Sprintf calls, and performs
  fewer allocations in general
* newProgressMax's callback checks whether the terminal supports
  line updates once instead of once per call
* the callback looks up the terminal width once per call instead of
  twice (on Windows)
* the status shortening now uses the Unicode-aware version from
  internal/ui/termstatus (future-proofing)
2021-09-03 11:48:22 +02:00
greatroar 5aaa3e93c1 internal/ui/termstatus: Optimize and publish Truncate
name               old time/op  new time/op  delta
TruncateASCII-8     347ns ± 1%    69ns ± 1%  -80.02%  (p=0.000 n=9+10)
TruncateUnicode-8   447ns ± 3%   348ns ± 1%  -22.04%  (p=0.000 n=10+10)
2021-09-03 11:48:22 +02:00
MichaelEischer ec2e3b260e
Merge pull request #3499 from greatroar/wrappedconn-pointer
rclone: Return one fewer value from run
2021-08-31 21:48:36 +02:00
rawtaz 26914abe62
doc: Add note about scheduling to backup section
Explains that restic doesn't have built-in scheduling
and mentions a few keywords one can search for.
2021-08-29 22:03:22 +02:00
greatroar 950b818274 rclone: Return one fewer value from run 2021-08-26 18:12:08 +02:00
Juergen Hoetzel defe19fdf6 Quote snapshot prefix in error string 2021-08-25 16:11:28 +02:00
Juergen Hoetzel 409e4936af Improve snapshot filter warning message
Include the root-cause why the snapshot prefix is ignored.
2021-08-25 15:46:21 +02:00
MichaelEischer 10b39d7591
Merge pull request #3487 from andreaso/rest-proto-readme-link
Fix REST protocol link in README file
2021-08-22 18:47:32 +02:00
Michael Eischer 194ed19557 Add changelog 2021-08-22 18:29:58 +02:00
Michael Eischer 877fc9f352 rebuild-index: test that invalid indexes are skipped and removed 2021-08-22 18:24:19 +02:00
Michael Eischer 64258a2c2a rebuild-index: Ignore broken index files
Previously, an e.g. truncated index could prevent rebuild-index from
working.
2021-08-22 18:23:47 +02:00
Andreas Olsson c520672982
Fix REST protocol link in README file
1. All other document links goes to the _Read the Docs_ site
2. The GitHub reStructuredText renderer doesn't appear to do includes,
   making for a rather empty read.
2021-08-22 13:16:51 +02:00
Alexander Neumann 9374c3ce81
Merge pull request #3485 from lbausch/fix_typo
Fix typo in changelog
2021-08-21 13:16:20 +02:00
Lorenz Bausch 4d56b34096
Fix typo in changelog 2021-08-21 12:16:04 +02:00
Alexander Neumann 66382b2861 Update Go to 1.17 2021-08-17 21:38:34 +02:00
Michael Eischer 1fab5892b5 Add changelog for #3429 2021-08-16 17:42:16 +02:00
MichaelEischer c898f7a6bf
Merge pull request #3479 from BUFU1610/patch-1
fixed --keep-within-X options in example
2021-08-15 20:06:26 +02:00
BUFU 7659790923 fixed --keep-within-X options in example
The order of the words was wrong in the example, i.e. fixed to --keep-within-hourly from --keep-hourly-within.
2021-08-15 19:56:22 +02:00
MichaelEischer ecf34783ef
Merge pull request #3480 from MichaelEischer/fix-rest-tests
Fix rest backend tests
2021-08-15 18:59:02 +02:00
Michael Eischer 68370feeee backends: Remove TestSaveFilenames test
Filenames are expected to match the sha256 sum of the file content. This
rule is now enforced by the rest server thus making this test useless.
2021-08-15 18:24:16 +02:00
Michael Eischer 574c83e47f rest: Fix test to use paths which are the sha256 sum of the data 2021-08-15 18:19:43 +02:00
Michael Eischer e6a5801155 rest: Fix test backend url
The rest config normally uses prepareURL to sanitize URLs and ensures
that the URL ends with a slash. However, the test used an URL without a
trailing slash, which after the rest server changes causes test
failures.
2021-08-15 18:16:17 +02:00
Alexander Neumann d90efd7704 Fix test 2021-08-09 10:30:10 +02:00
Alexander Neumann 9fe5a87785
Merge pull request #3429 from MichaelEischer/safe-key-change
key: Check that new key works before deleting the old one
2021-08-09 10:07:15 +02:00
Alexander Neumann 7f1608dc77
Merge pull request #3246 from restic/content-hash-for-upload
Calculate content hashes during upload
2021-08-08 17:24:08 +02:00
Michael Eischer f4c5dec05d backend: test that a wrong hash fails an upload 2021-08-04 22:17:46 +02:00
Michael Eischer 7c1903e1ee panic if hash returns an error
Add a sanity check that the interface contract is honoured.
2021-08-04 22:17:46 +02:00
Michael Eischer 51b7e3119b mem: calculate md5 content hash for uploads
The mem backend is primarily used for testing. This ensures that the
upload hash calculation gets appropriate test coverage.
2021-08-04 22:17:46 +02:00
Michael Eischer a009b39e4c gs/swift: calculate md5 content hash for upload 2021-08-04 22:17:46 +02:00
Michael Eischer 1d3e99f475 azure: check upload using md5 content hash
For files below 256MB this uses the md5 hash calculated while assembling
the pack file. For larger files the hash for each 100MB part is
calculated on the fly. That hash is also reused as temporary filename.
As restic only uploads encrypted data which includes among others a
random initialization vector, the file hash shouldn't be susceptible to
md5 collision attacks (even though the algorithm is broken).
2021-08-04 22:17:46 +02:00
Michael Eischer 9aa2eff384 Add plumbing to calculate backend specific file hash for upload
This enables the backends to request the calculation of a
backend-specific hash. For the currently supported backends this will
always be MD5. The hash calculation happens as early as possible, for
pack files this is during assembly of the pack file. That way the hash
would even capture corruptions of the temporary pack file on disk.
2021-08-04 22:17:46 +02:00
Michael Eischer ee2f14eaf0 s3: enable content hash calculation for uploads 2021-08-04 22:12:12 +02:00
MichaelEischer 553ea36ca6
Merge pull request #2838 from greatroar/cache-conflicts
Make cache concurrency-safe
2021-08-04 22:11:50 +02:00
greatroar 6586e90acf Modernize internal/cache error handling 2021-08-04 22:02:42 +02:00
greatroar ea04f40eb3 Save cached files to a temporary location first 2021-08-04 22:02:42 +02:00
greatroar f9b6f8fd45 Replace duplicate type checking in cache with a function 2021-08-04 22:02:42 +02:00
MichaelEischer 1b1a2115fa
Merge pull request #3436 from greatroar/local-save-tmp
Save files under temporary name in local backend
2021-08-04 22:01:53 +02:00
Michael Eischer 65908647e3 Add changelog 2021-08-04 21:51:53 +02:00
greatroar 81e2499d19 Sync directory to get durable writes in local backend 2021-08-04 21:51:53 +02:00
greatroar 195a5cf996 Save files under temporary name in local backend
Fixes #3435.
2021-08-04 21:51:53 +02:00
MichaelEischer bc97a3d1f9
Merge pull request #3300 from aawsome/backup-dryrun
backup: add --dry-run/-n flag
2021-08-04 21:50:38 +02:00
Michael Eischer 702cff636f Add use case to changelog 2021-08-04 21:19:29 +02:00
Alexander Weiss 780e11b7e2 Adapt changelog 2021-08-04 21:19:29 +02:00
erin 4126435663 resolve rawtaz's review comments
make majority of suggestions from review by @rawtaz verbatim, with one clarification on my part in changelog
2021-08-04 21:19:29 +02:00
Alexander Weiss d107a2cfdf Separate dry run tests 2021-08-04 21:19:29 +02:00
Alexander Weiss 38a8a48a25 Simplify dry run backend 2021-08-04 21:19:29 +02:00
Ryan Hitchman 77bf148460 backup: add --dry-run/-n flag to show what would happen.
This can be used to check how large a backup is or validate exclusions.
It does not actually write any data to the underlying backend. This is
implemented as a simple overlay backend that accepts writes without
forwarding them, passes through reads, and generally does the minimal
necessary to pretend that progress is actually happening.

Fixes #1542

Example usage:

$ restic -vv --dry-run . | grep add
new       /changelog/unreleased/issue-1542, saved in 0.000s (350 B added)
modified  /cmd/restic/cmd_backup.go, saved in 0.000s (16.543 KiB added)
modified  /cmd/restic/global.go, saved in 0.000s (0 B added)
new       /internal/backend/dry/dry_backend_test.go, saved in 0.000s (3.866 KiB added)
new       /internal/backend/dry/dry_backend.go, saved in 0.000s (3.744 KiB added)
modified  /internal/backend/test/tests.go, saved in 0.000s (0 B added)
modified  /internal/repository/repository.go, saved in 0.000s (20.707 KiB added)
modified  /internal/ui/backup.go, saved in 0.000s (9.110 KiB added)
modified  /internal/ui/jsonstatus/status.go, saved in 0.001s (11.055 KiB added)
modified  /restic, saved in 0.131s (25.542 MiB added)
Would add to the repo: 25.892 MiB
2021-08-04 21:19:29 +02:00