Commit Graph

1036 Commits

Author SHA1 Message Date
MichaelEischer ad4f4dbc7a
Merge pull request #3645 from greatroar/stdin-parent
Don't set a parent for --stdin backups
2022-02-19 11:36:51 +01:00
greatroar 63f6a9b085 Don't set a parent for --stdin backups
Loading any parent tree for these only wastes time and memory.
Fixes #3641, where it was shown that the most recent tree will get
picked.

--parent is now implicitly ignored when --stdin is given.
2022-02-19 10:41:33 +01:00
MichaelEischer fb4c5af5c4
Merge pull request #3642 from gco/master
Fix test failures on Solaris
2022-02-12 22:07:14 +01:00
MichaelEischer d8f58fb7bf
Merge pull request #3592 from vgt/jsondiff
Add json output for diff command
2022-02-07 21:33:30 +01:00
Michael Eischer 74f29ad09b diff: add basic test for json output format 2022-02-06 11:44:15 +01:00
Michael Eischer 5f34ad523f diff: fix test failure and add remark on quiet to changelog 2022-02-05 23:02:07 +01:00
Michael Eischer 8ae4d86a84 rename snapshot timestamp filter variable 2022-02-05 22:42:38 +01:00
Greg 2e9180638e Fix test failures on Solaris
Add exceptions for symlinks, sticky bits, and device nodes in the same places where the BSDSs and/or Darwin have them.
2022-01-25 18:05:56 -08:00
Aneesh Agrawal 058dfc20da Avoid choosing parent snapshot newer than time of current snapshot
Currently, `restic backup` (if a `--parent` is not provided)
will choose the most recent matching snapshot as the parent snapshot.
This makes sense in the usual case,
where we tag the snapshot-being-created with the current time.

However, this doesn't make sense if the user has passed `--time`
and is currently creating a snapshot older than the latest snapshot.
Instead, choose the most recent snapshot
which is not newer than the snapshot-being-created's timestamp,
to avoid any time travel.

Impetus for this change:
I'm using restic for the first time!
I have a number of existing BTRFS snapshots
I am backing up via restic to serve as my initial set of backups.
I initially `restic backup`'d the most recent snapshot to test,
then started backing up each of the other snapshots.
I noticed in `restic cat snapshot <id>` output
that all the remaining snapshots have the most recent as the parent.
2022-01-23 23:55:00 -05:00
David Vogt 77c850148a Add json output for diff command 2022-01-23 23:22:26 +01:00
MichaelEischer df89aa0087
Merge pull request #3623 from invine/issue-3464
Skip lock file creation on forget with --no-lock and --dry-run
2022-01-23 18:12:34 +01:00
Pavel Frolov 792523b28b [issue 3464] skip lock creation in case of dry-run 2022-01-23 06:44:41 +03:00
rawtaz abfbacf3d3
Merge pull request #3591 from MichaelEischer/prune-fix-max-repack
prune: Handle --max-repack-size=0 as expected
2022-01-13 03:53:20 +01:00
mattxtaz 6ff32ee4d3 Add missing colon in prune stats output and change padding to 14 chars to align the fields 2022-01-06 21:15:15 +00:00
MichaelEischer 2c3e5d943d
Merge pull request #3593 from DarkKirb/parallelize-restic-copy
Parallelize blob upload/download for restic copy
2021-12-29 22:31:54 +01:00
Charlotte 🦝 Delenk e2bb384a60
Parallelize blob upload/download for restic copy
Currently restic copy will copy each blob from every snapshot serially,
which has performance implications on high-latency backends such as b2.

This commit introduces 8x parallelism for blob downloads/uploads which
can improve restic copy operations up to 8x for repositories with many
small blobs on b2.

This commit also addresses the TODO comment in the copyTree function.

Related work:

A more thorough improvement of the restic copy performance can be found
in PR #3513
2021-12-29 18:59:09 +01:00
jtagcat f92130d878 logging: quiet 'removing n old cache dirs'
Closes #3595

Choosing to include `stdoutIsTerminal()` as:
 - all other instances with `!opts.JSON` do so
 - this likely will not affect anything, especially when autorun
 - this seems to not be a meaningful enough summary
     to include in auto-backup reports

JSON is still likely not guaranteed to work and this is a suboptimal
  solution to this. Ideally, #1804 should refactor all print statements,
  and define+document(+handle) when stdoutIsTerminal() should be used.
  Else, it may end up more inconsistent and bulky
  (duplicate lines, longer files).
2021-12-29 01:08:29 +02:00
rawtaz e6e51b84ac
Merge pull request #3605 from gum3ng/issue_3463
Improve clarity of error message in restic ls command
2021-12-27 21:45:23 +01:00
Gautam Menghani 017614c41a [#issue 3463] Improve clarity of error message in restic ls command 2021-12-27 22:42:27 +05:30
Michael Eischer 0cfdb82ea4 prune: Handle --max-repack-size=0 as expected
Previously the flag was ignored and `--max-repack-size=1` had to be
used.
2021-12-27 15:48:56 +01:00
Kyle Brennan 8eb83029a8
Make copy honor --no-lock 2021-12-03 09:50:28 -08:00
Manuel González 212b2f651f
Add file mode in symbolic notation to `ls --json`
This aligns `restic ls --json` with `restic find --json`, utilizing the same
naming.
2021-11-16 19:45:50 +02:00
Gurjeet Singh 34a6a24544 Use S3's proper product name, Amazon S3
Per Amazon's product page [1], S3 is officially called "Amazon S3". The
restic project uses the phrase "AWS S3" in some places. This patch
corrects the product name.

[1]:https://aws.amazon.com/s3/
2021-11-13 22:21:06 +01:00
MichaelEischer cb81ee9396
Merge pull request #3474 from kitone/fix-issue-3382
Honor RESTIC_CACHE_DIR environment variable
2021-11-07 17:57:54 +01:00
kitone 5904f80cfa restic cache should display the name of the cache without shortening it in the case of the restic check 2021-11-06 20:18:51 +01:00
Ben Boeckel 4d579c4387 check: wait for progress bar output
Further code will also output to the terminal and the bar's cursor
positioning causes its output to overlap with the remaining output in a
racy way.

Fixes: #3344
2021-11-06 15:05:09 -04:00
MichaelEischer 6c84ea1412
Merge pull request #3548 from gum3ng/issue_3490
Support for specifying amount of data in read-data-subset
2021-11-05 23:28:11 +01:00
MichaelEischer 78c7dd53ef
Merge pull request #3526 from greatroar/dump-refactor
Refactor internal/dump + concurrent load/write
2021-11-05 22:38:39 +01:00
Gautam Menghani 836fbb9133 [#issue 3490] Support for specifying file size in read-data-subset 2021-11-02 15:25:46 +05:30
greatroar c71729dfc4 Refactor internal/dump + concurrent load/write
Package internal/dump has been reworked so its API consists of a single
type Dumper that handles tar and zip formats. Tree loading and node
writing happen concurrently.
2021-11-01 23:01:55 +01:00
Leo R. Lundgren 711ceb0109 mount: Improve usage information when mounted 2021-11-01 20:59:20 +01:00
kitone 95eb859b54 Honor RESTIC_CACHE_DIR environment variable
Fix #3382: restic check doesn't obey the RESTIC_CACHE_DIR environment variable
2021-10-10 16:00:02 +02:00
jtagcat a7853057ab Change: selfupdate: 'writing restic to' as verbose
Running restic self-update --quiet no longer
prints "writing restic to /usr/local/bin/restic".

The only output printed with -q is failures or
"successfully updated restic to version 0.12.1"

https://github.com/restic/restic/pull/3535

fix test fail: changelog title can't end with `.`

shorten changelog title
2021-10-09 23:24:18 +03:00
Garry McNulty 708d7a2574 s3: Add warning if key ID or secret is empty
Also add debug message if no credential types are available.

Closes #2388
2021-10-06 23:13:40 +01:00
greatroar c892c0bab9 internal/restic: Don't allocate in Tree.Insert
name         old time/op    new time/op    delta
BuildTree-8    34.6µs ± 4%     7.0µs ± 3%  -79.68%  (p=0.000 n=18+19)

name         old alloc/op   new alloc/op   delta
BuildTree-8    34.0kB ± 0%     0.9kB ± 0%  -97.37%  (p=0.000 n=20+20)

name         old allocs/op  new allocs/op  delta
BuildTree-8       108 ± 0%         1 ± 0%  -99.07%  (p=0.000 n=20+20)
2021-09-26 18:08:48 +02:00
MichaelEischer 78dac2fd48
Merge pull request #3523 from greatroar/swift-v2
Upgrade ncw/swift to v2
2021-09-24 21:41:36 +02:00
MichaelEischer a5e103a212
Merge pull request #3522 from greatroar/dump-lru
Use LRU cache in restic dump
2021-09-24 20:33:58 +02:00
greatroar e7ec0453b1 Upgrade ncw/swift to v2 2021-09-24 19:08:37 +02:00
greatroar fe04d024c7 Use LRU cache in restic dump 2021-09-24 15:45:08 +02:00
Sam Lucidi 897d8e662c Add --insecure-tls flag to disable SSL cert verification
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
2021-09-21 10:52:40 -04:00
MichaelEischer 58efe21eca
Merge pull request #3264 from amozoss/upstream-master
Refactor backup progress
2021-09-19 14:54:42 +02:00
greatroar 92ae951ffa Report timing from restic restore --verify 2021-09-19 12:53:09 +02:00
Michael Eischer fbb0e6499a ui: consolidate backup ui in ui/backup package 2021-09-12 16:20:15 +02:00
Michael Eischer a144c986f2 backup: Reenable JSON status updates with redirected output
After the refactoring status updates were no longer printed in quiet
mode or when the output is not an interactive terminal. However, the
JSON output is often piped to e.g. another program. Thus, don't set the
update frequency to 0 in that case. The status updates are still
disabled for backup --quiet.

This also reduces the status update frequency to 60fps compared to a
potentially much higher value before the refactoring.
2021-09-12 15:26:40 +02:00
Michael Eischer d62bfed65d ui: move SetDryRun to ProgressReporter 2021-09-12 15:25:58 +02:00
Dan Willoughby 448419990c Refactor backup progress
Move the shared logic into the progress

Allows logic to be shared with forth coming restore status
2021-09-12 15:25:58 +02:00
Alexander Neumann 8fe122d675
Merge pull request #3488 from MichaelEischer/rebuild-broken-index
Fix `rebuild-index` for damaged index
2021-09-07 17:00:23 +02:00
MichaelEischer fa5ca8af81
Merge pull request #2856 from aawsome/remove-readahead
Simplify cache logic
2021-09-04 20:25:49 +02:00
MichaelEischer b45d88e124
Merge pull request #3496 from juergenhoetzel/imporove-snapshot-filter-warning-message
Improve snapshot filter warning message
2021-09-03 23:42:27 +02:00
MichaelEischer bc4cbd775b
Merge pull request #2880 from aawsome/enhance-recover
Improve recover command
2021-09-03 23:40:43 +02:00