Commit Graph

2918 Commits

Author SHA1 Message Date
Alexander Neumann eccc336319 Add entry to CHANGELOG 2017-09-26 14:18:37 +02:00
Alexander Neumann 7fe657ec71 Merge pull request #1282 from restic/rework-autogeneration
Rework generation of manpages and completion files
2017-09-26 14:16:41 +02:00
Alexander Neumann 77c07bfd19 Remove integration test for manpage 2017-09-26 13:16:55 +02:00
Alexander Neumann 4de938d97a Update manpages and auto-completion 2017-09-26 13:16:08 +02:00
Alexander Neumann dad1c87afe Rework generation of manpages and completion files
This commit removes the `manpages` and `autocomplet` commands and
replaces them with the more generic `generate` command. Also, zsh
completion file support was added.
2017-09-26 13:12:12 +02:00
Alexander Neumann 801dbb6d03 Merge pull request #1281 from prattmic/gcs_perms
gs: allow backend creation without storage.buckets.get
2017-09-26 09:38:33 +02:00
Michael Pratt fa0be82da8 gs: allow backend creation without storage.buckets.get
If the service account used with restic does not have the
storage.buckets.get permission (in the "Storage Admin" role), Create
cannot use Get to determine if the bucket is accessible.

Rather than always trying to create the bucket on Get error, gracefully
fall back to assuming the bucket is accessible. If it is, restic init
will complete successfully. If it is not, it will fail on a later call.

Here is what init looks like now in different cases.

Service account without "Storage Admin":

Bucket exists and is accessible (this is the case that didn't work
before):

$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
created restic backend c02e2edb67 at gs:this-bucket-does-exist:/

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

Bucket exists but is not accessible:

$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
create key in backend at gs:this-bucket-does-exist:/ failed:
service.Objects.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have
storage.objects.create access to object this-bucket-exists/keys/0fa714e695c8ecd58cb467cdeb04d36f3b710f883496a90f23cae0315daf0b93., forbidden

Bucket does not exist:

$ ./restic init -r gs:this-bucket-does-not-exist:/
create backend at gs:this-bucket-does-not-exist:/ failed:
service.Buckets.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to bucket this-bucket-does-not-exist., forbidden

Service account with "Storage Admin":

Bucket exists and is accessible: Same

Bucket exists but is not accessible: Same. Previously this would fail
when Create tried to create the bucket. Now it fails when trying to
create the keys.

Bucket does not exist:

$ ./restic init -r gs:this-bucket-does-not-exist:/
enter password for new backend:
enter password again:
created restic backend c3c48b481d at gs:this-bucket-does-not-exist:/

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
2017-09-25 22:25:51 -07:00
Alexander Neumann 7e8bc8d362 Merge pull request #1279 from restic/fix-eof-error
cache: Synchronize downloading
2017-09-25 16:39:20 +02:00
Alexander Neumann 0bb2a8e0d0 cache: Synchronize downloading
This commit adds code to synchronize downloading files to the cache.
Before, requests that came in for files currently downloading would fail
because the file was not completed in the cache. Now, the code waits
until the download is completed.

Closes #1278
2017-09-25 15:58:20 +02:00
Alexander Neumann 2e72b57f2f Correct debug message 2017-09-25 14:35:37 +02:00
Alexander Neumann bff1039e3a Add entry to CHANGELOG 2017-09-25 13:17:44 +02:00
Alexander Neumann 5a999cb77f Merge pull request #1040 from restic/add-cache
Add local cache
2017-09-25 13:13:07 +02:00
Alexander Neumann 3a2539e0ac doc: Update manpages 2017-09-24 23:13:04 +02:00
Alexander Neumann e262f35d0a cache: Auto-remove invalid files 2017-09-24 23:11:47 +02:00
Alexander Neumann 176bfa6529 backend: Improve ReadAt 2017-09-24 23:11:23 +02:00
Alexander Neumann 240c4cf2fd cache: In case of an error, fall back backend 2017-09-24 23:11:23 +02:00
Alexander Neumann db5ec5d876 repo: Automatically cache tree-only pack files 2017-09-24 23:11:23 +02:00
Alexander Neumann e1dfaf5d87 cache: Allow proactive caching of tree packs
This commit adds a function to the cache which can decide to proactively
load the complete pack file and store it in the cache. This is helpful
for pack files containing only tree blobs, as it is likely that the same
file is accessed again in the future.
2017-09-24 23:11:23 +02:00
Alexander Neumann 5436154f0d cache: Add PerformReadahead 2017-09-24 23:11:23 +02:00
Alexander Neumann 809e218d20 cache: Improve debug logs 2017-09-24 23:11:23 +02:00
Alexander Neumann 1eaad6cebb index: Add TreePacks() 2017-09-24 21:54:53 +02:00
Alexander Neumann 56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Alexander Neumann 3890a947ca Clear data files in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann e299272378 repo: Try cached pack files first 2017-09-24 21:54:53 +02:00
Alexander Neumann 70248bd05a repo: Clear indexes 2017-09-24 21:54:53 +02:00
Alexander Neumann 7a5fde8f5a repository: Save pack files for trees in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 62ba9f1950 check: Disable cache by default 2017-09-24 21:54:53 +02:00
Alexander Neumann 610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 58699e3c90 Write CACHEDIR.TAG to cache base directory 2017-09-24 21:54:53 +02:00
Alexander Neumann 9be24a1c9f Add cache
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00
Alexander Neumann 5ace41471e Merge pull request #1277 from prattmic/gcs_cleanup
Document GCS permission requirements
2017-09-24 20:49:02 +02:00
Michael Pratt 3b2106ed30 gs: document required permissions
In the manual, state which standard roles the service account must
have to work correctly, as well as the specific permissions required,
for creating even more specific custom roles.
2017-09-24 11:25:57 -07:00
Michael Pratt 5f4f997126 gs: minor comment cleanups
* Remove a reference to S3.
* Config can only be used for GCS, not other "gcs compatibile servers".
* Make comments complete sentences.
2017-09-24 10:10:56 -07:00
Alexander Neumann 49d397a419 Merge pull request #1275 from fawick/sort_snapshots
Always sort snapshots lists ascending by timestamp
2017-09-24 16:32:40 +02:00
Fabian Wickborn ea1ab96749 Always sort snapshots lists ascending by timestamp
Fixes #1219.
2017-09-24 13:01:13 +02:00
Alexander Neumann 24c62e719a Add entry to CHANGELOG 2017-09-23 22:15:10 +02:00
Alexander Neumann 9c6b7f688e Merge pull request #1270 from restic/sftp-allow-password-prompt
sftp: Allow password entry
2017-09-23 22:13:04 +02:00
Alexander Neumann d41dce5ecb Merge pull request #1272 from jniggemann/doc-faq-add-prio
doc: FAQ: Add info on IO and CPU prioritization
2017-09-23 20:08:17 +02:00
Jan Niggemann 52a3eafede doc: FAQ: Add info on IO and CPU prioritization 2017-09-23 19:32:07 +02:00
Alexander Neumann 55dfc85159 manual: Add hint for RESTIC_PASSWORD_FILE 2017-09-23 19:16:07 +02:00
Alexander Neumann a7a478a19e doc: Correct FAQ 2017-09-23 19:15:21 +02:00
Alexander Neumann 2080afd9de Merge pull request #1259 from jniggemann/doc-add-restic-check-advice
adds advice to run restic check regularly
2017-09-23 14:07:55 +02:00
Alexander Neumann 9aa136b982 Merge pull request #1260 from jniggemann/doc-add-info-on-special-items
Doc add info on special items
2017-09-23 14:07:20 +02:00
Alexander Neumann 3a191f37cb Add entry to CHANGELOG 2017-09-23 14:05:55 +02:00
Alexander Neumann 429106340f Merge pull request #1267 from harshavardhana/possible-fix-memory
Implement Size() and Len() to know the optimal size.
2017-09-23 14:04:15 +02:00
Alexander Neumann 530c73b457 Merge pull request #1269 from mrzv/forget-compact
Add --compact option to forget
2017-09-23 14:02:34 +02:00
Alexander Neumann fb9729fdb9 sftp: Allow password entry
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.

So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
2017-09-23 11:43:33 +02:00
Alexander Neumann 45a09c76ff Allow suspending SIGINT handler 2017-09-23 11:12:56 +02:00
Dmitriy Morozov efd65a1b65 Update manpage for forget 2017-09-22 16:35:58 -07:00
Dmitriy Morozov ae60188eb9 Add --compact option to forget 2017-09-22 16:32:59 -07:00