Commit Graph

2960 Commits

Author SHA1 Message Date
Alexander Neumann 0bb2a8e0d0 cache: Synchronize downloading
This commit adds code to synchronize downloading files to the cache.
Before, requests that came in for files currently downloading would fail
because the file was not completed in the cache. Now, the code waits
until the download is completed.

Closes #1278
2017-09-25 15:58:20 +02:00
Alexander Neumann 2e72b57f2f Correct debug message 2017-09-25 14:35:37 +02:00
Alexander Neumann bff1039e3a Add entry to CHANGELOG 2017-09-25 13:17:44 +02:00
Alexander Neumann 5a999cb77f Merge pull request #1040 from restic/add-cache
Add local cache
2017-09-25 13:13:07 +02:00
Alexander Neumann 3a2539e0ac doc: Update manpages 2017-09-24 23:13:04 +02:00
Alexander Neumann e262f35d0a cache: Auto-remove invalid files 2017-09-24 23:11:47 +02:00
Alexander Neumann 176bfa6529 backend: Improve ReadAt 2017-09-24 23:11:23 +02:00
Alexander Neumann 240c4cf2fd cache: In case of an error, fall back backend 2017-09-24 23:11:23 +02:00
Alexander Neumann db5ec5d876 repo: Automatically cache tree-only pack files 2017-09-24 23:11:23 +02:00
Alexander Neumann e1dfaf5d87 cache: Allow proactive caching of tree packs
This commit adds a function to the cache which can decide to proactively
load the complete pack file and store it in the cache. This is helpful
for pack files containing only tree blobs, as it is likely that the same
file is accessed again in the future.
2017-09-24 23:11:23 +02:00
Alexander Neumann 5436154f0d cache: Add PerformReadahead 2017-09-24 23:11:23 +02:00
Alexander Neumann 809e218d20 cache: Improve debug logs 2017-09-24 23:11:23 +02:00
Alexander Neumann 1eaad6cebb index: Add TreePacks() 2017-09-24 21:54:53 +02:00
Alexander Neumann 56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Alexander Neumann 3890a947ca Clear data files in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann e299272378 repo: Try cached pack files first 2017-09-24 21:54:53 +02:00
Alexander Neumann 70248bd05a repo: Clear indexes 2017-09-24 21:54:53 +02:00
Alexander Neumann 7a5fde8f5a repository: Save pack files for trees in cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 62ba9f1950 check: Disable cache by default 2017-09-24 21:54:53 +02:00
Alexander Neumann 610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 58699e3c90 Write CACHEDIR.TAG to cache base directory 2017-09-24 21:54:53 +02:00
Alexander Neumann 9be24a1c9f Add cache
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00
Alexander Neumann 5ace41471e Merge pull request #1277 from prattmic/gcs_cleanup
Document GCS permission requirements
2017-09-24 20:49:02 +02:00
Michael Pratt 3b2106ed30 gs: document required permissions
In the manual, state which standard roles the service account must
have to work correctly, as well as the specific permissions required,
for creating even more specific custom roles.
2017-09-24 11:25:57 -07:00
Michael Pratt 5f4f997126 gs: minor comment cleanups
* Remove a reference to S3.
* Config can only be used for GCS, not other "gcs compatibile servers".
* Make comments complete sentences.
2017-09-24 10:10:56 -07:00
Alexander Neumann 49d397a419 Merge pull request #1275 from fawick/sort_snapshots
Always sort snapshots lists ascending by timestamp
2017-09-24 16:32:40 +02:00
Fabian Wickborn ea1ab96749 Always sort snapshots lists ascending by timestamp
Fixes #1219.
2017-09-24 13:01:13 +02:00
Alexander Neumann 24c62e719a Add entry to CHANGELOG 2017-09-23 22:15:10 +02:00
Alexander Neumann 9c6b7f688e Merge pull request #1270 from restic/sftp-allow-password-prompt
sftp: Allow password entry
2017-09-23 22:13:04 +02:00
Alexander Neumann d41dce5ecb Merge pull request #1272 from jniggemann/doc-faq-add-prio
doc: FAQ: Add info on IO and CPU prioritization
2017-09-23 20:08:17 +02:00
Jan Niggemann 52a3eafede doc: FAQ: Add info on IO and CPU prioritization 2017-09-23 19:32:07 +02:00
Alexander Neumann 55dfc85159 manual: Add hint for RESTIC_PASSWORD_FILE 2017-09-23 19:16:07 +02:00
Alexander Neumann a7a478a19e doc: Correct FAQ 2017-09-23 19:15:21 +02:00
Alexander Neumann 2080afd9de Merge pull request #1259 from jniggemann/doc-add-restic-check-advice
adds advice to run restic check regularly
2017-09-23 14:07:55 +02:00
Alexander Neumann 9aa136b982 Merge pull request #1260 from jniggemann/doc-add-info-on-special-items
Doc add info on special items
2017-09-23 14:07:20 +02:00
Alexander Neumann 3a191f37cb Add entry to CHANGELOG 2017-09-23 14:05:55 +02:00
Alexander Neumann 429106340f Merge pull request #1267 from harshavardhana/possible-fix-memory
Implement Size() and Len() to know the optimal size.
2017-09-23 14:04:15 +02:00
Alexander Neumann 530c73b457 Merge pull request #1269 from mrzv/forget-compact
Add --compact option to forget
2017-09-23 14:02:34 +02:00
Alexander Neumann fb9729fdb9 sftp: Allow password entry
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.

So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
2017-09-23 11:43:33 +02:00
Alexander Neumann 45a09c76ff Allow suspending SIGINT handler 2017-09-23 11:12:56 +02:00
Dmitriy Morozov efd65a1b65 Update manpage for forget 2017-09-22 16:35:58 -07:00
Dmitriy Morozov ae60188eb9 Add --compact option to forget 2017-09-22 16:32:59 -07:00
Jan Niggemann 3b904525d9 manual: Add info on special items (device files) 2017-09-22 22:58:26 +02:00
Jan Niggemann 1e31f5202f manual: Add info on special items
Add info about handling of symlinks and bind-mounts.

Closes: #1014
2017-09-22 22:51:31 +02:00
Jan Niggemann f12d41138a Add advice to run check regularly 2017-09-22 22:27:10 +02:00
Harshavardhana 98369f6a5d Implement Size() and Len() to know the optimal size. 2017-09-22 12:09:17 -07:00
Alexander Neumann 8f9bf1995b Merge pull request #1265 from restic/improve-packers
Improve packers, prepare for cache
2017-09-22 16:16:10 +02:00
Alexander Neumann e7de3b5f9d Merge pull request #1266 from JaCoB1123/document_hostname_option
Add note about rescan to hostname flag (fixes #1221)
2017-09-22 15:40:24 +02:00
Alexander Neumann 3541d06d07 repo: Split packers for tree and data
The code now bundles tree blobs and data blobs into different pack
files, so we'll end up with pack files that either only contain data or
trees. This is in preparation to adding a cache (#1040), because
tree-only pack files can easily be cached later on.
2017-09-22 15:36:47 +02:00
Alexander Neumann db0e3cd772 repo: Remove packer limits
This commit simplifies finding a packer: The first open packer is taken,
and the upper limit for the pack file is removed.
2017-09-22 15:36:47 +02:00