Commit Graph

2037 Commits

Author SHA1 Message Date
Alexander Neumann 8e16931949 Merge pull request #716 from zcalusic/rest-server-new-location
Rest server moved to https://github.com/restic/rest-server
2017-01-02 11:12:37 +01:00
Zlatko Čalušić 2267aca296 Rest server moved to https://github.com/restic/rest-server 2017-01-01 16:22:46 +01:00
Zlatko Čalušić c70bc7ed0b Document REST backend
Closes #644
2016-12-31 13:14:44 +01:00
Alexander Neumann 8e3b81c5ec Merge pull request #713 from restic/update-travis
Update .travis.yml
2016-12-30 17:21:27 +01:00
Alexander Neumann 30975f7116 Update appveyor configuration 2016-12-30 17:07:42 +01:00
Alexander Neumann 0ef463d56a Update .travis.yml 2016-12-30 15:21:49 +01:00
Alexander Neumann 5132f5bfe6 Merge pull request #709 from restic/fix-708
Make sure cleanup is executed before exiting
2016-12-28 18:28:07 +01:00
Alexander Neumann 80457018d7 Make sure cleanup is executed before exiting
Closes #708
2016-12-28 10:53:31 +01:00
Alexander Neumann b0997d05fb Merge pull request #704 from restic/remove-timestamp
Remove timestamp from `version` command
2016-12-19 22:22:43 +01:00
Alexander Neumann 3add2f0acb Merge pull request #703 from sjoerdsimons/master
Avoid duplicate backup paths
2016-12-19 22:21:18 +01:00
Alexander Neumann 166d1811a1 Remove timestamp from `version` command
This enables reproducible builds, for details see
https://reproducible-builds.org/docs/timestamps/
2016-12-19 21:14:12 +01:00
Sjoerd Simons e1fc455079 Avoid duplicate backup paths
Target directories from the from-files argument get added to the command
line args, after which all command line args were appended to the same
variable again causing duplicates. Split the used variables to avoid
this.

Signed-off-by: Sjoerd Simons <sjoerd@luon.net>
2016-12-18 23:23:57 +01:00
Alexander Neumann 98237bf942 Add VERSION file for 0.3.2 2016-12-18 18:53:03 +01:00
Alexander Neumann 75f21f23ff Merge pull request #700 from restic/debug-panic
Make sure SaveFile always returns a node
2016-12-14 21:29:04 +01:00
Alexander Neumann 9885aeac3b Make sure SaveFile always returns a node 2016-12-14 18:56:11 +01:00
Alexander Neumann 85c87b9ab9 Add VERSION file for 0.3.1 2016-12-13 21:36:22 +01:00
Alexander Neumann 51cd78e16c Merge pull request #691 from restic/fix-604
Correctly save modified files
2016-12-10 17:31:20 +01:00
Alexander Neumann e6a40af06d Treat changed files as a warning, not an error 2016-12-10 17:14:13 +01:00
Alexander Neumann 3fcbb4ac25 Use new Node if file has changed
Closes #604
2016-12-10 16:54:20 +01:00
Alexander Neumann 7d71bad4eb Test if modified files are correctly saved 2016-12-10 16:36:58 +01:00
Alexander Neumann dbdfed6343 Merge pull request #690 from zcalusic/master
Even if file changes size during backup, still save it
2016-12-10 12:36:56 +01:00
Alexander Neumann 5e48c1fadc Merge pull request #688 from restic/fix-686
Save snapshot after saving all pack files
2016-12-10 12:33:58 +01:00
Zlatko Čalušić deb6dd7f72 Even if file changes size during backup, still save it
Previously such files (typically log files) wouldn't be backed up at
all!

The proper behaviour is to backup what we can, and warn the operator
that file is possibly not complete. But it is a warning, not an error.

Closes #689
2016-12-10 12:24:45 +01:00
Alexander Neumann c265673c8e Save snapshot after saving all pack files
Closes #686
2016-12-10 11:49:09 +01:00
Alexander Neumann 0fceeb20f1 Merge pull request #685 from jannic/patch-1
Update debug message
2016-12-06 08:16:33 +01:00
Jan Niehusmann c5897e0d62 Update debug message
Since client.BucketExists was changed to return a separate 'found' value, instead of reporting an error when the bucket doesn't exist, the error code path does no longer imply a call to client.MakeBucket. So the second part of the debug message, "...trying to create the bucket" doesn't apply any more.
Also, changed the name of the return value from 'ok' to 'found', matching the API documentation at https://docs.minio.io/docs/golang-client-api-reference#BucketExists.
2016-12-05 23:12:30 +01:00
Alexander Neumann 8d13f22c50 Merge pull request #683 from jannic/pr1
Omit "archived as %v" messages in quiet mode.
2016-12-03 11:15:24 +01:00
Alexander Neumann 1815536534 Update build.go 2016-12-03 11:14:30 +01:00
Jan Niehusmann 9267c25aa0 Omit "archived as %v" messages in quiet mode. 2016-12-03 10:28:49 +01:00
Alexander Neumann 281cbbdf2e Merge pull request #682 from jpmens/patch-1
Small typo in dry-run of remove snapshot
2016-12-03 10:11:58 +01:00
JP Mens 5996d671a0 Small typo in dry-run of remove snapshot 2016-12-02 17:33:05 +01:00
Alexander Neumann ef9b974bcd Merge pull request #681 from zcalusic/master
Stop trying to detect Go version
2016-12-02 11:15:29 +01:00
Zlatko Čalušić 7e66b73ce0 Stop trying to detect Go version
It fails on pre-release versions, anyway.  It's enough to mention the oldest
supported version in README.md.  Anything older than two latest Go releases
is bad idea, anyway, 'cause it's unsupported by Go development team.

Closes #680
2016-12-01 20:06:23 +01:00
Alexander Neumann 505a2097ad Manual: Add note about s3 bucket locations 2016-11-27 20:18:57 +01:00
Alexander Neumann 07380878fb Merge pull request #678 from restic/fix-676
Update github.com/elithrar/simple-scrypt
2016-11-19 19:22:44 +01:00
Alexander Neumann 3b29ae3c99 Update github.com/elithrar/simple-scrypt
Closes #676
2016-11-19 17:13:13 +01:00
Alexander Neumann e5617b5fd1 Merge pull request #675 from restic/parent-check-hostname
Use the hostname filter to find a parent snasphot
2016-11-19 12:42:40 +01:00
Alexander Neumann 11f23ae663 Merge pull request #673 from Novex/restore-directory-metadata-for-existing-directories
Don't consider a pre-existing directory in the restore path to be a failure
2016-11-19 12:42:31 +01:00
Alexander Neumann 2828003d60 Test that existing files and dirs are restored 2016-11-15 21:41:41 +01:00
Alexander Neumann 16cef3b4c6 Use the hostname filter to find a parent snasphot
Closes #674
2016-11-15 21:04:51 +01:00
Alexander Neumann 699f39e3cf FindLatestSnapshot: Rename parameter to clarify meaning 2016-11-15 21:03:54 +01:00
Seb Patane 33b6a7381b Don't consider a pre-existing directory in the restore path to be a failure
* When a directory already exists, CreateDirAt returns an error stating so
  * This means that the restoreMetadata step is skipped, so for directories which already exist no file permissions, owners, groups, etc will be restored on them
* Not returning the error if it's a "directory exists" error means the metadata will get restored
  * It also removes the superfluous "error for ...: mkdir ...: file exists" messages
* This makes the behaviour of directories consistent with that of files (which always have their content & metadata restored, regardless of whether they existed or not)
2016-11-14 17:53:09 +10:00
Alexander Neumann 190673b24a Merge pull request #657 from AlexanderThaller/read_backup_files_from_file
Read files to backup from a file
2016-11-12 21:47:11 +01:00
Alexander Thaller b7b03dbd4a Added new flag to backup subcommand that reads the files to backup from a file 2016-11-12 15:45:32 +01:00
Alexander Neumann 56009dd16e Merge pull request #670 from restic/remove-fadvise
Remove fadvise
2016-11-10 23:42:21 +01:00
Alexander Neumann b56bde3f61 Remove fadvise
This commit removes the use of FADV_DONTNEED, which also purges active
cached pages for other processes.
2016-11-10 22:21:22 +01:00
Alexander Neumann b1ed74eb43 Merge pull request #669 from zcalusic/master
Fix REST backend HTTP keepalive
2016-11-10 21:05:14 +01:00
Zlatko Čalušić d8f0e7cbd1 Fix REST backend HTTP keepalive
This is subtle.  A combination od fast client disk (read: SSD) with lots
of files and fast network connection to restic-server would suddenly
start getting lots of "dial tcp: connect: cannot assign requested
address" errors during backup stage.  Further inspection revealed that
client machine was plagued with TCP sockets in TIME_WAIT state.  When
ephemeral port range was finally exhausted, no more sockets could be
opened, so restic would freak out.

To understand the magnitude of this problem, with ~18k ports and default
timeout of 60 seconds, it means more than 300 HTTP connections per
seconds were created and teared down.  Yeah, restic-server is that
fast. :)

As it turns out, this behavior was product of 2 subtle issues:

1) The body of HTTP response wasn't read completely with io.ReadFull()
   at the end of the Load() function.  This deactivated HTTP keepalive,
   so already open connections were not reused, but closed instead, and
   new ones opened for every new request.  io.Copy(ioutil.Discard,
   resp.Body) before resp.Body.Close() remedies this.

2) Even with the above fix, somehow having MaxIdleConnsPerHost at its
   default value of 2 wasn't enough to stop reconnecting.  It is hard to
   understand why this would be so detrimental, it could even be some
   subtle Go runtime bug.  Anyhow, setting this value to match the
   connection limit, as set by connLimit global variable, finally nails
   this ugly bug.

I fixed several other places where the response body wasn't read in
full (or at all).  For example, json.NewDecoder() is also known not to
read the whole body of response.

Unfortunately, this is not over yet. :( The check command is firing up
to 40 simultaneous connections to the restic-server.  Then, once again,
MaxIdleConnsPerHost is too low to support keepalive, and sockets in the
TIME_WAIT state pile up.  But, as this kind of concurrency absolutely
kill the poor disk on the server side, this is a completely different
bug then.
2016-11-10 09:32:07 +01:00
Alexander Neumann 5e721afb5d doc/mkdocs: Improve code hilighting
Additionally, refresh the restic sample output.
2016-11-08 20:23:39 +01:00
Alexander Neumann 149c01a86a Merge pull request #659 from restic/device-freebsd
fs.DeviceID(): Return errors whehn fi is nil
2016-11-05 13:35:16 +01:00