Commit Graph

27 Commits

Author SHA1 Message Date
Michael Eischer a0cef9f247 limiter: move to internal/backend 2022-07-17 13:40:15 +02:00
Michael Eischer b0e64deb27 rclone: Fix timeout calculation 2021-11-07 17:49:33 +01:00
phcreery 43d173b042 rclone: add timeout option and documentation 2021-11-07 17:49:21 +01:00
greatroar 950b818274 rclone: Return one fewer value from run 2021-08-26 18:12:08 +02:00
greatroar fa3eed1998 Use rclone.wrappedConn by pointer
This shaves a kilobyte off the Linux binary by not generating a
non-pointer interface implementation.
2021-08-01 09:11:50 +02:00
Alexander Neumann f867e65bcd Fix issues reported by staticcheck 2021-01-30 20:43:53 +01:00
Alexander Neumann 3c753c071c errcheck: More error handling 2021-01-30 20:02:37 +01:00
Michael Eischer 37a5e2d681 rest: use global context on repository creation 2020-10-09 22:39:06 +02:00
Michael Eischer 8c36317b71 rclone: use configured number of connections during create 2020-09-19 19:11:43 +02:00
Michael Eischer 60a5c35de9 rclone: close connection to rclone if open fails 2020-09-19 19:11:43 +02:00
Michael Eischer c81b122374 rclone: Don't panic after unexpected subprocess exit
As the connection to the rclone child process is now closed after an
unexpected subprocess exit, later requests will cause the http2
transport to try to reestablish a new connection. As previously this never
should have happened, the connection called panic in that case. This
panic is now replaced with a simple error message, as it no longer
indicates an internal problem.
2020-08-01 12:17:40 +02:00
Michael Eischer 01b9581453 rclone: Better field names for stdio conn 2020-07-26 00:29:25 +02:00
Michael Eischer 3cd927d180 rclone: Give rclone time to finish before closing stdin pipe
Calling `Close()` on the rclone backend sometimes failed during test
execution with 'signal: Broken pipe'. The stdio connection closed both
the stdin and stdout file descriptors at the same moment, therefore
giving rclone no chance to properly send any final http2 data frames.

Now the stdin connection to rclone is closed first and will only be
forcefully closed after a timeout. In case rclone exits before the
timeout then the stdio connection will be closed normally.
2020-07-26 00:29:25 +02:00
Michael Eischer 8554332894 rclone: Close rclone side of stdio_conn pipes
restic did not notice when the rclone subprocess exited unexpectedly.

restic manually created pipes for stdin and stdout and used these for the
connection to the rclone subprocess. The process creating a pipe gets
file descriptors for the sender and receiver side of a pipe and passes
them on to the subprocess. The expected behavior would be that reads or
writes in the parent process fail / return once the child process dies
as a pipe would now just have a reader or writer but not both.

However, this never happened as restic kept the reader and writer
file descriptors of the pipes. `cmd.StdinPipe` and `cmd.StdoutPipe`
close the subprocess side of pipes once the child process was started
and close the parent process side of pipes once wait has finished. We
can't use these functions as we need access to the raw `os.File` so just
replicate that behavior.
2020-07-26 00:29:25 +02:00
greatroar 3e93b36ca4 Make rclone.New private 2020-07-26 00:28:45 +02:00
Alexander Neumann 3865b59716 rclone: Rework backend option parsing
This change allows passing no arguments to rclone, using `-o
rclone.args=""`. It is helpful when running rclone remotely via SSH
using a key with a forced command (via `command=` in `authorized_keys`).
2019-03-02 10:36:42 +01:00
Alexander Neumann 7ac683c360 rclone: Inject debug logger for HTTP 2018-10-21 19:58:40 +02:00
Alexander Neumann bfd923e81e rclone: Respect bandwith limits 2018-05-22 20:48:17 +02:00
Alexander Neumann 3f48e0e0f4 Add extra options to rclone
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann c43c94776b rclone: Make concurrent connections configurable 2018-04-01 10:18:38 +02:00
Alexander Neumann 0b776e63e7 backend/rclone: Request random file name
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann 737d93860a Extend first timeout to 60 seconds. 2018-04-01 10:18:38 +02:00
Alexander Neumann 4d5c7a8749 backend/rclone: Make sure rclone terminates 2018-04-01 10:18:38 +02:00
Alexander Neumann fc0295016a Address code review comments 2018-04-01 10:18:38 +02:00
Alexander Neumann 99b62c11b8 backend/rclone: Stop rclone in case of errors 2018-04-01 10:18:38 +02:00
Alexander Neumann 6d9a029e09 backend/rclone: Prefix all error messages 2018-04-01 10:18:38 +02:00
Alexander Neumann fe99340e40 Add rclone backend 2018-04-01 10:16:31 +02:00