Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
/*
|
2022-01-20 18:27:56 +00:00
|
|
|
* This file copyright Transmission authors and contributors.
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
*
|
|
|
|
* or any future license endorsed by Mnemosyne LLC.
|
|
|
|
*
|
|
|
|
*/
|
2022-01-20 18:27:56 +00:00
|
|
|
// It may be used under GPLv2 (SPDX: GPL-2.0), GPLv3 (SPDX: GPL-3.0),
|
|
|
|
// or any future license endorsed by Mnemosyne LLC.
|
|
|
|
// License text can be found in the licenses/ folder.
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
2021-09-19 20:41:35 +00:00
|
|
|
#include <algorithm>
|
|
|
|
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
#include "transmission.h"
|
|
|
|
#include "error.h"
|
|
|
|
#include "file.h"
|
|
|
|
|
|
|
|
#include "test-fixtures.h"
|
|
|
|
|
|
|
|
namespace libtransmission
|
|
|
|
{
|
|
|
|
|
|
|
|
namespace test
|
|
|
|
{
|
|
|
|
|
|
|
|
class CopyTest : public SandboxedTest
|
|
|
|
{
|
|
|
|
protected:
|
|
|
|
void testImpl(char const* filename1, char const* filename2, size_t const file_length)
|
|
|
|
{
|
2021-11-13 00:10:04 +00:00
|
|
|
auto const path1 = tr_strvPath(sandboxDir(), filename1);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
/* Create a file. */
|
|
|
|
char* file_content = static_cast<char*>(tr_malloc(file_length));
|
|
|
|
tr_rand_buffer(file_content, file_length);
|
|
|
|
createFileWithContents(path1, file_content, file_length);
|
|
|
|
tr_free(file_content);
|
|
|
|
|
2021-11-13 00:10:04 +00:00
|
|
|
auto const path2 = tr_strvPath(sandboxDir(), filename2);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
tr_error* err = nullptr;
|
|
|
|
/* Copy it. */
|
2021-11-13 00:10:04 +00:00
|
|
|
EXPECT_TRUE(tr_sys_path_copy(path1.c_str(), path2.c_str(), &err));
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
EXPECT_EQ(nullptr, err);
|
|
|
|
tr_error_clear(&err);
|
|
|
|
|
2021-11-13 00:10:04 +00:00
|
|
|
EXPECT_TRUE(filesAreIdentical(path1.c_str(), path2.c_str()));
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
/* Dispose of those files that we created. */
|
2021-11-13 00:10:04 +00:00
|
|
|
tr_sys_path_remove(path1.c_str(), nullptr);
|
|
|
|
tr_sys_path_remove(path2.c_str(), nullptr);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
uint64_t fillBufferFromFd(tr_sys_file_t fd, uint64_t bytes_remaining, char* buf, size_t buf_len)
|
|
|
|
{
|
|
|
|
memset(buf, 0, buf_len);
|
|
|
|
|
|
|
|
size_t buf_pos = 0;
|
|
|
|
while (buf_pos < buf_len && bytes_remaining > 0)
|
|
|
|
{
|
2021-09-19 20:41:35 +00:00
|
|
|
uint64_t const chunk_size = std::min(uint64_t{ buf_len - buf_pos }, bytes_remaining);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
uint64_t bytes_read = 0;
|
|
|
|
|
|
|
|
tr_sys_file_read(fd, buf + buf_pos, chunk_size, &bytes_read, nullptr);
|
|
|
|
|
|
|
|
EXPECT_LE(buf_pos + bytes_read, buf_len);
|
|
|
|
EXPECT_LE(bytes_read, bytes_remaining);
|
|
|
|
buf_pos += bytes_read;
|
|
|
|
bytes_remaining -= bytes_read;
|
|
|
|
}
|
|
|
|
|
|
|
|
return bytes_remaining;
|
|
|
|
}
|
|
|
|
|
|
|
|
bool filesAreIdentical(char const* fn1, char const* fn2)
|
|
|
|
{
|
2021-10-01 20:28:01 +00:00
|
|
|
bool identical = true;
|
|
|
|
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
tr_sys_file_t fd1 = tr_sys_file_open(fn1, TR_SYS_FILE_READ | TR_SYS_FILE_SEQUENTIAL, 0, nullptr);
|
|
|
|
tr_sys_file_t fd2 = tr_sys_file_open(fn2, TR_SYS_FILE_READ | TR_SYS_FILE_SEQUENTIAL, 0, nullptr);
|
|
|
|
EXPECT_NE(fd1, TR_BAD_SYS_FILE);
|
|
|
|
EXPECT_NE(fd2, TR_BAD_SYS_FILE);
|
|
|
|
|
|
|
|
tr_sys_path_info info1;
|
|
|
|
tr_sys_path_info info2;
|
|
|
|
tr_sys_file_get_info(fd1, &info1, nullptr);
|
|
|
|
tr_sys_file_get_info(fd2, &info2, nullptr);
|
|
|
|
EXPECT_EQ(info1.size, info2.size);
|
|
|
|
|
|
|
|
uint64_t bytes_left1 = info1.size;
|
|
|
|
uint64_t bytes_left2 = info2.size;
|
|
|
|
|
|
|
|
size_t const buflen = 2 * 1024 * 1024; /* 2 MiB buffer */
|
|
|
|
char* readbuf1 = static_cast<char*>(tr_malloc(buflen));
|
|
|
|
char* readbuf2 = static_cast<char*>(tr_malloc(buflen));
|
|
|
|
|
|
|
|
while (bytes_left1 > 0 || bytes_left2 > 0)
|
|
|
|
{
|
|
|
|
bytes_left1 = fillBufferFromFd(fd1, bytes_left1, readbuf1, buflen);
|
|
|
|
bytes_left2 = fillBufferFromFd(fd2, bytes_left2, readbuf2, buflen);
|
|
|
|
|
|
|
|
if (bytes_left1 != bytes_left2)
|
|
|
|
{
|
2021-10-01 20:28:01 +00:00
|
|
|
identical = false;
|
|
|
|
break;
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (memcmp(readbuf1, readbuf2, buflen) != 0)
|
|
|
|
{
|
2021-10-01 20:28:01 +00:00
|
|
|
identical = false;
|
|
|
|
break;
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
tr_free(readbuf1);
|
|
|
|
tr_free(readbuf2);
|
|
|
|
tr_sys_file_close(fd1, nullptr);
|
|
|
|
tr_sys_file_close(fd2, nullptr);
|
|
|
|
|
2021-10-01 20:28:01 +00:00
|
|
|
return identical;
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
TEST_F(CopyTest, copy)
|
|
|
|
{
|
|
|
|
char const* filename1 = "orig-blob.txt";
|
|
|
|
char const* filename2 = "copy-blob.txt";
|
|
|
|
auto const random_file_length = 1024 * 1024 * 10;
|
|
|
|
|
|
|
|
testImpl(filename1, filename2, random_file_length);
|
|
|
|
}
|
|
|
|
|
|
|
|
} // namespace test
|
|
|
|
|
|
|
|
} // namespace libtransmission
|