2022-08-08 18:05:39 +00:00
|
|
|
// This file Copyright (C) 2013-2022 Mnemosyne LLC.
|
|
|
|
// It may be used under GPLv2 (SPDX: GPL-2.0-only), GPLv3 (SPDX: GPL-3.0-only),
|
2022-01-20 18:27:56 +00:00
|
|
|
// or any future license endorsed by Mnemosyne LLC.
|
|
|
|
// License text can be found in the licenses/ folder.
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
2021-09-19 20:41:35 +00:00
|
|
|
#include <algorithm>
|
2022-08-17 16:08:36 +00:00
|
|
|
#include <cstring>
|
2022-08-04 13:44:18 +00:00
|
|
|
#include <vector>
|
2021-09-19 20:41:35 +00:00
|
|
|
|
2023-01-02 16:23:51 +00:00
|
|
|
#include <libtransmission/transmission.h>
|
|
|
|
#include <libtransmission/error.h>
|
|
|
|
#include <libtransmission/file.h>
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
#include "test-fixtures.h"
|
|
|
|
|
2022-11-14 20:16:29 +00:00
|
|
|
namespace libtransmission::test
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
{
|
|
|
|
|
|
|
|
class CopyTest : public SandboxedTest
|
|
|
|
{
|
|
|
|
protected:
|
|
|
|
void testImpl(char const* filename1, char const* filename2, size_t const file_length)
|
|
|
|
{
|
2022-04-22 16:35:56 +00:00
|
|
|
auto const path1 = tr_pathbuf{ sandboxDir(), '/', filename1 };
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
/* Create a file. */
|
2022-08-04 13:44:18 +00:00
|
|
|
auto contents = std::vector<char>{};
|
|
|
|
contents.resize(file_length);
|
|
|
|
tr_rand_buffer(std::data(contents), std::size(contents));
|
|
|
|
createFileWithContents(path1, std::data(contents), std::size(contents));
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
2022-04-22 16:35:56 +00:00
|
|
|
auto const path2 = tr_pathbuf{ sandboxDir(), '/', filename2 };
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
tr_error* err = nullptr;
|
|
|
|
/* Copy it. */
|
2022-04-22 16:35:56 +00:00
|
|
|
EXPECT_TRUE(tr_sys_path_copy(path1, path2, &err));
|
2022-03-21 20:22:50 +00:00
|
|
|
EXPECT_EQ(nullptr, err) << ' ' << *err;
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
tr_error_clear(&err);
|
|
|
|
|
2022-04-22 16:35:56 +00:00
|
|
|
EXPECT_TRUE(filesAreIdentical(path1, path2));
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
/* Dispose of those files that we created. */
|
2022-04-22 16:35:56 +00:00
|
|
|
tr_sys_path_remove(path1);
|
|
|
|
tr_sys_path_remove(path2);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
2022-01-24 19:07:55 +00:00
|
|
|
static uint64_t fillBufferFromFd(tr_sys_file_t fd, uint64_t bytes_remaining, char* buf, size_t buf_len)
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
{
|
|
|
|
memset(buf, 0, buf_len);
|
|
|
|
|
|
|
|
size_t buf_pos = 0;
|
|
|
|
while (buf_pos < buf_len && bytes_remaining > 0)
|
|
|
|
{
|
2021-09-19 20:41:35 +00:00
|
|
|
uint64_t const chunk_size = std::min(uint64_t{ buf_len - buf_pos }, bytes_remaining);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
uint64_t bytes_read = 0;
|
|
|
|
|
2022-03-27 17:37:29 +00:00
|
|
|
tr_sys_file_read(fd, buf + buf_pos, chunk_size, &bytes_read);
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
|
|
|
|
EXPECT_LE(buf_pos + bytes_read, buf_len);
|
|
|
|
EXPECT_LE(bytes_read, bytes_remaining);
|
|
|
|
buf_pos += bytes_read;
|
|
|
|
bytes_remaining -= bytes_read;
|
|
|
|
}
|
|
|
|
|
|
|
|
return bytes_remaining;
|
|
|
|
}
|
|
|
|
|
2022-08-04 13:44:18 +00:00
|
|
|
static bool filesAreIdentical(std::string_view filename1, std::string_view filename2)
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
{
|
2022-08-04 13:44:18 +00:00
|
|
|
auto contents1 = std::vector<char>{};
|
|
|
|
auto contents2 = std::vector<char>{};
|
2023-06-30 14:49:58 +00:00
|
|
|
return tr_file_read(filename1, contents1) && tr_file_read(filename2, contents2) && contents1 == contents2;
|
Add in-kernel file copying for several platforms. (#1092)
* Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA).
* Fix macro name USE_COPY_FILE_RANGE.
* Minor bugfixes for userspace fallback.
* Fix linux sendfile64 bugs.
* Remove some overzealous asserts.
* Allow transmission-test-copy to take an optional argument for an external reference file.
* Fix return value error of tr_sys_path_copy.
* Use COPYFILE_ALL for Macs without COPYFILE_CLONE.
* Add in-kernel file copying for several platforms.
Numerous operating systems now have support for copying files directly in the
kernel, which is generally more efficient than copying in a userspace read(2)/
write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage,
which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for
later kernels, copy_file_range(2). FreeBSD 13 will also support
copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA.
Operating systems lacking such a syscall continue to use the existing
read(2)/write(2) loop.
* Appease uncrustify.
* Appease uncrustify.
* copy-test: generate random content at run time.
* copy-test: Stylistic changes and more check()s.
* copy-test: files_are_identical should follow test idioms
* tr_sys_path_copy: numerous tweaks as requested by review.
* s/old file/source file; s/new file/destination file.
* tr_sys_path_copy: handle win32 wide characters in paths.
* Uncrustify.
* test-copy: Use non-string create_file_with_contents.
* tr_sys_path_copy: numerous fixes.
Per review: generate test file content at runtime; tidy use of check();
fix style; re-measure file sizes in the copy; define a macro when the
system does not provide it; use Unicode APIs on Windows; and fix
documentation.
* Updated as per comments.
* Rebase kernel-copy changes onto 3.0 with gtest.
* Undo irrelevant comment change.
* Fix syntax error.
* Use tr_malloc() instead of tr_valloc().
* Use EXPECT instead of TR_ASSERT in gtest.
* Add error handling.
* Acceptable coding style has changed again.
Now it's camelCase. Also use nullptr instead of NULL, etc.
* Fix east/west const.
Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
TEST_F(CopyTest, copy)
|
|
|
|
{
|
|
|
|
char const* filename1 = "orig-blob.txt";
|
|
|
|
char const* filename2 = "copy-blob.txt";
|
|
|
|
auto const random_file_length = 1024 * 1024 * 10;
|
|
|
|
|
|
|
|
testImpl(filename1, filename2, random_file_length);
|
|
|
|
}
|
|
|
|
|
2022-11-14 20:16:29 +00:00
|
|
|
} // namespace libtransmission::test
|