1
0
Fork 0
mirror of https://github.com/transmission/transmission synced 2024-12-22 07:42:37 +00:00
transmission/tests/libtransmission/copy-test.cc

91 lines
2.7 KiB
C++
Raw Permalink Normal View History

2022-08-08 18:05:39 +00:00
// This file Copyright (C) 2013-2022 Mnemosyne LLC.
// It may be used under GPLv2 (SPDX: GPL-2.0-only), GPLv3 (SPDX: GPL-3.0-only),
// or any future license endorsed by Mnemosyne LLC.
// License text can be found in the licenses/ folder.
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
#include <algorithm>
2023-07-08 15:24:03 +00:00
#include <cstdint> // uint64_t
2022-08-17 16:08:36 +00:00
#include <cstring>
2023-07-08 15:24:03 +00:00
#include <string_view>
2022-08-04 13:44:18 +00:00
#include <vector>
2023-07-08 15:24:03 +00:00
#include <libtransmission/crypto-utils.h>
#include <libtransmission/error.h>
#include <libtransmission/file.h>
2023-07-08 15:24:03 +00:00
#include <libtransmission/tr-strbuf.h>
#include <libtransmission/utils.h>
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
2023-07-08 15:24:03 +00:00
#include "gtest/gtest.h"
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
#include "test-fixtures.h"
namespace libtransmission::test
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
{
class CopyTest : public SandboxedTest
{
protected:
void testImpl(char const* filename1, char const* filename2, size_t const file_length)
{
auto const path1 = tr_pathbuf{ sandboxDir(), '/', filename1 };
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
/* Create a file. */
2022-08-04 13:44:18 +00:00
auto contents = std::vector<char>{};
contents.resize(file_length);
tr_rand_buffer(std::data(contents), std::size(contents));
createFileWithContents(path1, std::data(contents), std::size(contents));
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
auto const path2 = tr_pathbuf{ sandboxDir(), '/', filename2 };
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
/* Copy it. */
auto error = tr_error{};
EXPECT_TRUE(tr_sys_path_copy(path1, path2, &error));
EXPECT_FALSE(error) << error;
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
EXPECT_TRUE(filesAreIdentical(path1, path2));
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
/* Dispose of those files that we created. */
tr_sys_path_remove(path1);
tr_sys_path_remove(path2);
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
}
private:
static uint64_t fillBufferFromFd(tr_sys_file_t fd, uint64_t bytes_remaining, char* buf, size_t buf_len)
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
{
memset(buf, 0, buf_len);
size_t buf_pos = 0;
while (buf_pos < buf_len && bytes_remaining > 0)
{
uint64_t const chunk_size = std::min(uint64_t{ buf_len - buf_pos }, bytes_remaining);
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
uint64_t bytes_read = 0;
tr_sys_file_read(fd, buf + buf_pos, chunk_size, &bytes_read);
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
EXPECT_LE(buf_pos + bytes_read, buf_len);
EXPECT_LE(bytes_read, bytes_remaining);
buf_pos += bytes_read;
bytes_remaining -= bytes_read;
}
return bytes_remaining;
}
2022-08-04 13:44:18 +00:00
static bool filesAreIdentical(std::string_view filename1, std::string_view filename2)
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
{
2022-08-04 13:44:18 +00:00
auto contents1 = std::vector<char>{};
auto contents2 = std::vector<char>{};
return tr_file_read(filename1, contents1) && tr_file_read(filename2, contents2) && contents1 == contents2;
Add in-kernel file copying for several platforms. (#1092) * Add in-kernel copying support for Linux (sendfile64(2), copy_file_range(2)), FreeBSD 13 (copy_file_range(2)), MacOS (copyfile(2)), and Windows (CopyFileExA). * Fix macro name USE_COPY_FILE_RANGE. * Minor bugfixes for userspace fallback. * Fix linux sendfile64 bugs. * Remove some overzealous asserts. * Allow transmission-test-copy to take an optional argument for an external reference file. * Fix return value error of tr_sys_path_copy. * Use COPYFILE_ALL for Macs without COPYFILE_CLONE. * Add in-kernel file copying for several platforms. Numerous operating systems now have support for copying files directly in the kernel, which is generally more efficient than copying in a userspace read(2)/ write(2) loop. (This becomes particularly relevant for 4th gen PCI-E storage, which approaches the latency of DRAM.) For Linux I use sendfile64(2), and, for later kernels, copy_file_range(2). FreeBSD 13 will also support copy_file_range(2). MacOS has copyfile(2), and Windows has CopyFileExA. Operating systems lacking such a syscall continue to use the existing read(2)/write(2) loop. * Appease uncrustify. * Appease uncrustify. * copy-test: generate random content at run time. * copy-test: Stylistic changes and more check()s. * copy-test: files_are_identical should follow test idioms * tr_sys_path_copy: numerous tweaks as requested by review. * s/old file/source file; s/new file/destination file. * tr_sys_path_copy: handle win32 wide characters in paths. * Uncrustify. * test-copy: Use non-string create_file_with_contents. * tr_sys_path_copy: numerous fixes. Per review: generate test file content at runtime; tidy use of check(); fix style; re-measure file sizes in the copy; define a macro when the system does not provide it; use Unicode APIs on Windows; and fix documentation. * Updated as per comments. * Rebase kernel-copy changes onto 3.0 with gtest. * Undo irrelevant comment change. * Fix syntax error. * Use tr_malloc() instead of tr_valloc(). * Use EXPECT instead of TR_ASSERT in gtest. * Add error handling. * Acceptable coding style has changed again. Now it's camelCase. Also use nullptr instead of NULL, etc. * Fix east/west const. Co-authored-by: Mike Gelfand <mikedld@users.noreply.github.com>
2021-08-07 09:04:03 +00:00
}
};
TEST_F(CopyTest, copy)
{
char const* filename1 = "orig-blob.txt";
char const* filename2 = "copy-blob.txt";
auto const random_file_length = 1024 * 1024 * 10;
testImpl(filename1, filename2, random_file_length);
}
} // namespace libtransmission::test