7678 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Andreas Vögele	4b18980768	Installation: Update Fedora in distribution list ... Fixes #7357	2023-02-16 18:27:48 +01:00
TW	3849ebe312	Merge pull request #7349 from ThomasWaldmann/avoid-orphan-content-chunks3 ... avoid orphan content chunks (master)	2023-02-14 00:01:50 +01:00
Thomas Waldmann	71f8dd3a17	FilesystemObjectProcessors.process_pipe: also add same exception handler there	2023-02-13 20:46:48 +01:00
Thomas Waldmann	c9dbe323e3	TarfileObjectProcessors.process_file: also add same exception handler there	2023-02-13 20:46:46 +01:00
Thomas Waldmann	d0c61bbbf1	FilesystemObjectProcessors.process_file: clean up orphaned chunks in case of exceptions ... Note: no changes inside the indented block, just added the try and the except block.	2023-02-13 20:46:45 +01:00
Thomas Waldmann	f1981715e4	2nd+ hardlinks: add to item.chunks after incref'ing ... we do book-keeping in item.chunks: in case something goes wrong and we need to clean up, we will have a list with chunks to decref in item.chunks. also: - make variable naming more consistent - cosmetic changes	2023-02-13 20:46:21 +01:00
Thomas Waldmann	5cb3a17796	Revert "avoid orphan content chunks on BackupOSError, fixes #6709" ... This reverts commit ffe32316a5.	2023-02-13 18:24:28 +01:00
TW	6cfe77ebaf	Merge pull request #7350 from ThomasWaldmann/testing-chunker ... implement "fail" chunker for testing purposes	2023-02-13 18:17:56 +01:00
Thomas Waldmann	11fd6afb0f	use "fail" chunker to test erroneous input file skipping ... if a file can't be read (like here: there is a simulated I/O error in the 2nd chunk of file2), it should be logged with "E" status, skipped and backup shall proceed with next file(s). also, check that the repo has no orphan chunks (exception handling code needs to deal with 1st chunk of file2 which already has been written / incref'd in the repo).	2023-02-13 17:19:49 +01:00
Thomas Waldmann	7f973a5b34	implement "fail" chunker for testing purposes ... --chunker-params=fail,4096,rrrEErrrr means: - cut chunks of 4096b fixed size (last chunk in a file can be less) - read chunks 0, 1 and 2 successfully - error at chunk 3 and 4 (simulated OSError(errno.EIO)) - read successfully again for the next 4 chunks Chunks are counted inside the chunker instance, starting from 0, always increasing while the same instance is used. Read chunks as well as failed chunks count up by 1.	2023-02-13 17:15:45 +01:00
TW	52040bc043	Merge pull request #7347 from jdchristensen/hashindex_compact ... Make hashindex_compact simpler and probably faster	2023-02-12 17:01:18 +01:00
Dan Christensen	3daf7e186d	testsuite/hashindex.py: add comments about validity of hashtables	2023-02-12 09:39:07 -05:00
TW	ee7fee75fc	Merge pull request #7345 from ThomasWaldmann/update-changes ... update CHANGES	2023-02-12 14:56:18 +01:00
Dan Christensen	c705c8573d	testsuite/hashindex.py: add a few more tests	2023-02-12 08:28:14 -05:00
Dan Christensen	8eff0b2f02	_hashindex.c: fix comment	2023-02-12 08:24:27 -05:00
Dan Christensen	2f9f22e2a8	hashindex.py: make black happy	2023-02-11 20:33:07 -05:00
Dan Christensen	36aa395e49	_hashindex.c: set min_empty and num_empty even when permit_compact=True	2023-02-11 20:11:09 -05:00
Dan Christensen	60bea46eb7	_hashindex.c: rewrite hashindex_compact	2023-02-11 20:10:02 -05:00
Dan Christensen	6f5efb3f74	testsuite/hashindex.py: create and use compare_compact to avoid repetition	2023-02-11 20:06:38 -05:00
Dan Christensen	d3fd93f82b	testsuite/hashindex.py: use compare_indexes in one more spot	2023-02-11 19:53:28 -05:00
Dan Christensen	cc78ef8939	testsuite/hashindex.py: add two more tests	2023-02-11 19:44:56 -05:00
Dan Christensen	6aa53a5627	hashindex_pytest.py: add test_hashindex_compact_stress	2023-02-11 19:13:20 -05:00
Dan Christensen	51b29ab426	hashindex.py: make .compact tests independent of order	2023-02-11 17:37:53 -05:00
Dan Christensen	133ffebc8d	hashindex_pytest.py: factor out verify_hash_table	2023-02-11 17:37:20 -05:00
Thomas Waldmann	b42d2bf6dd	update CHANGES	2023-02-11 18:56:30 +01:00
TW	6db21fa330	Merge pull request #7344 from RayyanAnsari/parseformat-rm-hack ... parseformat: remove Windows-specific hack	2023-02-11 18:23:47 +01:00
TW	098fca7861	Merge pull request #7343 from ThomasWaldmann/timestamp-comparisons-master ... relaxed timestamp comparisons (master)	2023-02-11 18:15:02 +01:00
Rayyan Ansari	03a27654e0	parseformat: remove Windows-specific hack ... This fixes the rest of the tests.	2023-02-11 16:32:19 +00:00
Thomas Waldmann	aee17a87e2	tests: use same_ts_ns for all timestamp comparisons	2023-02-11 17:08:56 +01:00
TW	d90ebc8bef	Merge pull request #7340 from ThomasWaldmann/fix-recreate-rechunkify-master ... recreate: --chunker-params must default to None, fixes #7337	2023-02-11 14:11:03 +01:00
TW	ce04521a9a	Merge pull request #7333 from ThomasWaldmann/excluded-included-status-chars ... better included/excluded status chars, docs, fixes #7321	2023-02-11 13:48:43 +01:00
TW	f39d1bcd7b	Merge pull request #7334 from ThomasWaldmann/list-progress-json-is-nice ... allow mixing --progress and --list in log-json mode	2023-02-11 13:42:59 +01:00
Thomas Waldmann	2ef09eaa77	same_ts_ns: add a timestamp comparison function ... also: HAVE_UTIMES means 1us resolution (*) (*) at least as far as the utimes() call is concerned. Whether there are other limitations has to be seen.	2023-02-11 01:19:01 +01:00
TW	b233dbb4db	Merge pull request #7339 from ThomasWaldmann/test-dir-mtime-master ... add test for extracted directory mtime (master)	2023-02-11 01:13:32 +01:00
Thomas Waldmann	ce5f1589ea	recreate: --chunker-params must default to None, fixes #7336 ... also add a test: recreate without --chunker-params shall not rechunk before the fix, it triggered rechunking if an archive was created with non-default chunker params. but it only should rechunk if borg recreate is invoked with explicitly giving --chunker-params=....	2023-02-11 01:08:01 +01:00
Thomas Waldmann	fdc48fdd1b	more directory timestamp tests	2023-02-11 00:42:30 +01:00
Thomas Waldmann	a42cd161de	add test for extracted directory mtime	2023-02-11 00:29:57 +01:00
Thomas Waldmann	ff564de1cc	allow mixing --progress and --list in log-json mode	2023-02-10 01:45:59 +01:00
Thomas Waldmann	303c474f21	better included/excluded status chars, docs, fixes #7321 ... more consistent now between dry-run and non-dry-run mode. --filter=... users might need to update the status chars they filter for.	2023-02-10 01:13:21 +01:00
TW	366731ba00	Merge pull request #7300 from RayyanAnsari/borg-platformdirs ... use platformdirs	2023-02-10 00:09:44 +01:00
TW	16f4bf5d7b	Merge pull request #7328 from ThomasWaldmann/hashindex-compact ... hashindex minor fixes, tweaks, tests	2023-02-09 22:46:48 +01:00
Thomas Waldmann	3d57dc0590	hashindex: add tests, misc. minor fixes/changes ... test hashtable expansion/rebuild. hashindex_lookup: - return -2 for a compact / completely full hashtable - return -1 and (via start_idx pointer) the deleted/tombstone bucket index. fix size assertion (we add 1 element to trigger rebuild) fix upper_limit check - since we'll be adding 1 to num_entries below, the condition should be >=: hashindex_compact: set min_empty/upper_limit Co-authored-by: Dan Christensen <jdc+github@uwo.ca>	2023-02-09 22:14:07 +01:00
Thomas Waldmann	1ed8ac0408	hashindex_pytest: move hashtable create into separate function	2023-02-09 21:58:40 +01:00
Thomas Waldmann	e556233e30	hashindex_pytest: add a comment	2023-02-09 21:58:37 +01:00
Thomas Waldmann	4c2af9fb96	renamed hashindex_stress -> hashindex_pytest	2023-02-09 21:58:23 +01:00
TW	79d6835738	Merge pull request #7331 from ThomasWaldmann/undefine-ndebug-master ... undef NDEBUG for chunker and hashindex	2023-02-09 21:52:20 +01:00
Thomas Waldmann	311ac55ebd	undef NDEBUG for chunker and hashindex ... note: intentionally not undef'ing it for all the cython-only generated C code.	2023-02-09 21:01:03 +01:00
TW	2b2586a098	Merge pull request #7330 from ThomasWaldmann/update-codecov ... upgrade codecov-action to v3	2023-02-09 20:37:27 +01:00
Thomas Waldmann	142eb65223	upgrade codecov-action to v3	2023-02-09 18:46:36 +01:00
TW	406da3d419	Merge pull request #7325 from ThomasWaldmann/hashtable-fixes-master ... hashindex bugfix and refactoring (master)	2023-02-08 23:53:24 +01:00