mirror/borg
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2024-12-26 17:57:59 +00:00
Code Issues Releases Wiki Activity

revert incorrect fix for put updating shadow_index, fixes #5661

Browse source 
A) the compaction code needs the shadow index only for this case:

segment A: PUT x, segment B: DEL x, with A < B  (DEL shadows the PUT).

B) for the following case, we have no shadowing DEL (or rather: it does not matter,
because there is a PUT right after the DEL) and x is in the repo index,
thus the shadow_index is not needed for the special case in the compaction code:

segment A: PUT x, segment B: DEL x PUT x

see also PR #5636.

reverts f079a83fed
and clarifies the code by more comments.

we keep the code deduplication of 5f32b5666a
and just add a update_shadow_index param to make it not look like there was
something accidentally forgotten, which was the whole reason for the reverted
"fix".
This commit is contained in:
Thomas Waldmann 2021-02-04 02:29:43 +01:00
parent f195a71e24
commit 3d0c61a184
1 changed files with 13 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
src/borg/repository.py
View file

@ -1191,9 +1191,11 @@ def put(self, id, data, wait=True):
except KeyError: except KeyError:
pass pass
else: else:
# note: doing a delete first will do some bookkeeping, # note: doing a delete first will do some bookkeeping.
# like updating the shadow_index, quota, ... # we do not want to update the shadow_index here, because
self._delete(id, segment, offset) # we know already that we will PUT to this id, so it will
# be in the repo index (and we won't need it in the shadow_index).
self._delete(id, segment, offset, update_shadow_index=False)
segment, offset = self.io.write_put(id, data) segment, offset = self.io.write_put(id, data)
self.storage_quota_use += len(data) + self.io.put_header_fmt.size self.storage_quota_use += len(data) + self.io.put_header_fmt.size
self.segments.setdefault(segment, 0) self.segments.setdefault(segment, 0)
@ -1216,11 +1218,16 @@ def delete(self, id, wait=True):
segment, offset = self.index.pop(id) segment, offset = self.index.pop(id)
except KeyError: except KeyError:
raise self.ObjectNotFound(id, self.path) from None raise self.ObjectNotFound(id, self.path) from None
self._delete(id, segment, offset) # if we get here, there is an object with this id in the repo,
# we write a DEL here that shadows the respective PUT.
# after the delete, the object is not in the repo index any more,
# for the compaction code, we need to update the shadow_index in this case.
self._delete(id, segment, offset, update_shadow_index=True)
def _delete(self, id, segment, offset): def _delete(self, id, segment, offset, *, update_shadow_index):
# common code used by put and delete # common code used by put and delete
self.shadow_index.setdefault(id, []).append(segment) if update_shadow_index:
self.shadow_index.setdefault(id, []).append(segment)
self.segments[segment] -= 1 self.segments[segment] -= 1
size = self.io.read(segment, offset, id, read_data=False) size = self.io.read(segment, offset, id, read_data=False)
self.storage_quota_use -= size self.storage_quota_use -= size