2014-12-16 14:10:21 +00:00
|
|
|
.. include:: global.rst.inc
|
|
|
|
|
.. _internals:
|
|
|
|
|
|
|
|
|
|
Internals
|
|
|
|
|
=========
|
|
|
|
|
|
2014-12-16 15:04:35 +00:00
|
|
|
This page documents the internal data structures and storage
|
|
|
|
|
mechanisms of |project_name|. It is partly based on `mailing list
|
2015-06-07 00:15:13 +00:00
|
|
|
discussion about internals`_ and also on static code analysis.
|
|
|
|
|
|
2017-02-17 22:31:47 +00:00
|
|
|
Borg uses a low-level, key-value store, the :ref:`repository`, and
|
|
|
|
|
implements a more complex data structure on top of it, which is made
|
|
|
|
|
up of the :ref:`manifest <manifest>`, :ref:`archives <archive>`,
|
|
|
|
|
:ref:`items <item>` and data :ref:`chunks`.
|
2015-06-07 00:15:13 +00:00
|
|
|
|
2017-02-17 22:31:47 +00:00
|
|
|
Each repository can hold multiple :ref:`archives <archive>`, which
|
|
|
|
|
represent individual backups that contain a full archive of the files
|
|
|
|
|
specified when the backup was performed.
|
2014-12-16 15:04:35 +00:00
|
|
|
|
2017-02-17 18:29:03 +00:00
|
|
|
Deduplication is performed globally across all data in the repository
|
2017-02-22 14:28:27 +00:00
|
|
|
(multiple backups and even multiple hosts), both on data and file
|
|
|
|
|
metadata, using :ref:`chunks` created by the chunker using the
|
|
|
|
|
Buzhash_ algorithm.
|
2015-06-07 00:15:13 +00:00
|
|
|
|
2017-02-17 22:31:47 +00:00
|
|
|
To actually perform the repository-wide deduplication, a hash of each
|
2017-02-22 14:28:27 +00:00
|
|
|
chunk is checked against the :ref:`chunks cache <cache>`, which is a
|
|
|
|
|
hash-table of all chunks that already exist.
|
2017-02-22 14:53:15 +00:00
|
|
|
|
|
|
|
|
.. toctree::
|
|
|
|
|
:caption: Contents
|
|
|
|
|
|
|
|
|
|
internals/security
|
|
|
|
|
internals/data-structures
|
2017-02-26 00:20:03 +00:00
|
|
|
internals/frontends
|
