diff --git a/doc/045_working_with_repos.rst b/doc/045_working_with_repos.rst index 8dba8439f..f31e75c84 100644 --- a/doc/045_working_with_repos.rst +++ b/doc/045_working_with_repos.rst @@ -305,6 +305,13 @@ In order to preview the changes which ``rewrite`` would make, you can use the modifying the repository. Instead restic will only print the actions it would perform. +.. note:: The ``rewrite`` command verifies that it does not modify snapshots in + unexpected ways and fails with an ``cannot encode tree at "[...]" without loosing information`` + error otherwise. This can occur when rewriting a snapshot created by a newer + version of restic or some third-party implementation. + + To convert a snapshot into the format expected by the ``rewrite`` command + use ``restic repair snapshots ``. Modifying metadata of snapshots =============================== diff --git a/doc/design.rst b/doc/design.rst index 7fb8b71b2..c974e997a 100644 --- a/doc/design.rst +++ b/doc/design.rst @@ -126,8 +126,8 @@ the option ``-o local.layout=default``, valid values are ``default`` and ``s3legacy``. The option for the sftp backend is named ``sftp.layout``, for the s3 backend ``s3.layout``. -S3 Legacy Layout ----------------- +S3 Legacy Layout (deprecated) +----------------------------- Unfortunately during development the Amazon S3 backend uses slightly different paths (directory names use singular instead of plural for ``key``, @@ -152,8 +152,7 @@ the ``data`` directory. The S3 Legacy repository layout looks like this: /snapshot └── 22a5af1bdc6e616f8a29579458c49627e01b32210d09adb288d1ecda7c5711ec -The S3 backend understands and accepts both forms, new backends are -always created with the default layout for compatibility reasons. +Restic 0.17 is the last version that supports the legacy layout. Pack Format =========== @@ -234,7 +233,9 @@ Individual files for the index, locks or snapshots are encrypted and authenticated like Data and Tree Blobs, so the outer structure is ``IV || Ciphertext || MAC`` again. In repository format version 1 the plaintext always consists of a JSON document which must either be an -object or an array. +object or an array. The JSON encoder must deterministically encode the +document and should match the behavior of the Go standard library implementation +in ``encoding/json``. Repository format version 2 adds support for compression. The plaintext now starts with a header to indicate the encoding version to distinguish @@ -473,6 +474,10 @@ A snapshot references a tree by the SHA-256 hash of the JSON string representation of its contents. Trees and data are saved in pack files in a subdirectory of the directory ``data``. +The JSON encoder must deterministically encode the document and should +match the behavior of the Go standard library implementation in ``encoding/json``. +This ensures that trees can be properly deduplicated. + The command ``restic cat blob`` can be used to inspect the tree referenced above (piping the output of the command to ``jq .`` so that the JSON is indented): @@ -507,12 +512,11 @@ this metadata is generated: - The name is quoted using `strconv.Quote `__ before being saved. This handles non-unicode names, but also changes the representation of names containing ``"`` or ``\``. - - The filemode saved is the mode defined by `fs.FileMode `__ masked by ``os.ModePerm | os.ModeType | os.ModeSetuid | os.ModeSetgid | os.ModeSticky`` - -When the entry references a directory, the field ``subtree`` contains the plain text -ID of another tree object. +- When the entry references a directory, the field ``subtree`` contains the plain text + ID of another tree object. +- Check the implementation for a full struct definition. When the command ``restic cat blob`` is used, the plaintext ID is needed to print a tree. The tree referenced above can be dumped as follows: