Antoine R. Dumont
--- a/docs/user/software-origins/nixguix.rst

+ 73

− 9
+++ b/docs/user/software-origins/nixguix.rst

+ 73

− 9
-.. _user-software-origins-nixguix:
+'.. _user-software-origins-nixguix:

 Nix and Guix
 ============
 @@ -8,12 +8,76 @@ Nix and Guix

 .. include:: dynamic/nixguix_status.inc

-TODO:
+This page documents how |swh| archives source packages from the
+`GNU Guix <https://guix.gnu.org/>`_ and `Nix <https://nixos.org/>`_
+distributions.

-* description of the software origin
-* summary of the lister's algorithm
-* summary of the loader's algorithm
-* URL pattern
-* collect extrinsic metadata?
-* index extrinsic metadata?
-* index intrinsic metadata?
+Those distributions provide functional package managers, respectively GNU Guix and Nix
+with similar properties (e.g. transactional, declarative up to the operating system,
+reproducible, ...). Definition of packages is dependent on their respective DSL. As it's
+not parsable easily nor any listing api existed, community effort was done to provide
+regular extraction of origins listing (as json manifest).
+
+|swh|'s NixGuix lister queries respectively those origins listing. As they contain
+various types of origins, |swh| has a lister and various loaders to deal with those
+origins.
+
+Various kind of origins can be listed and ingested:
+
+- url targeting a simple file. The :ref:`Content Loader
+  <swh.loader.core.loader.ContentLoader>` loader ingests origin of type 'content'.
+- url targeting a tarball. The :ref:`Tarball Directory Loader
+  <swh.loader.core.loader.TarballDirectoryLoader>` ingests origin with type
+  'tarball-directory'
+- :ref:`Svn <user-software-origins-svn>` repository. The :ref:`Svn loader
+  <swh.loader.svn.loader.SvnLoader>` ingests origin with type svn.
+- :ref:`Svn <user-software-origins-svn>` repository at a specific revision. The
+  :ref:`Svn export loader <swh.loader.svn.directory.SvnExportLoader>` ingests origins
+  with type 'svn-export'.
+- :ref:`Git <user-software-origins-git>` repository. The :ref:`Git loader
+  <swh.loader.git.loader.GitLoader>` ingests origin with type' or at a specific commit,
+  visit type 'git-checkout')
+- :ref:`Git <user-software-origins-git>` repository at a specific git commit. Then
+  :ref:`Git checkout loader <swh.loader.git.directory.GitCheckoutLoader>` ingests origin
+  with type 'git-checkout'
+- :ref:`Mercurial <user-software-origins-mercurial>` repository. The :ref:`Mercurial
+  loader <swh.loader.mercurial.loader.HgLoader>` ingests origin with type 'hg'
+- :ref:`Mercurial <user-software-origins-mercurial>` repository. The :ref:`Mercurial
+  checkout loader <swh.loader.mercurial.directory.HgCheckoutLoader>` ingests with type
+  'hg-checkout'.
+
+Origin URLs match each main url provided in the manifest.
+
+For some cases, there can be mirror urls which are used as fallback artifact retrieval
+when the main url is no longer available.
+
+No extrinsic nor intrinsic metadata collection is happening on the lister's side.
+
+For some kind of origins ('content', 'tarball-directory', 'svn-export', 'hg-checkout',
+'git-checkout'), extra intrinsic information regarding the checksums used ('standard',
+e.g. sha256, or :ref:`Nar <swh.loader.core.Nar>`, specific intrinsic identifier used by
+Guix and Nix), are transmitted to their respective loaders. During their respective
+ingestion, those checksums are checked against. If the checksum does not match, the
+artifact is rejected and the visit is failed. If not, the artifact is ingested. Note
+also that a new entry is recorded in the ExtID table to map the SWHID content (of the
+content file) or the SWHID directory (for the other kind) ingested to their their
+original checksum.
+
+Sample:
+
+-----------------+---------------+--------------------------------------------------------------------+-------------+--------------------------------------------+
+| extid_type      | extid_version | extid                                                              | target_type | target                                     |
+-----------------+---------------+--------------------------------------------------------------------+-------------+--------------------------------------------+
+| checksum-sha256 |             1 | \x00001a5b5be28bde9bc8c353afe546d8fe84e49b269a70393c1616957b0e1cce | directory   | \xbe186100480d766ebdf0cfaeac0c90198f4b42e7 |
+| nar-sha256      |             1 | \x00002584a56a9bce85793515604298f8b3b1e9497e00fc6361a0c2e731c063f3 | directory   | \x1e1ace5b0ef56e188d3cf99059070cc5448d7454 |
+-----------------+---------------+--------------------------------------------------------------------+-------------+--------------------------------------------+
+
+Resources:
+
+* `Gnu Guix git repository <https://git.savannah.gnu.org/cgit/guix.git>`__
+
+* `Nixpkgs git repository <https://github.com/nixos/nixpkgs>`__
+
+* `Remote nixpkgs json manifest <https://nix-community.github.io/nixpkgs-swh/sources-unstable-full.json>`__
+
+- `Remote guix json manifest <https://nix-community.github.io/nixpkgs-swh/>`__