diff --git a/docs/devel/contributing/phabricator.rst b/docs/devel/contributing/phabricator.rst
deleted file mode 100644
index 4f434be5fbfe74b4c475c21505e32ea6c043781f..0000000000000000000000000000000000000000
--- a/docs/devel/contributing/phabricator.rst
+++ /dev/null
@@ -1,308 +0,0 @@
-:orphan:
-
-.. highlight:: bash
-
-.. admonition:: Intended audience
- :class: important
-
- Contributors
-
-Important
-=========
-
-We have moved our development from Phabricator to a GitLab instance at
-https://gitlab.softwareheritage.org/
-
-The content below is no longer relevant and will be updated soon.
-
-Submitting patches
-==================
-
-`Phabricator`_ is the tool that Software Heritage uses as its
-coding/collaboration forge.
-
-Software Heritage's Phabricator instance can be found at
-https://forge.softwareheritage.org/
-
-.. _Phabricator: http://phabricator.org/
-
-Code Review in Phabricator
---------------------------
-
-We use the Differential application of Phabricator to perform
-:ref:`code reviews <code-review>` in the context of Software Heritage.
-
-* we use Git and ``history.immutable=true``
- (but beware as that is partly a Phabricator misnomer, read on)
-* when code reviews are required, developers will be allowed to push
- directly to master once an accepted Differential diff exists
-
-Configuration
-+++++++++++++
-
-.. _arcanist-configuration:
-
-Arcanist configuration
-^^^^^^^^^^^^^^^^^^^^^^
-
-Authentication
-~~~~~~~~~~~~~~
-
-First, you should install Arcanist and authenticate it to Phabricator::
-
- sudo apt-get install arcanist
- arc set-config default https://forge.softwareheritage.org/
- arc install-certificate
-
-arc will prompt you to login into Phabricator via web
-(which will ask your personal Phabricator credentials).
-You will then have to copy paste the API token from the web page to arc,
-and hit Enter to complete the certificate installation.
-
-Immutability
-~~~~~~~~~~~~
-
-When using git, Arcanist by default mess with the local history,
-rewriting commits at the time of first submission.
-To avoid that we use so called `history immutability`_
-
-.. _history immutability: https://secure.phabricator.com/book/phabricator/article/arcanist_new_project/#history-mutability-git
-
-To that end, you shall configure your ``arc`` accordingly::
-
- arc set-config history.immutable true
-
-Note that this does **not** mean that you are forbidden to rewrite
-your local branches (e.g., with ``git rebase``).
-Quite the contrary: you are encouraged to locally rewrite branches
-before pushing to ensure that commits are logically separated
-and your commit history easy to bisect.
-The above setting just means that *arc* will not rewrite commit
-history under your nose.
-
-Enabling ``git push`` to our forge
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The way we've configured our review setup for continuous integration
-needs you to configure git to allow pushes to our forge.
-There's two ways you can do this : setting a ssh key to push over ssh,
-or setting a specific password for git pushes over https.
-
-SSH key for pushes
-~~~~~~~~~~~~~~~~~~
-
-In your forge User settings page (On the top right, click on your avatar,
-then click *Settings*), you have access to a *Authentication* >
-*SSH Public Keys* section (Direct link:
-``hxxps://forge.softwareheritage.org/settings/user/<your username>/page/ssh/``).
-You then have the option to upload a SSH public key,
-which will authenticate your pushes.
-
-You then need to configure ssh/git to use that key pair,
-for instance by editing the ``~/.ssh/config`` file.
-
-Finally, you should configure git to push over ssh when pushing to
-https://forge.softwareheritage.org, by running the following command::
-
- git config --global url.git@forge.softwareheritage.org:.pushInsteadOf https://forge.softwareheritage.org
-
-This lets git know that it should use ``git@forge.softwareheritage.org:``
-as a base url when pushing repositories cloned from
-forge.softwareheritage.org over https.
-
-VCS password for pushes
-~~~~~~~~~~~~~~~~~~~~~~~
-
-.. warning:: Please, only use this if you're completely unable to use ssh.
-
-As a fallback to the ssh setup, you have the option of setting a VCS password. This
-password, *separate from your account password*, allows Phabricator to authenticate your
-uploads over HTTPS.
-
-In your forge User settings page (On the top right, click on your avatar, then click
-*Settings*), you need to use the *Authentication* > *VCS Password* section to set your
-VCS password (Direct link: ``hxxps://forge.softwareheritage.org/settings/user/<your
-username>/page/vcspassword/``).
-
-If you still get a 403 error on push, this means you need a forge administrator to
-enable HTTPS pushes for the repository (which wasn't done by default in historical
-repositories). Please drop by on IRC and let us know!
-
-Workflow
-++++++++
-
-* work in a feature branch: ``git checkout -b my-feat``
-* initial review request: hack/commit/hack/commit ;
- ``arc diff origin/master``
-* react to change requests: hack/commit/hack/commit ;
- ``arc diff --update Dxx origin/master``
-* landing change: ``git checkout master ; git merge my-feat ; git push``
-
-Starting a new feature and submit it for review
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Use a **one branch per feature** workflow, with well-separated
-**logical commits** (:ref:`following those conventions <git-style-guide>`).
-Please open one diff per logical commit to keep the diff size to a minimum.
-
-.. code-block::
-
- git checkout -b my-shiny-feature
- ... hack hack hack ...
- git commit -m 'architecture skeleton for my-shiny-feature'
- ... hack hack hack ...
- git commit -m 'my-shiny-feature: implement module foo'
- ... etc ...
-
-Please, follow the
-To **submit your code for review** the first time::
-
- arc diff origin/master
-
-arc will prompt for a **code review message**. Provide the following information:
-
-* first line: *short description* of the overall work
- (i.e., the feature you're working on).
- This will become the title of the review
-* *Summary* field (optional): *long description* of the overall work;
- the field can continue in subsequent lines, up to the next field.
- This will become the "Summary" section of the review
-* *Test Plan* field (optional): write here if something special is needed
- to test your change
-* *Reviewers* field (optional): the (Phabricator) name(s) of
- desired reviewers.
- If you don't specify one (recommended) the default reviewers will be chosen
-* *Subscribers* field (optional): the (Phabricator) name(s) of people that
- will be notified about changes to this review request.
- In most cases it should be left empty
-
-For example::
-
- mercurial loader
-
- Summary: first stab at a mercurial loader (T329)
-
- The implementation follows the plan detailed in F2F discussion with @foo.
-
- Performances seem decent enough for a first trial (XXX seconds for YYY repository
- that contains ZZZ patches).
-
- Test plan:
-
- Reviewers:
-
- Subscribers: foo
-
-After completing the message arc will submit the review request
-and tell you its number and URL::
-
- [...]
- Created a new Differential revision:
- Revision URI: https://forge.softwareheritage.org/Dxx
-
-.. _arc-update:
-
-Updating your branch to reflect requested changes
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Your feature might get accepted as is, YAY!
-Or, reviewers might request changes; no big deal!
-
-Use the Differential web UI to follow-up to received comments, if needed.
-
-To implement requested changes in the code, hack on your branch as usual by:
-
-* adding new commits, and/or
-* rewriting old commits with git rebase (to preserve a nice, easy to bisect history)
-* pulling on master and rebasing your branch against it if meanwhile someone
- landed commits on master:
-
-.. code-block::
-
- git checkout master
- git pull
- git checkout my-shiny-feature
- git rebase master
-
-
-When you're ready to **update your review request**::
-
- arc diff --update Dxx HEAD~
-
-Arc will prompt you for a message: **describe what you've changed
-w.r.t. the previous review request**, free form.
-This means you should not repeat the title of your diff (which is
-often the default if you squashed/amended your commits)
-
-Your message will become the changelog entry in Differential
-for this new version of the diff, and will help reviewers
-understand what changes you made since they last read your diff.
-
-Differential only care about the code diff, and not about the commits
-or their order.
-Therefore each "update" can be a completely different series of commits,
-possibly rewritten from the previous submission.
-
-Dependencies between diffs
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Note that you can manage diff dependencies within the same module
-with the following keyword in the diff description::
-
- Depends on Dxx
-
-That allows to keep a logical view in your diff.
-It's not strictly necessary (because the tooling now deals with it properly)
-but it might help reviewers or yourself to do so.
-
-Landing your change onto master
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Once your change has been approved in Differential,
-you will be able to land it onto the master branch.
-
-Before doing so, you're encouraged to **clean up your git commit history**,
-reordering/splitting/merging commits as needed to have separate
-logical commits and an easy to bisect history.
-Update the diff :ref:`following the prior section <arc-update>`
-(It'd be good to let the CI build finish to make sure everything is still green).
-
-Once you're happy you can **push to origin/master** directly, e.g.::
-
- git checkout master
- git merge --ff-only my-shiny-feature
- git push
-
-``--ff-only`` is optional, and makes sure you don't unintentionally
-create a merge commit.
-
-Optionally you can then delete your local feature branch::
-
- git branch -d my-shiny-feature
-
-Reviewing locally / landing someone else's changes
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-You can do local reviews of code with arc patch::
-
- arc patch Dxyz
-
-This will create a branch **arcpatch-Dxyz** containing the changes
-on your local checkout.
-
-You can then merge those changes upstream with::
-
- git checkout master
- git merge --ff arcpatch-Dxyz
- git push origin master
-
-or, alternatively::
-
- arc land --squash
-
-
-See also
---------
-
-* :ref:`code-review` for guidelines on how code is reviewed
- when developing for Software Heritage
diff --git a/docs/devel/contributing/sphinx.rst b/docs/devel/contributing/sphinx.rst
index 82f0c48b5037fe95eaf3d7eb582f22bdc4c8ef6c..2078662cec83dc95676dd79fbf8e1c5a60305f1f 100644
--- a/docs/devel/contributing/sphinx.rst
+++ b/docs/devel/contributing/sphinx.rst
@@ -3,7 +3,12 @@
Sphinx gotchas
==============
-Here is a list of common gotchas when formatting Python docstrings for `Sphinx <https://www.sphinx-doc.org/en/stable/>`_ and the `Napoleon <https://www.sphinx-doc.org/en/stable/ext/napoleon.html>`_ style.
+Here is a list of common gotchas when formatting Python docstrings for `Sphinx
+<https://www.sphinx-doc.org/en/stable/>`_ and the `Napoleon
+<https://www.sphinx-doc.org/en/stable/ext/napoleon.html>`_ style.
+
+.. highlight:: rst
+
Sphinx
------
@@ -11,12 +16,12 @@ Sphinx
Lists
+++++
-All sorts of `lists <https://www.sphinx-doc.org/en/stable/rest.html#lists-and-quote-like-blocks>`_
-require an empty line before the first bullet and after the last one,
-to be properly interpreted as list.
-No indentation is required for list elements w.r.t. surrounding text,
-and line continuations should be indented like the first character
-after the bullet.
+All sorts of `lists
+<https://www.sphinx-doc.org/en/stable/rest.html#lists-and-quote-like-blocks>`_
+require an empty line before the first bullet and after the last one, to be
+properly interpreted as list. No indentation is required for list elements
+w.r.t. surrounding text, and line continuations should be indented like the
+first character after the bullet.
Bad::
@@ -177,10 +182,13 @@ Good::
Args:
foo (int): first argument
+
bar: second argument, which happen to have a fairly
- long description of what it does
+ long description of what it does
+
baz (bool): third argument
+
Returns
+++++++
@@ -232,6 +240,7 @@ Good::
ValueError: if you botched it
RuntimeError: if we botched it
+
See also
--------
diff --git a/docs/devel/getting-started/api.rst b/docs/devel/getting-started/api.rst
index c3f603cc54adad3b8cbbde897e45620e76bd99bd..ade722b93d2a0883189d76c0669eae70354693c6 100644
--- a/docs/devel/getting-started/api.rst
+++ b/docs/devel/getting-started/api.rst
@@ -60,7 +60,9 @@ This article uses Python 3.x on the client side, and the ``requests``
Python module to manipulate the HTTP requests. Note however that any
language that provides HTTP requests (GET, POST) can access the API and
could be used. Firstly let’s make sure we have the correct Python
-version and module installed::
+version and module installed:
+
+.. code-block:: console
boris@castalia:notebook$ python3 -V
Python 3.7.3
@@ -76,15 +78,15 @@ Heritage API, namely ``json`` and the aforementioned ``requests``
modules. We also define a utility function to pretty-print json data
easily:
-.. code:: python
+.. code-block:: python
- import json
- import requests
+ import json
+ import requests
- # Utility to pretty-print json.
- def jprint(obj):
- # create a formatted string of the Python JSON object
- print(json.dumps(obj, sort_keys=True, indent=4))
+ # Utility to pretty-print json.
+ def jprint(obj):
+ # create a formatted string of the Python JSON object
+ print(json.dumps(obj, sort_keys=True, indent=4))
The syntax mentioned in the :swh_web:`API documentation <api/1/>` is rather
@@ -125,21 +127,21 @@ YAML if we wanted to, with a custom ``Request Headers`` set to
.. code-block:: python
- resp = requests.get("https://archive.softwareheritage.org/api/1/stat/counters/")
- counters = resp.json()
- jprint(counters)
+ resp = requests.get("https://archive.softwareheritage.org/api/1/stat/counters/")
+ counters = resp.json()
+ jprint(counters)
.. code-block:: python
- {
- "content": 10049535736,
- "directory": 8390591308,
- "origin": 156388918,
- "person": 42263568,
- "release": 17218891,
- "revision": 2109783249
- }
+ {
+ "content": 10049535736,
+ "directory": 8390591308,
+ "origin": 156388918,
+ "person": 42263568,
+ "release": 17218891,
+ "revision": 2109783249
+ }
There are almost 10bn blobs (aka files) in the archive and 8bn+
@@ -171,38 +173,41 @@ as:
A (truncated) example of a result from this endpoint is shown below:
-::
+.. code-block:: json
[
{
"origin_visits_url": "https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visits/",
"url": "https://github.com/borisbaldassari/alambic"
}
- ...
]
As an example we will look for instances of *alambic* in the archive’s
-analysed repositories::
+analysed repositories:
+
+.. code-block:: python
+
+ resp = requests.get("https://archive.softwareheritage.org/api/1/origin/search/alambic/")
+ origins = resp.json()
+ print(f"We found {len(origins)} entries.")
+ for origin in origins[1:10]:
+ print(f"- {origin['url']}")
- resp = requests.get("https://archive.softwareheritage.org/api/1/origin/search/alambic/")
- origins = resp.json()
- print(f"We found {len(origins)} entries.")
- for origin in origins[1:10]:
- print(f"- {origin['url']}")
+Which produces:
-Which produces::
+.. code-block:: console
- We found 52 entries.
- - https://github.com/royal-alambic-club/sauron
- - https://github.com/scamberlin/alambic
- - https://github.com/WebTales/alambic-connector-mongodb
- - https://github.com/WebTales/alambic
- - https://github.com/AssoAlambic/alambic-website
- - https://bitbucket.org/nayoub/alambic.git
- - https://github.com/Alexandru-Dobre/alambic-connector-rest
- - https://github.com/WebTales/alambic-connector-diffbot
- - https://github.com/WebTales/alambic-connector-firebase
+ We found 52 entries.
+ - https://github.com/royal-alambic-club/sauron
+ - https://github.com/scamberlin/alambic
+ - https://github.com/WebTales/alambic-connector-mongodb
+ - https://github.com/WebTales/alambic
+ - https://github.com/AssoAlambic/alambic-website
+ - https://bitbucket.org/nayoub/alambic.git
+ - https://github.com/Alexandru-Dobre/alambic-connector-rest
+ - https://github.com/WebTales/alambic-connector-diffbot
+ - https://github.com/WebTales/alambic-connector-firebase
There are obviously many projects and repositories that embed the word
@@ -236,14 +241,14 @@ like this:
``/api/1/origin/https://github.com/borisbaldassari/alambic/get/``
-.. code:: python
+.. code-block:: python
- resp = requests.get("https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/get/")
- found = resp.json()
- jprint(found)
+ resp = requests.get("https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/get/")
+ found = resp.json()
+ jprint(found)
-.. code::
+.. code-block:: json
{
"origin_visits_url": "https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visits/",
@@ -263,41 +268,41 @@ syntax:
We will use the same query as before about the main Alambic repository.
-.. code:: python
+.. code-block:: python
- resp = requests.get("https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visits/")
- found = resp.json()
- length = len(found)
- print(f"Number of visits found: {format(length)}.")
- print("With dates:")
- for visit in found:
- print(f"- {visit['visit']} {visit['date']}")
- print("\nExample of a single visit entry:")
- jprint(found[0])
+ resp = requests.get("https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visits/")
+ found = resp.json()
+ length = len(found)
+ print(f"Number of visits found: {format(length)}.")
+ print("With dates:")
+ for visit in found:
+ print(f"- {visit['visit']} {visit['date']}")
+ print("\nExample of a single visit entry:")
+ jprint(found[0])
-.. code::
+.. code-block:: console
- Number of visits found: 5.
- With dates:
- - 5 2021-01-01T19:35:41.308336+00:00
- - 4 2020-02-06T10:41:45.700641+00:00
- - 3 2019-09-01T22:38:12.056537+00:00
- - 2 2019-06-16T04:52:18.162914+00:00
- - 1 2019-01-30T07:19:20.799217+00:00
+ Number of visits found: 5.
+ With dates:
+ - 5 2021-01-01T19:35:41.308336+00:00
+ - 4 2020-02-06T10:41:45.700641+00:00
+ - 3 2019-09-01T22:38:12.056537+00:00
+ - 2 2019-06-16T04:52:18.162914+00:00
+ - 1 2019-01-30T07:19:20.799217+00:00
- Example of a single visit entry:
- {
- "date": "2021-01-01T19:35:41.308336+00:00",
- "metadata": {},
- "origin": "https://github.com/borisbaldassari/alambic",
- "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visit/5/",
- "snapshot": "6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc",
- "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc/",
- "status": "full",
- "type": "git",
- "visit": 5
- }
+ Example of a single visit entry:
+ {
+ "date": "2021-01-01T19:35:41.308336+00:00",
+ "metadata": {},
+ "origin": "https://github.com/borisbaldassari/alambic",
+ "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://github.com/borisbaldassari/alambic/visit/5/",
+ "snapshot": "6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc",
+ "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc/",
+ "status": "full",
+ "type": "git",
+ "visit": 5
+ }
Get the content
@@ -308,16 +313,16 @@ at a given time with links to all branches and releases. In this example
we will work on the snapshot ID of the last visit to Alambic, as returned
by the previous command we executed.
-.. code:: python
+.. code-block:: python
- # Store snapshot id
- snapshot = found[0]['snapshot']
- print(f"Snapshot is {format(snapshot)}.")
+ # Store snapshot id
+ snapshot = found[0]['snapshot']
+ print(f"Snapshot is {format(snapshot)}.")
-.. code::
+.. code-block:: console
- Snapshot is 6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc.
+ Snapshot is 6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc.
Note that the latest visit to the repository can also be directly retrieved using the
@@ -338,56 +343,56 @@ commits in a git context), which themselves point to the set of directories and
the branch at the time of analysis. Let’s follow this chain of links, starting with the
snapshot’s list of revisions (branches):
-.. code:: python
+.. code-block:: python
- snapshotr = requests.get("https://archive.softwareheritage.org/api/1/snapshot/{}/".format(snapshot))
- snapshotj = snapshotr.json()
- jprint(snapshotj)
+ snapshotr = requests.get("https://archive.softwareheritage.org/api/1/snapshot/{}/".format(snapshot))
+ snapshotj = snapshotr.json()
+ jprint(snapshotj)
-.. code::
+.. code-block:: json
- {
- "branches": {
- "HEAD": {
- "target": "refs/heads/master",
- "target_type": "alias",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
- },
- "refs/heads/devel": {
- "target": "e298b8c5692b18928013a68e41fd185419515075",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/e298b8c5692b18928013a68e41fd185419515075/"
- },
- "refs/heads/features/cr152_anonymise_data": {
- "target": "ba3e0dcbfa0cb212a7186e9e62efb6dafe7fe162",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/ba3e0dcbfa0cb212a7186e9e62efb6dafe7fe162/"
- },
- "refs/heads/features/cr164_github_project": {
- "target": "0005abb080e4c67a97533ee923e9d28142877752",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/0005abb080e4c67a97533ee923e9d28142877752/"
- },
- "refs/heads/features/cr165_github_its": {
- "target": "0005abb080e4c67a97533ee923e9d28142877752",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/0005abb080e4c67a97533ee923e9d28142877752/"
- },
- "refs/heads/features/cr89_gitlabwizard": {
- "target": "b941fd5f93a6cfc2349358b891e47d0fffe0ed2d",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/b941fd5f93a6cfc2349358b891e47d0fffe0ed2d/"
- },
- "refs/heads/master": {
- "target": "6dd0504b43b4459d52e9f13f71a91cc0fc445a19",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
- }
- },
- "id": "6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc",
- "next_branch": null
- }
+ {
+ "branches": {
+ "HEAD": {
+ "target": "refs/heads/master",
+ "target_type": "alias",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
+ },
+ "refs/heads/devel": {
+ "target": "e298b8c5692b18928013a68e41fd185419515075",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/e298b8c5692b18928013a68e41fd185419515075/"
+ },
+ "refs/heads/features/cr152_anonymise_data": {
+ "target": "ba3e0dcbfa0cb212a7186e9e62efb6dafe7fe162",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/ba3e0dcbfa0cb212a7186e9e62efb6dafe7fe162/"
+ },
+ "refs/heads/features/cr164_github_project": {
+ "target": "0005abb080e4c67a97533ee923e9d28142877752",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/0005abb080e4c67a97533ee923e9d28142877752/"
+ },
+ "refs/heads/features/cr165_github_its": {
+ "target": "0005abb080e4c67a97533ee923e9d28142877752",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/0005abb080e4c67a97533ee923e9d28142877752/"
+ },
+ "refs/heads/features/cr89_gitlabwizard": {
+ "target": "b941fd5f93a6cfc2349358b891e47d0fffe0ed2d",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/b941fd5f93a6cfc2349358b891e47d0fffe0ed2d/"
+ },
+ "refs/heads/master": {
+ "target": "6dd0504b43b4459d52e9f13f71a91cc0fc445a19",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
+ }
+ },
+ "id": "6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc",
+ "next_branch": null
+ }
Get the root directory
@@ -399,49 +404,52 @@ corresponding link in the ``target_url`` attribute. We will follow the
this case (a git repository) the revision is equivalent to a commit, with
an ID and message.
-.. code:: python
+.. code-block:: python
- print(f"Revision ID is {snapshotj['id']}.")
- master_url = snapshotj['branches']['refs/heads/master']['target_url']
- masterr = requests.get(master_url)
- masterj = masterr.json()
- jprint(masterj)
+ print(f"Revision ID is {snapshotj['id']}.")
+ master_url = snapshotj['branches']['refs/heads/master']['target_url']
+ masterr = requests.get(master_url)
+ masterj = masterr.json()
+ jprint(masterj)
-.. code::
+.. code-block::
- Revision ID is 6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc
- {
- "author": {
- "email": "boris.baldassari@gmail.com",
- "fullname": "Boris Baldassari <boris.baldassari@gmail.com>",
- "name": "Boris Baldassari"
- },
- "committer": {
- "email": "boris.baldassari@gmail.com",
- "fullname": "Boris Baldassari <boris.baldassari@gmail.com>",
- "name": "Boris Baldassari"
- },
- "committer_date": "2020-11-01T12:55:13+01:00",
- "date": "2020-11-01T12:55:13+01:00",
- "directory": "fd9fe3477db3b9b7dea63509832b3fa99bdd7eb8",
- "directory_url": "https://archive.softwareheritage.org/api/1/directory/fd9fe3477db3b9b7dea63509832b3fa99bdd7eb8/",
- "extra_headers": [],
- "history_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/log/",
- "id": "6dd0504b43b4459d52e9f13f71a91cc0fc445a19",
- "merge": false,
- "message": "#163 Fix dygraphs zero padding in forums plugin.\n",
- "metadata": {},
- "parents": [
- {
- "id": "a4a2d8925c1cc43612602ac28e4ca9a31728b151",
- "url": "https://archive.softwareheritage.org/api/1/revision/a4a2d8925c1cc43612602ac28e4ca9a31728b151/"
- }
- ],
- "synthetic": false,
- "type": "git",
- "url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
- }
+ Revision ID is 6436d2c9b06cf9bd9efb0b4e463c3fe6b868eadc
+
+.. code-block:: json
+
+ {
+ "author": {
+ "email": "boris.baldassari@gmail.com",
+ "fullname": "Boris Baldassari <boris.baldassari@gmail.com>",
+ "name": "Boris Baldassari"
+ },
+ "committer": {
+ "email": "boris.baldassari@gmail.com",
+ "fullname": "Boris Baldassari <boris.baldassari@gmail.com>",
+ "name": "Boris Baldassari"
+ },
+ "committer_date": "2020-11-01T12:55:13+01:00",
+ "date": "2020-11-01T12:55:13+01:00",
+ "directory": "fd9fe3477db3b9b7dea63509832b3fa99bdd7eb8",
+ "directory_url": "https://archive.softwareheritage.org/api/1/directory/fd9fe3477db3b9b7dea63509832b3fa99bdd7eb8/",
+ "extra_headers": [],
+ "history_url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/log/",
+ "id": "6dd0504b43b4459d52e9f13f71a91cc0fc445a19",
+ "merge": false,
+ "message": "#163 Fix dygraphs zero padding in forums plugin.\n",
+ "metadata": {},
+ "parents": [
+ {
+ "id": "a4a2d8925c1cc43612602ac28e4ca9a31728b151",
+ "url": "https://archive.softwareheritage.org/api/1/revision/a4a2d8925c1cc43612602ac28e4ca9a31728b151/"
+ }
+ ],
+ "synthetic": false,
+ "type": "git",
+ "url": "https://archive.softwareheritage.org/api/1/revision/6dd0504b43b4459d52e9f13f71a91cc0fc445a19/"
+ }
The revision references the root directory of the project. We can list all files and
@@ -454,7 +462,7 @@ following syntax:
The structure of the response is an **array of directory entries**.
**Content entries** are represented like this:
-::
+.. code-block:: json
{
"checksums": {
@@ -474,7 +482,7 @@ The structure of the response is an **array of directory entries**.
And **directory entries** are represented with:
-::
+.. code-block:: console
{
"dir_id": "3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200",
@@ -489,32 +497,32 @@ And **directory entries** are represented with:
We will print the list of contents and directories located at the root of
the repository at the time of analysis:
-.. code:: python
+.. code-block:: python
- root_url = masterj['directory_url']
- rootr = requests.get(root_url)
- rootj = rootr.json()
- for f in rootj:
- print(f"- {f['name']}.")
+ root_url = masterj['directory_url']
+ rootr = requests.get(root_url)
+ rootj = rootr.json()
+ for f in rootj:
+ print(f"- {f['name']}.")
-.. code::
+.. code-block:: console
- - .dockerignore
- - .env
- - .gitignore
- - CODE_OF_CONDUCT.html
- - CODE_OF_CONDUCT.md
- - LICENCE.html
- - LICENCE.md
- - Readme.md
- - doc
- - docker
- - docker-compose.run.yml
- - docker-compose.test.yml
- - dockercfg.encrypted
- - mojo
- - resources
+ - .dockerignore
+ - .env
+ - .gitignore
+ - CODE_OF_CONDUCT.html
+ - CODE_OF_CONDUCT.md
+ - LICENCE.html
+ - LICENCE.md
+ - Readme.md
+ - doc
+ - docker
+ - docker-compose.run.yml
+ - docker-compose.test.yml
+ - dockercfg.encrypted
+ - mojo
+ - resources
We could follow the links up (or down) to the leaves in order to rebuild
@@ -550,23 +558,23 @@ job result and download the archive. See the `Software Heritage documentation
In this example we will fetch the content of the root directory that we
previously identified.
-.. code:: python
+.. code-block:: python
- mealr = requests.post("https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/")
- mealj = mealr.json()
- jprint(mealj)
+ mealr = requests.post("https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/")
+ mealj = mealr.json()
+ jprint(mealj)
-.. code::
+.. code-block:: json
- {
- "fetch_url": "https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/raw/",
- "id": 379321799,
- "obj_id": "3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200",
- "obj_type": "directory",
- "progress_message": null,
- "status": "done"
- }
+ {
+ "fetch_url": "https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/raw/",
+ "id": 379321799,
+ "obj_id": "3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200",
+ "obj_type": "directory",
+ "progress_message": null,
+ "status": "done"
+ }
Ask if it’s ready
@@ -575,23 +583,23 @@ Ask if it’s ready
We can use a GET request on the same URL to get information about the
process status:
-.. code:: python
+.. code-block:: python
- statusr = requests.get("https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/")
- statusj = statusr.json()
- jprint(statusj)
+ statusr = requests.get("https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/")
+ statusj = statusr.json()
+ jprint(statusj)
-.. code::
+.. code-block::
- {
- "fetch_url": "https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/raw/",
- "id": 379321799,
- "obj_id": "3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200",
- "obj_type": "directory",
- "progress_message": null,
- "status": "done"
- }
+ {
+ "fetch_url": "https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/raw/",
+ "id": 379321799,
+ "obj_id": "3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200",
+ "obj_type": "directory",
+ "progress_message": null,
+ "status": "done"
+ }
Get the plate
@@ -601,7 +609,7 @@ Once the processing is finished (it can take up to a few minutes) the
tar.gz archive can be downloaded through the ``fetch_url`` link, and
extracted as a tar.gz archive:
-::
+.. code-block:: console
boris@castalia:downloads$ curl https://archive.softwareheritage.org/api/1/vault/directory/3ee1366c6dd0b7f4ba9536e9bcc300236ac8f200/raw/ -o myarchive.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
@@ -632,6 +640,3 @@ its API**: searching for a repository, identifying projects and downloading spec
snapshots of a repository. There is a lot more to the Archive and its API than what we
have seen, and all features are generously documented on the :swh_web:`Software Heritage
web site <api/>`.
-
-
-
diff --git a/docs/devel/tutorials/add-new-package.rst b/docs/devel/tutorials/add-new-package.rst
index ff19d0a710bd5f3b45fb08ef1a00d35581db4681..bba9383b1d2f438ee19cb4b777d11d290ee5d66e 100644
--- a/docs/devel/tutorials/add-new-package.rst
+++ b/docs/devel/tutorials/add-new-package.rst
@@ -27,13 +27,13 @@ The following commands need to run from the base directory
1. Use ``bin/init-py-repo`` to initialize the repository with a project
template and create the corresponding gitlab project:
- .. code:: bash
+ .. code-block:: console
bin/init-py-repo swh-foo
2. Install the pre-commit hook:
- .. code:: bash
+ .. code-block:: console
pre-commit install
@@ -80,7 +80,7 @@ Make an initial release
Releases are made automatically by Jenkins when a tag is pushed to a module
repository. Making an initial release is thus done by doing:
-.. code:: bash
+.. code-block:: console
git tag v0.0.0
git push origin --tags v0.0.0
@@ -107,7 +107,7 @@ To add a new module to the documentation:
- Add the package with a concise description to the index of the development part, located in
``docs/devel/index.rst``.
- ::
+ .. code-block:: rst
:ref:`swh.foo <swh-foo>`
short description of the repository
diff --git a/docs/sysadm/puppet/howto-manage-third-party-modules.rst b/docs/sysadm/puppet/howto-manage-third-party-modules.rst
index be68a3b41fabf4d278f197652a3892d31df54730..1bf66f47f57fd37d7f660b550d63d4443015278b 100644
--- a/docs/sysadm/puppet/howto-manage-third-party-modules.rst
+++ b/docs/sysadm/puppet/howto-manage-third-party-modules.rst
@@ -29,12 +29,6 @@ specifier.
Adding a new external puppet module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-.. admonition:: Deprecation notice
- :class: important
-
- This needs refactoring as the script mentioned in this section is not yet ported to
- use our new gitlab instance
-
In the *puppet-environment* repository, the ``bin/import-puppet-module`` takes care of
the following tasks:
@@ -49,10 +43,9 @@ the following tasks:
To be able to use the script, you need:
- Be a member of the `System Administrators
- <https://forge.softwareheritage.org/project/members/7/>`_ Phabricator group
-- Have the :ref:`Arcanist <arcanist-configuration>` API key setup
-- A pair of python dependencies: ``python3-phabricator`` and ``python3-requests`` (pull
- them from testing if needed).
+ <https://gitlab.softwareheritage.org/groups/teams/sysadmin>`_ gitlab group
+- Have the gitlab API key setup
+- A few python dependencies: ``python3-click``, ``python3-gitlab`` and ``python3-requests``.
Example usage to pull the `elastic/elasticsearch
<https://forge.puppetlabs.com/elastic/elasticsearch>`_ module