Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • lunar/swh-deposit
  • anlambert/swh-deposit
  • swh/devel/swh-deposit
  • douardda/swh-deposit
  • ardumont/swh-deposit
  • marmoute/swh-deposit
  • rboyer/swh-deposit
7 results
Show changes
Hide whitespace changes
Inline Side-by-side
Showing
with 463 additions and 458 deletions
docs/cli.rst 0 → 100644
View file @ d2456bdf
.. _swh-deposit-cli:
Command-line interface
======================
Shared command-line interface
-----------------------------
.. click:: swh.deposit.cli:deposit
:prog: swh deposit
:nested: short
Administration utilities
------------------------
.. click:: swh.deposit.cli.admin:admin
:prog: swh deposit admin
:nested: full
.. _swh-deposit-cli-client:
Deposit client tools
--------------------
.. click:: swh.deposit.cli.client:upload
:prog: swh deposit
:nested: full
.. click:: swh.deposit.cli.client:status
:prog: swh deposit
:nested: full
.. click:: swh.deposit.cli.client:metadata_only
:prog: swh deposit
:nested: full
docs/conf.py
View file @ d2456bdf
import os import os
import django import django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", os.environ.setdefault("DJANGO_SETTINGS_MODULE", "swh.deposit.settings.development")
"swh.deposit.settings.development")
django.setup() django.setup()
from swh.docs.sphinx.conf import * # NoQA from swh.docs.sphinx.conf import * # NoQA
docs/endpoints/collection.rst
View file @ d2456bdf
.. _API-create-deposit:
Create deposit Create deposit
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
.. http:post:: /1/<collection-name>/ .. http:post:: /1/(str:collection-name)/
Create deposit in a collection. Create deposit in a collection which name is ``collection-name``.
The client sends a deposit request to a specific collection with: The client sends a deposit request to a specific collection with:
...@@ -13,61 +15,74 @@ Create deposit ...@@ -13,61 +15,74 @@ Create deposit
Also known as: COL-IRI Also known as: COL-IRI
:param text <name><pass>: the client's credentials **Example query**:
:param text Content-Type: accepted mimetype
:param int Content-Length: tarball size
:param text Content-MD5: md5 checksum hex encoded of the tarball
:param text Content-Disposition: attachment; filename=[filename]; the filename
parameter must be text (ascii)
:param text Content-Disposition: for the metadata file set name parameter
to 'atom'.
:param bool In-progress: true if not final; false when final request.
:statuscode 201: success for deposit on POST
:statuscode 401: Unauthorized
:statuscode 404: access to an unknown collection
:statuscode 415: unsupported media type
Sample request .. code:: shell
~~~~~~~~~~~~~~~
.. code:: shell
curl -i -u hal:<pass> \ curl -i -u hal:<pass> \
-F "file=@../deposit.json;type=application/zip;filename=payload" \ -F "file=@deposit.json;type=application/zip;filename=payload" \
-F "atom=@../atom-entry.xml;type=application/atom+xml;charset=UTF-8" \ -F "atom=@atom-entry.xml;type=application/atom+xml;charset=UTF-8" \
-H 'In-Progress: false' \ -H 'In-Progress: false' \
-H 'Slug: some-external-id' \
-XPOST https://deposit.softwareheritage.org/1/hal/ -XPOST https://deposit.softwareheritage.org/1/hal/
Sample response .. code:: http
~~~~~~~~~~~~~~~
POST /1/hal/ HTTP/1.1
.. code:: shell Host: deposit.softwareheritage.org
Authorization: Basic xxxxxxxxxxxx=
HTTP/1.0 201 Created In-Progress: false
Date: Tue, 26 Sep 2017 10:32:35 GMT Content-Length: 123456
Server: WSGIServer/0.2 CPython/3.5.3 Content-Type: multipart/form-data; boundary=----------------------123456798
Vary: Accept, Cookie
Allow: GET, POST, PUT, DELETE, HEAD, OPTIONS **Example response**:
Location: /1/hal/10/metadata/
X-Frame-Options: SAMEORIGIN .. code:: http
Content-Type: application/xml
HTTP/1.1 201 Created
<entry xmlns="http://www.w3.org/2005/Atom" Date: Tue, 26 Sep 2017 10:32:35 GMT
xmlns:sword="http://purl.org/net/sword/" Server: WSGIServer/0.2 CPython/3.5.3
xmlns:dcterms="http://purl.org/dc/terms/"> Vary: Accept, Cookie
<deposit_id>10</deposit_id> Allow: GET, POST, PUT, DELETE, HEAD, OPTIONS
<deposit_date>Sept. 26, 2017, 10:32 a.m.</deposit_date> Location: /1/hal/10/metadata/
<deposit_archive>None</deposit_archive> X-Frame-Options: SAMEORIGIN
<deposit_status>deposited</deposit_status> Content-Type: application/xml
<!-- Edit-IRI --> <entry xmlns="http://www.w3.org/2005/Atom"
<link rel="edit" href="/1/hal/10/metadata/" /> xmlns:sword="http://purl.org/net/sword/"
<!-- EM-IRI --> xmlns:dcterms="http://purl.org/dc/terms/"
<link rel="edit-media" href="/1/hal/10/media/"/> xmlns:swhdeposit="https://www.softwareheritage.org/schema/2018/deposit"
<!-- SE-IRI --> >
<link rel="http://purl.org/net/sword/terms/add" href="/1/hal/10/metadata/" /> <swhdeposit:deposit_id>10</swhdeposit:deposit_id>
<!-- State-IRI --> <swhdeposit:deposit_date>Sept. 26, 2017, 10:32 a.m.</swhdeposit:deposit_date>
<link rel="alternate" href="/1/<collection-name>/10/status/"/> <swhdeposit:deposit_archive>None</swhdeposit:deposit_archive>
<swhdeposit:deposit_status>deposited</swhdeposit:deposit_status>
<sword:packaging>http://purl.org/net/sword/package/SimpleZip</sword:packaging>
</entry> <!-- Edit-IRI -->
<link rel="edit" href="/1/hal/10/metadata/" />
<!-- EM-IRI -->
<link rel="edit-media" href="/1/hal/10/media/"/>
<!-- SE-IRI -->
<link rel="http://purl.org/net/sword/terms/add" href="/1/hal/10/metadata/" />
<!-- State-IRI -->
<link rel="alternate" href="/1/hal/10/status/"/>
<sword:packaging>http://purl.org/net/sword/package/SimpleZip</sword:packaging>
</entry>
Note: older versions of the deposit used the ``http://www.w3.org/2005/Atom``
namespace instead of ``https://www.softwareheritage.org/schema/2018/deposit``.
Tags in the Atom namespace are still provided for backward compatibility, but
are deprecated.
:reqheader Authorization: Basic authentication token
:reqheader Content-Type: accepted mimetype
:reqheader Content-Length: tarball size
:reqheader Content-MD5: md5 checksum hex encoded of the tarball
:reqheader Content-Disposition: attachment; filename=[filename]; the filename
parameter must be text (ascii); for the metadata file set name parameter
to 'atom'.
:reqheader In-progress: ``true`` if not final; ``false`` when final request.
:statuscode 201: success for deposit on POST
:statuscode 401: Unauthorized
:statuscode 404: access to an unknown collection
:statuscode 415: unsupported media type
docs/endpoints/content.rst
View file @ d2456bdf
Display content Display content
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
.. http:get:: /1/<collection-name>/<deposit-id>/content/ .. http:get:: /1/(str:collection-name)/(int:deposit-id)/content/
Display information on the content's representation in the sword Display information on the content's representation in the sword
server. server.
...@@ -9,6 +9,48 @@ Display content ...@@ -9,6 +9,48 @@ Display content
Also known as: CONT-FILE-IRI Also known as: CONT-FILE-IRI
:param text <name><pass>: the client's credentials **Example query**:
.. code:: http
GET /deposit/1/test/1/content/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic xxxxxxxxxx
Connection: keep-alive
Host: deposit.softwareheritage.org
**Example response**:
.. code:: http
HTTP/1.1 200 OK
Allow: GET, POST, PUT, DELETE, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 1760
Content-Type: application/xml
Date: Thu, 05 Nov 2020 14:31:50 GMT
Server: nginx/1.19.2
Vary: Accept
X-Frame-Options: SAMEORIGIN
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:sword="http://purl.org/net/sword/"
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:swhdeposit="https://www.softwareheritage.org/schema/2018/deposit"
>
<swhdeposit:deposit_id>1</swhdeposit:deposit_id>
<swhdeposit:deposit_date>Oct. 28, 2020, 3:58 p.m.</swhdeposit:deposit_date>
<swhdeposit:deposit_status>done</swhdeposit:deposit_status>
<swhdeposit:deposit_status_detail>The deposit has been successfully loaded into the Software Heritage archive</swhdeposit:deposit_status_detail>
</entry>
Note: older versions of the deposit used the ``http://www.w3.org/2005/Atom``
namespace instead of ``https://www.softwareheritage.org/schema/2018/deposit``.
Tags in the Atom namespace are still provided for backward compatibility, but
are deprecated.
:reqheader Authorization: Basic authentication token
:statuscode 200: no error :statuscode 200: no error
:statuscode 401: Unauthorized :statuscode 401: Unauthorized
docs/endpoints/service-document.rst
View file @ d2456bdf
...@@ -11,38 +11,54 @@ Service document ...@@ -11,38 +11,54 @@ Service document
Also known as: SD-IRI - The Service Document IRI Also known as: SD-IRI - The Service Document IRI
:param text <name><pass>: the client's credentials **Example query**:
:statuscode 200: no error
:statuscode 401: Unauthorized .. code:: http
GET /1/servicedocument/ HTTP/1.1
Host: deposit.softwareheritage.org
Authorization: Basic xxxxxxxxxxxx=
**Example response**:
.. code:: http
HTTP/1.1 200 OK
Allow: GET, POST, PUT, DELETE, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 1247
Content-Type: application/xml
Date: Thu, 05 Nov 2020 14:27:05 GMT
Server: nginx/1.19.2
Vary: Accept
X-Frame-Options: SAMEORIGIN
Sample response <?xml version="1.0" ?>
~~~~~~~~~~~~~~~ <service xmlns:dcterms="http://purl.org/dc/terms/"
.. code:: xml xmlns:sword="http://purl.org/net/sword/terms/"
xmlns:atom="http://www.w3.org/2005/Atom"
<?xml version="1.0" ?> xmlns="http://www.w3.org/2007/app">
<service xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:sword="http://purl.org/net/sword/terms/" <sword:version>2.0</sword:version>
xmlns:atom="http://www.w3.org/2005/Atom" <sword:maxUploadSize>20971520</sword:maxUploadSize>
xmlns="http://www.w3.org/2007/app">
<workspace>
<sword:version>2.0</sword:version> <atom:title>The Software Heritage (SWH) archive</atom:title>
<sword:maxUploadSize>20971520</sword:maxUploadSize> <collection href="https://deposit.softwareherigage.org/1/hal/">
<atom:title>SWH Software Archive</atom:title>
<workspace> <accept>application/zip</accept>
<atom:title>The Software Heritage (SWH) archive</atom:title> <accept>application/x-tar</accept>
<collection href="https://deposit.softwareherigage.org/1/hal/"> <sword:collectionPolicy>Collection Policy</sword:collectionPolicy>
<atom:title>SWH Software Archive</atom:title> <dcterms:abstract>Software Heritage Archive</dcterms:abstract>
<accept>application/zip</accept> <sword:mediation>false</sword:mediation>
<accept>application/x-tar</accept> <sword:metadataRelevantHeader>false</sword:metadataRelevantHeader>
<sword:collectionPolicy>Collection Policy</sword:collectionPolicy> <sword:treatment>Collect, Preserve, Share</sword:treatment>
<dcterms:abstract>Software Heritage Archive</dcterms:abstract> <sword:acceptPackaging>http://purl.org/net/sword/package/SimpleZip</sword:acceptPackaging>
<sword:mediation>false</sword:mediation> <sword:service>https://deposit.softwareheritage.org/1/hal/</sword:service>
<sword:metadataRelevantHeader>false</sword:metadataRelevantHeader> </collection>
<sword:treatment>Collect, Preserve, Share</sword:treatment> </workspace>
<sword:acceptPackaging>http://purl.org/net/sword/package/SimpleZip</sword:acceptPackaging> </service>
<sword:service>https://deposit.softwareheritage.org/1/hal/</sword:service>
</collection> :reqheader Authorization: Basic authentication token
</workspace> :statuscode 200: no error
</service> :statuscode 401: Unauthorized
docs/endpoints/status.rst
View file @ d2456bdf
Retrieve status Retrieve status
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
.. http:get:: /1/<collection-name>/<deposit-id>/ .. http:get:: /1/(str:collection-name)/(int:deposit-id)/status/
Returns deposit's status. Returns deposit's status.
...@@ -17,7 +17,52 @@ Retrieve status ...@@ -17,7 +17,52 @@ Retrieve status
Also known as STATE-IRI Also known as STATE-IRI
:param text <name><pass>: the client's credentials
**Example query**:
.. code:: http
GET /1/hal/1/status/ HTTP/1.1
Host: deposit.softwareheritage.org
Authorization: Basic xxxxxxxxxxxx=
**Example successful deposit response**:
.. code:: xml
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:sword="http://purl.org/net/sword/"
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:swhdeposit="https://www.softwareheritage.org/schema/2018/deposit"
>
<swhdeposit:deposit_id>160</swhdeposit:deposit_id>
<swhdeposit:deposit_status>done</swhdeposit:deposit_status>
<swhdeposit:deposit_status_detail>The deposit has been successfully loaded into the Software Heritage archive</swhdeposit:deposit_status_detail>
<swhdeposit:deposit_swh_id>swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9</swhdeposit:deposit_swh_id>
<swhdeposit:deposit_swh_id_context>swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/;visit=swh:1:snp:68c0d26104d47e278dd6be07ed61fafb561d0d20;anchor=swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;path=/</swhdeposit:deposit_swh_id>
</entry>
**Example rejeced deposit response**:
.. code:: xml
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:sword="http://purl.org/net/sword/"
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:swhdeposit="https://www.softwareheritage.org/schema/2018/deposit"
>
<swhdeposit:deposit_id>148</swhdeposit:deposit_id>
<swhdeposit:deposit_status>rejected</swhdeposit:deposit_status>
<swhdeposit:deposit_status_detail>- At least one url field must be compatible with the client&#39;s domain name (codemeta:url)</swhdeposit:deposit_status_detail>
</entry>
Note: older versions of the deposit used the ``http://www.w3.org/2005/Atom``
namespace instead of ``https://www.softwareheritage.org/schema/2018/deposit``.
Tags in the Atom namespace are still provided for backward compatibility, but
are deprecated.
:reqheader Authorization: Basic authentication token
:statuscode 201: with the deposit's status :statuscode 201: with the deposit's status
:statuscode 401: Unauthorized :statuscode 401: Unauthorized
:statuscode 404: access to an unknown deposit :statuscode 404: access to an unknown deposit
...@@ -42,35 +87,3 @@ Many reasons are possibles, here are some: ...@@ -42,35 +87,3 @@ Many reasons are possibles, here are some:
- Deposit with unsupported archive format - Deposit with unsupported archive format
- Deposit with missing metadata - Deposit with missing metadata
Sample response
~~~~~~~~~~~~~~~
Successful deposit:
.. code:: xml
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:sword="http://purl.org/net/sword/"
xmlns:dcterms="http://purl.org/dc/terms/">
<deposit_id>160</deposit_id>
<deposit_status>done</deposit_status>
<deposit_status_detail>The deposit has been successfully loaded into the Software Heritage archive</deposit_status_detail>
<deposit_swh_id>swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9</deposit_swh_id>
<deposit_swh_id_context>swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/</deposit_swh_id>
<deposit_swh_anchor_id>swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb</deposit_swh_id>
<deposit_swh_anchor_id_context>swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/</deposit_swh_id>
</entry>
Rejected deposit:
.. code:: xml
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:sword="http://purl.org/net/sword/"
xmlns:dcterms="http://purl.org/dc/terms/">
<deposit_id>148</deposit_id>
<deposit_status>rejected</deposit_status>
<deposit_status_detail>- At least one url field must be compatible with the client&#39;s domain name (codemeta:url)</deposit_status_detail>
</entry>
docs/endpoints/update-media.rst
View file @ d2456bdf
Update content Update content
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
.. http:post:: /1/<collection-name>/<deposit-id>/media/ .. http:post:: /1/(str:collection-name)/(int:deposit-id)/media/
Add archive(s) to a deposit. Only possible if the deposit's status Add archive(s) to a deposit. Only possible if the deposit's status
is partial. is partial.
.. http:put:: /1/<collection-name>/<deposit-id>/media/ .. http:put:: /1/(str:collection-name)/(int:deposit-id)/media/
Replace all content by submitting a new archive. Only possible if Replace all content by submitting a new archive. Only possible if
the deposit's status is partial. the deposit's status is partial.
...@@ -14,13 +14,13 @@ Update content ...@@ -14,13 +14,13 @@ Update content
Also known as: *update iri* (EM-IRI) Also known as: *update iri* (EM-IRI)
:param text <name><pass>: the client's credentials :reqheader Authorization: Basic authentication token
:param text Content-Type: accepted mimetype :reqheader Content-Type: accepted mimetype
:param int Content-Length: tarball size :reqheader Content-Length: tarball size
:param text Content-MD5: md5 checksum hex encoded of the tarball :reqheader Content-MD5: md5 checksum hex encoded of the tarball
:param text Content-Disposition: attachment; filename=[filename] ; the filename :reqheader Content-Disposition: attachment; filename=[filename] ; the filename
parameter must be text (ascii) parameter must be text (ascii)
:param bool In-progress: true if not final; false when final request. :reqheader In-progress: ``true`` if not final; ``false`` when final request.
:statuscode 204: success without payload on PUT :statuscode 204: success without payload on PUT
:statuscode 201: success for deposit on POST :statuscode 201: success for deposit on POST
:statuscode 401: Unauthorized :statuscode 401: Unauthorized
......
docs/endpoints/update-metadata.rst
View file @ d2456bdf
Update metadata Update metadata
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
.. http:post:: /1/<collection-name>/<deposit-id>/metadata/ .. http:post:: /1/(str:collection-name)/(int:deposit-id)/metadata/
Add metadata to a deposit. Only possible if the deposit's status Add metadata to a deposit. Only possible if the deposit's status
is partial. is partial.
.. http:put:: /1/<collection-name>/<deposit-id>/metadata/ .. http:put:: /1/(str:collection-name)/(int:deposit-id)/metadata/
Replace all metadata by submitting a new metadata file. Only possible if Replace all metadata by submitting a new metadata file. Only possible if
the deposit's status is partial. the deposit's status is partial.
...@@ -14,10 +14,10 @@ Update metadata ...@@ -14,10 +14,10 @@ Update metadata
Also known as: *update iri* (SE-IRI) Also known as: *update iri* (SE-IRI)
:param text <name><pass>: the client's credentials :reqheader Authorization: Basic authentication token
:param text Content-Disposition: attachment; filename=[filename] ; the filename :reqheader Content-Disposition: attachment; filename=[filename] ; the filename
parameter must be text (ascii), with a name parameter set to 'atom'. parameter must be text (ascii), with a name parameter set to 'atom'.
:param bool In-progress: true if not final; false when final request. :reqheader In-progress: `true` if not final; `false` when final request.
:statuscode 204: success without payload on PUT :statuscode 204: success without payload on PUT
:statuscode 201: success for deposit on POST :statuscode 201: success for deposit on POST
:statuscode 401: Unauthorized :statuscode 401: Unauthorized
......
docs/getting-started.rst deleted 100644 → 0
View file @ 7351df1c
Getting Started
===============
This is a guide for how to prepare and push a software deposit with
the `swh deposit` commands.
The API is rooted at https://deposit.softwareheritage.org/1.
For more details, see the `main documentation <./index.html>`__.
Requirements
------------
You need to be referenced on SWH's client list to have:
* credentials (needed for the basic authentication step)
- in this document we reference ``<name>`` as the client's name and
``<pass>`` as its associated authentication password.
* an associated collection_.
.. _collection: https://bitworking.org/projects/atom/rfc5023#rfc.section.8.3.3
`Contact us for more information.
<https://www.softwareheritage.org/contact/>`__
Prepare a deposit
-----------------
* compress the files in a supported archive format:
- zip: common zip archive (no multi-disk zip files).
- tar: tar archive without compression or optionally any of the
following compression algorithm gzip (`.tar.gz`, `.tgz`), bzip2
(`.tar.bz2`) , or lzma (`.tar.lzma`)
* prepare a metadata file (`more details <./metadata.html>`__.):
- specify metadata schema/vocabulary (CodeMeta is strongly recommended)
- specify *MUST* metadata (url, authors, software name and
the external\_identifier)
- add all available information under the compatible metadata term.
Here is an example of an atom entry file with CodeMeta terms:
.. code:: xml
<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0">
<title>Je suis GPL</title>
<client>swh</client>
<external_identifier>je-suis-gpl</external_identifier>
<codemeta:url>https://forge.softwareheritage.org/source/jesuisgpl/</codemeta:url>
<codemeta:dateCreated>2018-01-05</codemeta:dateCreated>
<codemeta:description>Je suis GPL is a modified version of GNU Hello whose
sole purpose is to showcase the usage of
Software Heritage for license compliance purposes.</codemeta:description>
<codemeta:version>0.1</codemeta:version>
<codemeta:runtimePlatform>GNU/Linux</codemeta:runtimePlatform>
<codemeta:developmentStatus>stable</codemeta:developmentStatus>
<codemeta:programmingLanguage>C</codemeta:programmingLanguage>
<codemeta:license>
<codemeta:name>GNU General Public License v3.0 or later</codemeta:name>
<codemeta:url>https://spdx.org/licenses/GPL-3.0-or-later.html</codemeta:url>
</codemeta:license>
<codemeta:author>
<codemeta:name>Stefano Zacchiroli</codemeta:name>
<codemeta:jobTitle>Maintainer</codemeta:jobTitle>
</codemeta:author>
</entry>
Push deposit
------------
You can push a deposit with:
* a single deposit (archive + metadata):
The user posts in one query a software
source code archive and associated metadata.
The deposit is directly marked with status ``deposited``.
* a multisteps deposit:
1. Create an incomplete deposit (marked with status ``partial``)
2. Add data to a deposit (in multiple requests if needed)
3. Finalize deposit (the status becomes ``deposited``)
Single deposit
^^^^^^^^^^^^^^
Once the files are ready for deposit, we want to do the actual deposit
in one shot, sending exactly one POST query:
* 1 archive (content-type ``application/zip`` or ``application/x-tar``)
* 1 metadata file in atom xml format (``content-type: application/atom+xml;type=entry``)
For this, we need to provide the:
* arguments: ``--username 'name' --password 'pass'`` as credentials
* archive's path (example: ``--archive path/to/archive-name.tgz``) :
* (optionally) metadata file's path ``--metadata
path/to/file.metadata.xml``. If not provided, the archive's filename
will be used to determine the metadata file, e.g:
``path/to/archive-name.tgz.metadata.xml``
* (optionally) ``--slug 'your-id'`` argument, a reference to a
unique identifier the client uses for the software object.
You can do this with the following command:
minimal deposit
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive je-suis-gpl.tgz
with client's external identifier (``slug``)
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive je-suis-gpl.tgz \
--slug je-suis-gpl
to a specific client's collection
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive je-suis-gpl.tgz \
--collection 'second-collection'
You just posted a deposit to your collection on Software Heritage
If everything went well, the successful response will contain the
elements below:
.. code:: shell
{
'deposit_status': 'deposited',
'deposit_id': '7',
'deposit_date': 'Jan. 29, 2018, 12:29 p.m.'
}
Note: As the deposit is in ``deposited`` status, you can no longer
update the deposit after this query. It will be answered with a 403
forbidden answer.
If something went wrong, an equivalent response will be given with the
`error` and `detail` keys explaining the issue, e.g.:
.. code:: shell
{
'error': 'Unknown collection name xyz',
'detail': None,
'deposit_status': None,
'deposit_status_detail': None,
'deposit_swh_id': None,
'status': 404
}
multisteps deposit
^^^^^^^^^^^^^^^^^^^^^^^^^
The steps to create a multisteps deposit:
1. Create an incomplete deposit
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First use the ``--partial`` argument to declare there is more to come
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive foo.tar.gz \
--partial
2. Add content or metadata to the deposit
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Continue the deposit by using the ``--deposit-id`` argument given as a response
for the first step. You can continue adding content or metadata while you use
the ``--partial`` argument.
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive add-foo.tar.gz \
--deposit-id 42 \
--partial
In case you want to add only one new archive without metadata:
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive add-foo.tar.gz \
--archive-deposit \
--deposit-id 42 \
--partial
If you want to add only metadata, use:
.. code:: shell
$ swh deposit upload --username name --password secret \
--metadata add-foo.tar.gz.metadata.xml \
--metadata-deposit \
--deposit-id 42 \
--partial
3. Finalize deposit
~~~~~~~~~~~~~~~~~~~
On your last addition, by not declaring it as ``--partial``, the
deposit will be considered as completed and its status will be changed
to ``deposited``:
.. code:: shell
$ swh deposit upload --username name --password secret \
--metadata add-foo.tar.gz.metadata.xml \
--metadata-deposit \
--deposit-id 42
Update deposit
----------------
* replace deposit:
- only possible if the deposit status is ``partial`` and
``--deposit-id <id>`` is provided
- by using the ``--replace`` flag
- ``--metadata-deposit`` replaces associated existing metadata
- ``--archive-deposit`` replaces associated archive(s)
- by default, with no flag or both, you'll replace associated
metadata and archive(s):
.. code:: shell
$ swh deposit upload --username name --password secret \
--deposit-id 11 \
--archive updated-je-suis-gpl.tgz \
--replace
* update a loaded deposit with a new version:
- by using the external-id with the ``--slug`` argument, you will
link the new deposit with its parent deposit:
.. code:: shell
$ swh deposit upload --username name --password secret \
--archive je-suis-gpl-v2.tgz \
--slug 'je-suis-gpl' \
Check the deposit's status
--------------------------
You can check the status of the deposit by using the ``--deposit-id`` argument:
.. code:: shell
$ swh deposit status --username name --password secret \
--deposit-id 11
.. code:: json
{
'deposit_id': '11',
'deposit_status': 'deposited',
'deposit_swh_id': None,
'deposit_status_detail': 'Deposit is ready for additional checks \
(tarball ok, metadata, etc...)'
}
The different statuses:
- **partial**: multipart deposit is still ongoing
- **deposited**: deposit completed
- **rejected**: deposit failed the checks
- **verified**: content and metadata verified
- **loading**: loading in-progress
- **done**: loading completed successfully
- **failed**: the deposit loading has failed
When the deposit has been loaded into the archive, the status will be
marked ``done``. In the response, will also be available the
<deposit_swh_id>, <deposit_swh_id_context>, <deposit_swh_anchor_id>,
<deposit_swh_anchor_id_context>. For example:
.. code:: json
{
'deposit_id': '11',
'deposit_status': 'done',
'deposit_swh_id': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9',
'deposit_swh_id_context': 'swh:1:dir:d83b7dda887dc790f7207608474650d4344b8df9;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
'deposit_swh_anchor_id': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb',
'deposit_swh_anchor_id_context': 'swh:1:rev:e76ea49c9ffbb7f73611087ba6e999b19e5d71eb;origin=https://forge.softwareheritage.org/source/jesuisgpl/',
'deposit_status_detail': 'The deposit has been successfully \
loaded into the Software Heritage archive'
}
docs/images/.gitignore 0 → 100644
View file @ d2456bdf
*.png
*.svg
docs/images/Makefile 0 → 100644
View file @ d2456bdf
UML_DIAGS_SRC = $(wildcard *.uml)
UML_DIAGS = $(patsubst %.uml,%.svg,$(UML_DIAGS_SRC))
all: $(UML_DIAGS)
%.svg: %.uml
DISPLAY="" plantuml -tsvg $<
clean:
-rm -f $(UML_DIAGS)
docs/images/deposit-authentication-basic.uml 0 → 100644
View file @ d2456bdf
@startuml
participant CLIENT as "SWORD client\n(eg. HAL)"
participant DEPOSIT as "swh-deposit"
participant AUTH_BACKEND as "deposit database"
activate CLIENT
activate DEPOSIT
activate AUTH_BACKEND
CLIENT ->> DEPOSIT: GET /1/<service-document>/
DEPOSIT ->> AUTH_BACKEND: check authentication
alt credentials mismatch or inexistent user
AUTH_BACKEND ->> DEPOSIT: return ko
DEPOSIT -->> CLIENT: return 401, Unauthorized
else credentials ok
AUTH_BACKEND ->> DEPOSIT: return deposit_client
DEPOSIT -->> CLIENT: return 200, <service-document>
end
deactivate CLIENT
deactivate DEPOSIT
deactivate AUTH_BACKEND
@enduml
docs/images/deposit-authentication-keycloak.uml 0 → 100644
View file @ d2456bdf
@startuml
participant CLIENT as "SWORD client\n(eg. HAL)"
participant DEPOSIT as "swh-deposit"
participant AUTH_BACKEND as "keycloak"
activate CLIENT
activate DEPOSIT
activate AUTH_BACKEND
CLIENT ->> DEPOSIT: GET /1/<service-document>/
DEPOSIT ->> AUTH_BACKEND: forwards authentication to keycloak
alt credentials mismatch or inexistent user
AUTH_BACKEND ->> DEPOSIT: return 401, Unauthorized
DEPOSIT -->> CLIENT: return 401, Unauthorized
else credentials ok
AUTH_BACKEND ->> DEPOSIT: return oidc_profile
DEPOSIT ->> DEPOSIT: decodes oidc_profile, checks deposit user permissions
alt no permission matches 'swh.deposit.api'
DEPOSIT -->> CLIENT: return 401, Unauthorized
else at least one permission matches 'swh.deposit.api'
DEPOSIT -->> CLIENT: return 200, <service-document>
end
end
deactivate CLIENT
deactivate DEPOSIT
deactivate AUTH_BACKEND
@enduml
docs/images/deposit-create-chart.png deleted 100644 → 0
View file @ 7351df1c
docs/images/deposit-create-chart.png

52.4 KiB

docs/images/deposit-create-chart.uml 0 → 100644
View file @ d2456bdf
@startuml
participant CLIENT as "SWORD client\n(eg. HAL)"
participant DEPOSIT as "swh-deposit"
participant DEPOSIT_DATABASE as "deposit database"
participant SCHEDULER as "swh-scheduler"
activate CLIENT
activate DEPOSIT
activate DEPOSIT_DATABASE
activate SCHEDULER
CLIENT ->> DEPOSIT: GET /1/<service-document>/
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok (if client exists and credentials ok)
DEPOSIT -->> CLIENT: return 200, <service-document>
CLIENT ->> DEPOSIT: POST /1/<collection-name>/
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok (if client exists and credentials ok)
DEPOSIT ->> DEPOSIT_DATABASE: create new deposit
DEPOSIT_DATABASE -->> DEPOSIT: return deposit_id
DEPOSIT ->> SCHEDULER: schedule load for <deposit_id>
DEPOSIT -->> CLIENT: return 201, <deposit receipt>
@enduml
docs/images/deposit-delete-chart.png deleted 100644 → 0
View file @ 7351df1c
docs/images/deposit-delete-chart.png

55.6 KiB

docs/images/deposit-delete-chart.uml 0 → 100644
View file @ d2456bdf
@startuml
participant CLIENT as "SWORD client\n(eg. HAL)"
participant DEPOSIT as "swh-deposit"
participant DEPOSIT_DATABASE as "deposit database"
activate CLIENT
activate DEPOSIT
activate DEPOSIT_DATABASE
CLIENT ->> DEPOSIT: POST /1/<collection-name>/\nHEADER In-Progress: true
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok (if client exists and credentials ok)
DEPOSIT -->> CLIENT: return 201, <deposit receipt>
CLIENT -> DEPOSIT: DELETE /1/<collection-name>/<deposit-id>/media/\nDELETE /1/<collection-name>/<deposit-id>/metadata/
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok
DEPOSIT ->> DEPOSIT_DATABASE: check inputs()
alt status is 'partial'
DEPOSIT_DATABASE -->> DEPOSIT: return ok
DEPOSIT ->> DEPOSIT_DATABASE: delete-deposit-or-deposit-archives()
DEPOSIT_DATABASE -->> DEPOSIT: return ok
DEPOSIT -->> CLIENT: return 204
else status is not 'partial'
DEPOSIT_DATABASE -->> DEPOSIT: return ko
DEPOSIT -->> CLIENT: return 400, "You can only act on deposit with status partial"
end
@enduml
docs/images/deposit-update-chart.png deleted 100644 → 0
View file @ 7351df1c
docs/images/deposit-update-chart.png

60.1 KiB

docs/images/deposit-update-chart.uml 0 → 100644
View file @ d2456bdf
@startuml
participant CLIENT as "SWORD client\n(eg. HAL)"
participant DEPOSIT as "swh-deposit"
participant DEPOSIT_DATABASE as "deposit database"
participant SCHEDULER as "swh-scheduler"
activate CLIENT
activate DEPOSIT
activate DEPOSIT_DATABASE
activate SCHEDULER
CLIENT ->> DEPOSIT: POST /1/<collection-name>/\nHEADER In-Progress: true
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok (if client exists and credentials ok)
DEPOSIT -->> CLIENT: return 201, <deposit receipt>
CLIENT -> DEPOSIT: POST/PUT /1/<collection-name>/<deposit-id>/media/\nPOST/PUT /1/<collection-name>/<deposit-id>/metadata/
DEPOSIT ->> DEPOSIT_DATABASE: check authentication
DEPOSIT_DATABASE -->> DEPOSIT: return ok
DEPOSIT ->> DEPOSIT_DATABASE: check inputs()
alt status is 'partial'
DEPOSIT_DATABASE -->> DEPOSIT: return ok
alt HEADER: In-Progress = true
DEPOSIT ->> DEPOSIT_DATABASE: add-or-replace-data-and-update-status('partial')
else HEADER: In-Progress = false
DEPOSIT ->> SCHEDULER: schedule load for <deposit_id>
DEPOSIT ->> DEPOSIT_DATABASE: add-or-replace-data-and-update-status('deposited')
end
DEPOSIT_DATABASE -->> DEPOSIT: return ok
DEPOSIT -->> CLIENT: return 204
else status is not partial
DEPOSIT_DATABASE -->> DEPOSIT: return ko
DEPOSIT -->> CLIENT: return 400, "You can only act on deposit with status partial"
end
@enduml
docs/images/deposit-workflow-checking.uml 0 → 100644
View file @ d2456bdf
@startuml
participant DEPOSIT as "deposit API"
participant DEPOSIT_DATABASE as "deposit DB"
participant CHECKER_TASK as "checker task"
participant CELERY as "celery"
participant SCHEDULER as "swh-scheduler"
activate DEPOSIT
activate DEPOSIT_DATABASE
activate CELERY
activate SCHEDULER
SCHEDULER ->> CELERY: new "check-deposit"\ntask available
CELERY ->> CHECKER_TASK: start task
activate CHECKER_TASK
CHECKER_TASK ->> DEPOSIT: GET /{collection}/{deposit_id}/check/
DEPOSIT ->> DEPOSIT_DATABASE: get deposit requests
DEPOSIT_DATABASE ->> DEPOSIT: deposit requests
loop for each request
DEPOSIT ->> DEPOSIT_DATABASE: get archive
DEPOSIT_DATABASE ->> DEPOSIT: archive content
DEPOSIT ->> DEPOSIT: check archive in the request
end
DEPOSIT ->> DEPOSIT_DATABASE: mark deposit as "verified"
DEPOSIT ->> SCHEDULER: schedule load
DEPOSIT ->> CHECKER_TASK: done
CHECKER_TASK ->> CELERY: done
deactivate CHECKER_TASK
CELERY ->> SCHEDULER: done
@enduml