diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 667fb8f6952498c710f7aefbb586ab35d3d87274..4b2ba24d114f9be8e37f46aa159fad822b018e94 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -1,19 +1,21 @@
+exclude: ^swh/deposit/tests/data/atom/.*$
+
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
- rev: v4.1.0
+ rev: v4.3.0
hooks:
- id: trailing-whitespace
- id: check-json
- id: check-yaml
- - repo: https://gitlab.com/pycqa/flake8
- rev: 4.0.1
+ - repo: https://github.com/pycqa/flake8
+ rev: 5.0.4
hooks:
- id: flake8
- additional_dependencies: [flake8-bugbear==22.3.23]
+ additional_dependencies: [flake8-bugbear==22.9.23]
- repo: https://github.com/codespell-project/codespell
- rev: v2.1.0
+ rev: v2.2.2
hooks:
- id: codespell
name: Check source code spelling
@@ -31,11 +33,11 @@ repos:
types: [python]
- repo: https://github.com/PyCQA/isort
- rev: 5.10.1
+ rev: 5.11.5
hooks:
- id: isort
- repo: https://github.com/python/black
- rev: 22.3.0
+ rev: 22.10.0
hooks:
- id: black
diff --git a/PKG-INFO b/PKG-INFO
index f3b69f5f4f89d14147d05ee47febeeb2b5fb5467..1c8b6bc3a5739957182900caa8002d213f833257 100644
--- a/PKG-INFO
+++ b/PKG-INFO
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: swh.deposit
-Version: 1.2.0
+Version: 1.2.1
Summary: Software Heritage Deposit Server
Home-page: https://forge.softwareheritage.org/source/swh-deposit/
Author: Software Heritage developers
@@ -37,7 +37,7 @@ Description
-----------
Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
the responsibility of the SWH project to gather and collect source code artifacts that
way.
diff --git a/README.rst b/README.rst
index 669519c560bd266f3060d3c93acd50405e758503..c9dc6efaca0b2a1abaa8b154dd9cbed1489c95a2 100644
--- a/README.rst
+++ b/README.rst
@@ -14,7 +14,7 @@ Description
-----------
Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
the responsibility of the SWH project to gather and collect source code artifacts that
way.
diff --git a/docs/README.rst b/docs/README.rst
index 669519c560bd266f3060d3c93acd50405e758503..c9dc6efaca0b2a1abaa8b154dd9cbed1489c95a2 100644
--- a/docs/README.rst
+++ b/docs/README.rst
@@ -14,7 +14,7 @@ Description
-----------
Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
the responsibility of the SWH project to gather and collect source code artifacts that
way.
diff --git a/docs/api/metadata.rst b/docs/api/metadata.rst
index 6319c7b7cdd48bc6a572660a83a1da28217d8b41..2e94b70025d6905f8d59e60b42db953eeb596eac 100644
--- a/docs/api/metadata.rst
+++ b/docs/api/metadata.rst
@@ -104,7 +104,7 @@ Using Atom with CodeMeta
<codemeta:identifier> article id </codemeta:identifier>
</codemeta:referencePublication>
<codemeta:isPartOf>
- <codemeta:type> Collaboration/Projet </codemeta:type>
+ <codemeta:type> Collaboration/Project </codemeta:type>
<codemeta:name> project name</codemeta:name>
<codemeta:identifier> id </codemeta:identifier>
</codemeta:isPartOf>
diff --git a/docs/api/register-account.rst b/docs/api/register-account.rst
index 5d7f2258eeebe501670ee1bed1f55660c4ca753f..f37cece2af9e10b38fd855eac80c318afc81bafd 100644
--- a/docs/api/register-account.rst
+++ b/docs/api/register-account.rst
@@ -15,9 +15,9 @@ As a deposit client
-------------------
For this, as a client, you need to register an account on the swh keycloak `production
-<https://archive.softwareheritage.org/oidc/login/?next_path=https://archive.softwareheritage.org/>`_
+<https://archive.softwareheritage.org/oidc/login/>`_
or `staging
-<https://webapp.staging.swh.network/oidc/login/?next_path=https://webapp.staging.swh.network/>`_
+<https://webapp.staging.swh.network/oidc/login/>`_
instance.
.. _swh-deposit-register-account-as-sysadm:
diff --git a/docs/index.rst b/docs/index.rst
index bc7e90039461b51d3ea6111e24894fe48d7c90e3..551263a7ac2bad7ebfdf1d9f3211183f0571eea3 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -18,4 +18,12 @@ Reference Documentation
:maxdepth: 2
cli
- /apidoc/swh.deposit
+
+.. only:: standalone_package_doc
+
+ Indices and tables
+ ------------------
+
+ * :ref:`genindex`
+ * :ref:`modindex`
+ * :ref:`search`
diff --git a/docs/specs/protocol-reference.rst b/docs/specs/protocol-reference.rst
index f02ea0fa5270177ebfc0cf2054e0a59bbd9abc3c..98986ec2144ec9f848711722c4006f637d90bcbf 100644
--- a/docs/specs/protocol-reference.rst
+++ b/docs/specs/protocol-reference.rst
@@ -144,11 +144,30 @@ of Software Heritage.
While CodeMeta is designed for use in JSON-LD, it is easy to reuse its vocabulary
and embed it in an XML document, in three steps:
-1. use the JSON-LD compact representation of the CodeMeta document
-2. replace ``@context`` declarations with XML namespaces
-3. unfold JSON lists to sibling XML subtrees
-
-For example, this CodeMeta document:
+1. use the `JSON-LD compact representation`_ of the CodeMeta document with
+ ``@context: "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"`` and no other context;
+ which implies that:
+
+ 1. Codemeta properties (whether in the ``https://codemeta.github.io/terms/``
+ or ``http://schema.org/`` namespaces) are unprefixed terms
+ 2. other properties in the ``http://schema.org/`` namespace use `compact IRIs`_
+ with the ``schema`` prefix
+ 3. other properties are absolute
+2. replace ``@context`` declarations with a XMLNS declaration with
+ ``https://doi.org/10.5063/SCHEMA/CODEMETA-2.0`` as namespace
+ (eg. ``xmlns="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"``
+ or ``xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"``)
+3. if using a non-default namespace, apply its prefix to any unprefixed term
+ (ie. any term defined in https://doi.org/10.5063/SCHEMA/CODEMETA-2.0 )
+4. add XMLNS declarations for any other prefix (eg. ``xmlns:schema="http://schema.org/"``
+ if any property in that namespace is used)
+5. unfold JSON lists to sibling XML subtrees
+
+.. _JSON-LD compact representation: https://www.w3.org/TR/json-ld11/#compacted-document-form
+.. _compact IRIs: https://www.w3.org/TR/json-ld11/#compact-iris
+
+Example Codemeta document
+"""""""""""""""""""""""""
.. code:: json
@@ -201,6 +220,56 @@ Or, equivalently:
</entry>
+Note that in both these examples, ``codemeta:name`` is used even though
+the property is actually ``http://schema.org/name``.
+
+Example generic JSON-LD document
+""""""""""""""""""""""""""""""""
+
+Another example using properties not part of Codemeta:
+
+.. code:: json
+
+ {
+ "@context": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
+ "name": "My Software",
+ "schema:sameAs": "http://example.org/my-software"
+ }
+
+which is equivalent to:
+
+.. code:: json
+
+ {
+ "@context": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
+ "name": "My Software",
+ "http://schema.org/sameAs": "http://example.org/my-software"
+ }
+
+becomes this XML document:
+
+.. code:: xml
+
+ <?xml version="1.0"?>
+ <atom:entry xmlns:atom="http://www.w3.org/2005/Atom"
+ xmlns="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"
+ xmlns:schema="http://schema.org/">
+ <name>My Software</name>
+ <schema:sameAs>http://example.org/my-software</schema:sameAs>
+ </atom:entry>
+
+Or, equivalently:
+
+.. code:: xml
+
+ <?xml version="1.0"?>
+ <entry xmlns="http://www.w3.org/2005/Atom"
+ xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"
+ xmlns:schema="http://schema.org/">
+ <codemeta:name>My Software</codemeta:name>
+ <schema:sameAs>http://example.org/my-software</schema:sameAs>
+ </entry>
+
.. _mandatory-attributes:
Mandatory attributes
diff --git a/docs/specs/spec-loading.rst b/docs/specs/spec-loading.rst
index 29b29ab0d0bbd0b371ad5dc434c9069348e1e528..c1420b4c091c48a949a769a25318e9b2ad655c33 100644
--- a/docs/specs/spec-loading.rst
+++ b/docs/specs/spec-loading.rst
@@ -63,15 +63,16 @@ For examples:
.. code-block:: bash
- $ http -pb https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-02560320/get/
+ $ http -pb https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-01883795/get/
would result in:
.. code-block:: json
{
- "origin_visits_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-02560320/visits/",
- "url": "https://hal.archives-ouvertes.fr/hal-02560320"
+ "url": "https://hal.archives-ouvertes.fr/hal-01883795",
+ "origin_visits_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-01883795/visits/",
+ "metadata_authorities_url": "https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/swhid/swh:1:ori:0094225e66277f3b2de66155b3cb30ca25f12565/authorities/"
}
@@ -85,7 +86,7 @@ For examples:
.. code-block:: bash
- $ http -pb https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-02560320/visits/
+ $ http -pb https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-01883795/visits/
would result in:
@@ -93,30 +94,29 @@ would result in:
[
{
- "date": "2020-05-14T11:59:55.942964+00:00",
+ "date": "2023-03-29T12:12:08.960810+00:00",
"metadata": {},
- "origin": "https://hal.archives-ouvertes.fr/hal-02560320",
- "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-02560320/visit/2/",
- "snapshot": "e5e82d064a9c3df7464223042e0c55d72ccff7f0",
- "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/e5e82d064a9c3df7464223042e0c55d72ccff7f0/",
+ "origin": "https://hal.archives-ouvertes.fr/hal-01883795",
+ "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-01883795/visit/2/",
+ "snapshot": "e59379a4f88c297066e964703893c23b08264ec8",
+ "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/e59379a4f88c297066e964703893c23b08264ec8/",
"status": "full",
"type": "deposit",
"visit": 2
},
{
- "date": "2020-05-14T11:59:41.094260+00:00",
+ "date": "2019-01-10T12:30:26.326411+00:00",
"metadata": {},
- "origin": "https://hal.archives-ouvertes.fr/hal-02560320",
- "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-02560320/visit/1/",
- "snapshot": "3e95ef6e04c381a34cc2f314576bc5644f2c797f",
- "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/3e95ef6e04c381a34cc2f314576bc5644f2c797f/",
+ "origin": "https://hal.archives-ouvertes.fr/hal-01883795",
+ "origin_visit_url": "https://archive.softwareheritage.org/api/1/origin/https://hal.archives-ouvertes.fr/hal-01883795/visit/1/",
+ "snapshot": "fd1b8fc1bdd3ebeac913eb6dd377a646a3149747",
+ "snapshot_url": "https://archive.softwareheritage.org/api/1/snapshot/fd1b8fc1bdd3ebeac913eb6dd377a646a3149747/",
"status": "full",
"type": "deposit",
"visit": 1
}
]
-
Snapshot artifact
~~~~~~~~~~~~~~~~~
@@ -127,7 +127,7 @@ For example:
.. code-block:: bash
- $ http -pb https://archive.softwareheritage.org/api/1/snapshot/3e95ef6e04c381a34cc2f314576bc5644f2c797f/
+ $ http -pb https://archive.softwareheritage.org/api/1/snapshot/e59379a4f88c297066e964703893c23b08264ec8/
would result in:
@@ -136,26 +136,45 @@ would result in:
{
"branches": {
"HEAD": {
- "target": "2122424b547a8eca9282ba3131ec61ff1d8df7d4",
- "target_type": "revision",
- "target_url": "https://archive.softwareheritage.org/api/1/revision/2122424b547a8eca9282ba3131ec61ff1d8df7d4/"
+ "target": "fc8e44c5bb3fabe81e5ebe46ac013a2510271616",
+ "target_type": "release",
+ "target_url": "https://archive.softwareheritage.org/api/1/release/fc8e44c5bb3fabe81e5ebe46ac013a2510271616/"
}
},
- "id": "3e95ef6e04c381a34cc2f314576bc5644f2c797f",
+ "id": "e59379a4f88c297066e964703893c23b08264ec8",
"next_branch": null
}
-Note that previous versions of the deposit-loader named the branch ``master``
-instead, and created release branches under certain conditions.
+Note that previous versions of the deposit-loader created a release instead of a revision.
+For example:
-Release artifact
-~~~~~~~~~~~~~~~~
-.. warning::
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/snapshot/fd1b8fc1bdd3ebeac913eb6dd377a646a3149747/
+
+resulted in:
+
+.. code-block:: json
+
+ {
+ "branches": {
+ "master": {
+ "target": "66ff08f00acc06131fe610be0f9878a6c78bfe44",
+ "target_type": "revision",
+ "target_url": "https://archive.softwareheritage.org/api/1/revision/66ff08f00acc06131fe610be0f9878a6c78bfe44/"
+ }
+ },
+ "id": "fd1b8fc1bdd3ebeac913eb6dd377a646a3149747",
+ "next_branch": null
+ }
+
+Even older versions named the branch ``master`` instead of ``HEAD``, and created
+release branches (pointing to revisions) under certain conditions.
- This part of the specification is not implemented yet, only revisions are
- currently being created.
+Release artifact
+~~~~~~~~~~~~~~~~
The content is deposited with a set of descriptive metadata in the CodeMeta
vocabulary. The following CodeMeta terms implies that the
@@ -169,9 +188,9 @@ If present, a release artifact will be created with the mapping below:
+-------------------+-----------------------------------+-----------------+----------------+
| SWH release field | Description | CodeMeta term | Fallback value |
+===================+===================================+=================+================+
-| target | revision containing all metadata | X |X |
+| target | directory containing all metadata | X |X |
+-------------------+-----------------------------------+-----------------+----------------+
-| target_type | revision | X |X |
+| target_type | directory | X |X |
+-------------------+-----------------------------------+-----------------+----------------+
| name | release or tag name (mandatory) | softwareVersion | X |
+-------------------+-----------------------------------+-----------------+----------------+
@@ -183,31 +202,36 @@ If present, a release artifact will be created with the mapping below:
+-------------------+-----------------------------------+-----------------+----------------+
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/release/fc8e44c5bb3fabe81e5ebe46ac013a2510271616/
+
.. code-block:: json
{
- "release": {
- "author": {
- "email": "hal@ccsd.cnrs.fr",
- "fullname": "HAL <phal@ccsd.cnrs.fr>",
- "name": "HAL"
- },
- "author_url": "/api/1/person/x/",
- "date": "2019-05-27T16:28:33+02:00",
- "id": "a9f3396f372ed4a51d75e15ca16c1c2df1fc5c97",
- "message": "AffectationRO Version 1.1 - added new feature\n",
- "name": "1.1",
- "synthetic": true,
- "target": "396b1ff29f7c75a0a3cc36f30e24ff7bae70bb52",
- "target_type": "revision",
- "target_url": "/api/1/revision/396b1ff29f7c75a0a3cc36f30e24ff7bae70bb52/"
- }
+ "author": {
+ "email": "robot@softwareheritage.org",
+ "fullname": "Software Heritage",
+ "name": "Software Heritage"
+ },
+ "date": "2021-01-01T00:00:00+00:00",
+ "id": "fc8e44c5bb3fabe81e5ebe46ac013a2510271616",
+ "message": "hal: Deposit 2753 in collection hal\n\n- Replace qmake with CMake.- Fix bugs.- Move repository.\n",
+ "name": "HEAD",
+ "synthetic": true,
+ "target": "7057a716afab8ca80728aa7c6c2cc4bd03b0f45b",
+ "target_type": "directory",
+ "target_url": "https://archive.softwareheritage.org/api/1/directory/7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/"
}
Revision artifact
~~~~~~~~~~~~~~~~~
+.. note::
+
+ Revision artifacts are no longer created by the deposit.
+
The metadata sent with the deposit is stored outside the revision,
and does not affect the hash computation.
It contains the same fields as any revision object; in particular:
@@ -229,6 +253,81 @@ It contains the same fields as any revision object; in particular:
| committer_date | see below |
+-------------------+-----------------------------------------+
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/revision/66ff08f00acc06131fe610be0f9878a6c78bfe44/
+
+.. code-block:: json
+
+ {
+ "author": {
+ "email": "robot@softwareheritage.org",
+ "fullname": "Software Heritage",
+ "name": "Software Heritage"
+ },
+ "committer": {
+ "email": "robot@softwareheritage.org",
+ "fullname": "Software Heritage",
+ "name": "Software Heritage"
+ },
+ "committer_date": "2019-01-10T12:27:59.639536+00:00",
+ "date": "2019-01-10T12:27:59.639536+00:00",
+ "directory": "70c73de7d406938315d6cf30bf87bb9eb480017e",
+ "directory_url": "https://archive.softwareheritage.org/api/1/directory/70c73de7d406938315d6cf30bf87bb9eb480017e/",
+ "extra_headers": [],
+ "history_url": "https://archive.softwareheritage.org/api/1/revision/66ff08f00acc06131fe610be0f9878a6c78bfe44/log/",
+ "id": "66ff08f00acc06131fe610be0f9878a6c78bfe44",
+ "merge": false,
+ "message": "hal: Deposit 225 in collection hal",
+ "metadata": {
+ "@xmlns": "http://www.w3.org/2005/Atom",
+ "@xmlns:codemeta": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
+ "author": {
+ "email": "hal@ccsd.cnrs.fr",
+ "name": "HAL"
+ },
+ "client": "hal",
+ "codemeta:applicationCategory": "sdu.ocean",
+ "codemeta:author": {
+ "codemeta:affiliation": "LaMP",
+ "codemeta:name": "D. Picard"
+ },
+ "codemeta:codeRepository": "https://forge.clermont-universite.fr/git/libszdist",
+ "codemeta:dateCreated": "2018-09-28T16:58:05+02:00",
+ "codemeta:description": "libszdist is a C++ library and command line tools that implement the algorithm used to process the data of instruments called SMPS/DMPS. These instruments measure the size distribution of aerosol particles. The algorithm is known as ''inversion''.",
+ "codemeta:developmentStatus": "Actif",
+ "codemeta:keywords": "SMPS,DMPS,Aerosol Size Distribution",
+ "codemeta:license": {
+ "codemeta:name": "GNU GPLv3"
+ },
+ "codemeta:name": "libszdist",
+ "codemeta:operatingSystem": [
+ "Linux",
+ "Windows",
+ "Mac OS X",
+ "ARM"
+ ],
+ "codemeta:programmingLanguage": "C++",
+ "codemeta:runtimePlatform": [
+ "qmake",
+ "gcc"
+ ],
+ "codemeta:softwareVersion": "v.0.10.4",
+ "codemeta:url": "https://hal.archives-ouvertes.fr/hal-01883795",
+ "codemeta:version": "1",
+ "committer": "David Picard",
+ "external_identifier": "hal-01883795",
+ "id": "hal-01883795"
+ },
+ "parents": [],
+ "synthetic": true,
+ "type": "tar",
+ "url": "https://archive.softwareheritage.org/api/1/revision/66ff08f00acc06131fe610be0f9878a6c78bfe44/"
+ }
+
+Note that the metadata field is deprecated. The "extrinsic metadata" endpoints described
+below should be used instead.
+
The date mapping
^^^^^^^^^^^^^^^^
@@ -276,117 +375,49 @@ A release contains one date:
| date |release date = publication date | datePublished | reception_date |
+-------------------+----------------------------------+----------------+-----------------+
-
-.. code-block:: json
-
- {
- "revision": {
- "author": {
- "email": "robot@softwareheritage.org",
- "fullname": "Software Heritage",
- "id": 18233048,
- "name": "Software Heritage"
- },
- "author_url": "/api/1/person/18233048/",
- "committer": {
- "email": "robot@softwareheritage.org",
- "fullname": "Software Heritage",
- "id": 18233048,
- "name": "Software Heritage"
- },
- "committer_date": "2019-05-27T16:28:33+02:00",
- "committer_url": "/api/1/person/18233048/",
- "date": "2012-01-01T00:00:00+00:00",
- "directory": "fb13b51abbcfd13de85d9ba8d070a23679576cd7",
- "directory_url": "/api/1/directory/fb13b51abbcfd13de85d9ba8d070a23679576cd7/",
- "history_url": "/api/1/revision/396b1ff29f7c75a0a3cc36f30e24ff7bae70bb52/log/",
- "id": "396b1ff29f7c75a0a3cc36f30e24ff7bae70bb52",
- "merge": false,
- "message": "hal: Deposit 282 in collection hal",
- "metadata": {
- "@xmlns": "http://www.w3.org/2005/Atom",
- "@xmlns:codemeta": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
- "author": {
- "email": "hal@ccsd.cnrs.fr",
- "name": "HAL"
- },
- "codemeta:applicationCategory": "info",
- "codemeta:author": {
- "codemeta:name": "Morane Gruenpeter"
- },
- "codemeta:codeRepository": "www.code-repository.com",
- "codemeta:contributor": {
- "codemeta:name": "Morane Gruenpeter",
- },
- "codemeta:dateCreated": "2012",
- "codemeta:datePublished": "2019-05-27T16:28:33+02:00",
- "codemeta:description": "description\\_en test v2",
- "codemeta:developmentStatus": "Inactif",
- "codemeta:keywords": "mot_cle_en,mot_cle_2_en,mot_cle_fr",
- "codemeta:license": [
- {
- "codemeta:name": "MIT License"
- },
- {
- "codemeta:name": "CeCILL Free Software License Agreement v1.1"
- }
- ],
- "codemeta:name": "Test\\_20190527\\_01",
- "codemeta:operatingSystem": "OS",
- "codemeta:programmingLanguage": "Java",
- "codemeta:referencePublication": null,
- "codemeta:relatedLink": null,
- "codemeta:releaseNotes": "releaseNote",
- "codemeta:runtimePlatform": "outil",
- "codemeta:softwareVersion": "1.0.1",
- "codemeta:url": "https://hal.archives-ouvertes.fr/hal-02140606",
- "codemeta:version": "2",
- "id": "hal-02140606",
- "original_artifact": [
- {
- "archive_type": "zip",
- "blake2s256": "96be3ddedfcee9669ad9c42b0bb3a706daf23824d04311c63505a4d8db02df00",
- "length": 193072,
- "name": "archive.zip",
- "sha1": "5b6ecc9d5bb113ff69fc275dcc9b0d993a8194f1",
- "sha1_git": "bd10e4d3ede17162692d7e211e08e87e67994488",
- "sha256": "3e2ce93384251ce6d6da7b8f2a061a8ebdaf8a28b8d8513223ca79ded8a10948"
- }
- ]
- },
- "parents": [
- {
- "id": "a9fdc3937d2b704b915852a64de2ab1b4b481003",
- "url": "/api/1/revision/a9fdc3937d2b704b915852a64de2ab1b4b481003/"
- }
- ],
- "synthetic": true,
- "type": "tar",
- "url": "/api/1/revision/396b1ff29f7c75a0a3cc36f30e24ff7bae70bb52/"
- }
- }
-
Directory artifact
~~~~~~~~~~~~~~~~~~
The directory artifact is the archive(s)' raw content deposited.
-.. code-block:: json
+.. code-block:: bash
- {
- "directory": [
- {
- "dir_id": "fb13b51abbcfd13de85d9ba8d070a23679576cd7",
- "length": null,
- "name": "AffectationRO",
- "perms": 16384,
- "target": "fbc418f9ac2c39e8566b04da5dc24b14e65b23b1",
- "target_url": "/api/1/directory/fbc418f9ac2c39e8566b04da5dc24b14e65b23b1/",
- "type": "dir"
- }
- ]
- }
+ http -pb https://archive.softwareheritage.org/api/1/directory/7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/
+
+.. code-block:: json
+ [
+ {
+ "checksums": {
+ "sha1": "cadfc0e77c0119a025a5ed45d07f71df4071f645",
+ "sha1_git": "b89214f14acaca84efb65ff6542cb5d790b6ac5c",
+ "sha256": "47c165ad20425a13f65ebd9db61447363bb9cf3ce0b0fa4418d9cfc951f157e3"
+ },
+ "dir_id": "7057a716afab8ca80728aa7c6c2cc4bd03b0f45b",
+ "length": 150,
+ "name": ".gitignore",
+ "perms": 33188,
+ "status": "visible",
+ "target": "b89214f14acaca84efb65ff6542cb5d790b6ac5c",
+ "target_url": "https://archive.softwareheritage.org/api/1/content/sha1_git:b89214f14acaca84efb65ff6542cb5d790b6ac5c/",
+ "type": "file"
+ },
+ {
+ "checksums": {
+ "sha1": "816fde05704e5b7c8a744044949b9f7944702993",
+ "sha1_git": "de6f1f373a44be2b16232b2ff9744f31fe7e3715",
+ "sha256": "09585c721573beadc56a98754745f9381c15626f6471b7da18475366e4e8f2cb"
+ },
+ "dir_id": "7057a716afab8ca80728aa7c6c2cc4bd03b0f45b",
+ "length": 51,
+ "name": "AUTHORS",
+ "perms": 33188,
+ "status": "visible",
+ "target": "de6f1f373a44be2b16232b2ff9744f31fe7e3715",
+ "target_url": "https://archive.softwareheritage.org/api/1/content/sha1_git:de6f1f373a44be2b16232b2ff9744f31fe7e3715/",
+ "type": "file"
+ }
+ ]
Questions raised concerning loading
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -472,3 +503,120 @@ Metadata loading
- ``authority`` is computed from the deposit client information, and ``fetcher``
is the deposit loader.
+
+They can be queried using the directory SWHID.
+
+First, we need to get the list of authorities which published metadata on this directory:
+
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/swhid/swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/authorities/
+
+.. code-block:: json
+
+ [
+ {
+ "metadata_list_url": "https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/swhid/swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/?authority=deposit_client%20https://hal.archives-ouvertes.fr/",
+ "type": "deposit_client",
+ "url": "https://hal.archives-ouvertes.fr/"
+ },
+ {
+ "metadata_list_url": "https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/swhid/swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/?authority=registry%20https://softwareheritage.org/",
+ "type": "registry",
+ "url": "https://softwareheritage.org/"
+ }
+ ]
+
+The former is HAL, the latter is Software Heritage itself (to provide attestation of tarball checksums).
+We can get the list of metadata provided by HAL:
+
+
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/swhid/swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b/\?authority\=deposit_client%20https://hal.archives-ouvertes.fr/
+
+.. code-block:: json
+
+ [
+ {
+ "authority": {
+ "type": "deposit_client",
+ "url": "https://hal.archives-ouvertes.fr/"
+ },
+ "discovery_date": "2023-03-29T12:11:53+00:00",
+ "fetcher": {
+ "name": "swh-deposit",
+ "version": "1.1.0"
+ },
+ "format": "sword-v2-atom-codemeta-v2",
+ "metadata_url": "https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/get/c65992f8f3efe416ccf2666f8ff09753ea94377d/?filename=swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b_metadata",
+ "origin": "https://hal.archives-ouvertes.fr/hal-01883795",
+ "release": "swh:1:rel:fc8e44c5bb3fabe81e5ebe46ac013a2510271616",
+ "target": "swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b"
+ }
+ ]
+
+and finally, we got the URL to the metadata blob itself:
+
+.. code-block:: bash
+
+ http -pb https://archive.softwareheritage.org/api/1/raw-extrinsic-metadata/get/c65992f8f3efe416ccf2666f8ff09753ea94377d/\?filename\=swh:1:dir:7057a716afab8ca80728aa7c6c2cc4bd03b0f45b_metadata
+
+.. code-block:: xml
+
+ <?xml version="1.0" encoding="utf-8"?>
+ <entry xmlns="http://www.w3.org/2005/Atom" xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0" xmlns:schema="http://schema.org/" xmlns:swh="https://www.softwareheritage.org/schema/2018/deposit">
+ <id>hal-01883795</id>
+ <swh:deposit>
+ <swh:create_origin>
+ <swh:origin url="https://hal.archives-ouvertes.fr/hal-01883795"/>
+ </swh:create_origin>
+ <swh:metadata-provenance>
+ <schema:url>https://hal.archives-ouvertes.fr/hal-01883795</schema:url>
+ </swh:metadata-provenance>
+ </swh:deposit>
+ <author>
+ <name>HAL</name>
+ <email>hal@ccsd.cnrs.fr</email>
+ </author>
+ <codemeta:name>libszdist</codemeta:name>
+ <codemeta:description>libszdist is a C++ library and command line tools that implement the algorithm used to process the data of instruments called SMPS/DMPS. These instruments measure the size distribution of aerosol particles. The algorithm is known as ''inversion''.</codemeta:description>
+ <codemeta:dateCreated>2021-01-01</codemeta:dateCreated>
+ <codemeta:datePublished>2023-03-16</codemeta:datePublished>
+ <codemeta:license>
+ <codemeta:name>GNU GPLv3</codemeta:name>
+ </codemeta:license>
+ <schema:identifier>
+ <codemeta:type>schema:PropertyValue</codemeta:type>
+ <schema:propertyID>HAL-ID</schema:propertyID>
+ <schema:value>hal-01883795</schema:value>
+ </schema:identifier>
+ <codemeta:applicationCategory>sdu.ocean</codemeta:applicationCategory>
+ <codemeta:keywords>SMPS,DMPS,Aerosol Size Distribution,MPSS</codemeta:keywords>
+ <codemeta:institution>CNRS</codemeta:institution>
+ <codemeta:codeRepository>https://forge.clermont-universite.fr/git/libszdist</codemeta:codeRepository>
+ <codemeta:relatedLink>https://gitlab.in2p3.fr/david.picard/libszdist</codemeta:relatedLink>
+ <codemeta:programmingLanguage>C++</codemeta:programmingLanguage>
+ <codemeta:runtimePlatform>gcc</codemeta:runtimePlatform>
+ <codemeta:runtimePlatform>CMake</codemeta:runtimePlatform>
+ <codemeta:operatingSystem>Linux</codemeta:operatingSystem>
+ <codemeta:operatingSystem>Windows</codemeta:operatingSystem>
+ <codemeta:operatingSystem>Mac OS X</codemeta:operatingSystem>
+ <codemeta:operatingSystem>ARM</codemeta:operatingSystem>
+ <codemeta:operatingSystem>PC</codemeta:operatingSystem>
+ <codemeta:version>2</codemeta:version>
+ <codemeta:softwareVersion>v.0.11.1</codemeta:softwareVersion>
+ <codemeta:dateModified>2023-03-24</codemeta:dateModified>
+ <codemeta:releaseNotes>- Replace qmake with CMake.- Fix bugs.- Move repository.</codemeta:releaseNotes>
+ <codemeta:developmentStatus>Actif</codemeta:developmentStatus>
+ <codemeta:author>
+ <codemeta:name>D. Picard</codemeta:name>
+ <codemeta:affiliation>LPC</codemeta:affiliation>
+ <codemeta:affiliation>LaMP</codemeta:affiliation>
+ </codemeta:author>
+ <codemeta:contributor>
+ <codemeta:name>David PICARD</codemeta:name>
+ </codemeta:contributor>
+ </entry>
+
+which is the exact document provided by HAL when uploading the deposit.
diff --git a/requirements-server.txt b/requirements-server.txt
index 1da85404ac731d6e7ab3c40d5dc274ae7fec7f44..f433b444ac81f0f8f5d5e85a06b47cc5aab81773 100644
--- a/requirements-server.txt
+++ b/requirements-server.txt
@@ -1,5 +1,5 @@
-django >= 2, < 4
+django
djangorestframework
-psycopg2 < 2.9
+psycopg2
setuptools
xmlschema
diff --git a/requirements-swh.txt b/requirements-swh.txt
index 07448bc2819fa332d7472d367eb06660cc68a3a9..1d5432fb2d8b4fbb7484e3912f89d87ba4532253 100644
--- a/requirements-swh.txt
+++ b/requirements-swh.txt
@@ -1,2 +1,3 @@
swh.core[http] >= 0.4
swh.model >= 4.4.0
+swh.scheduler
diff --git a/swh.deposit.egg-info/PKG-INFO b/swh.deposit.egg-info/PKG-INFO
index f3b69f5f4f89d14147d05ee47febeeb2b5fb5467..1c8b6bc3a5739957182900caa8002d213f833257 100644
--- a/swh.deposit.egg-info/PKG-INFO
+++ b/swh.deposit.egg-info/PKG-INFO
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: swh.deposit
-Version: 1.2.0
+Version: 1.2.1
Summary: Software Heritage Deposit Server
Home-page: https://forge.softwareheritage.org/source/swh-deposit/
Author: Software Heritage developers
@@ -37,7 +37,7 @@ Description
-----------
Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
the responsibility of the SWH project to gather and collect source code artifacts that
way.
diff --git a/swh.deposit.egg-info/SOURCES.txt b/swh.deposit.egg-info/SOURCES.txt
index 23dc3e4b84d7aa6dd9c0ae397096bd7e191f7eb9..de372a3f91c03c1b1b6a27f5122ca9f4cc0312b4 100644
--- a/swh.deposit.egg-info/SOURCES.txt
+++ b/swh.deposit.egg-info/SOURCES.txt
@@ -185,6 +185,7 @@ swh/deposit/tests/conftest.py
swh/deposit/tests/test_backend.py
swh/deposit/tests/test_client_module.py
swh/deposit/tests/test_common.py
+swh/deposit/tests/test_config.py
swh/deposit/tests/test_gunicorn_config.py
swh/deposit/tests/test_init.py
swh/deposit/tests/test_migrations.py
diff --git a/swh.deposit.egg-info/requires.txt b/swh.deposit.egg-info/requires.txt
index cd826cb66a0d81146908e7a7e03a8af66df655d8..04dcd9c52c624a33d832b6059fae5cf388c8a3f9 100644
--- a/swh.deposit.egg-info/requires.txt
+++ b/swh.deposit.egg-info/requires.txt
@@ -4,11 +4,12 @@ requests
sentry-sdk
swh.core[http]>=0.4
swh.model>=4.4.0
+swh.scheduler
[server]
-django<4,>=2
+django
djangorestframework
-psycopg2<2.9
+psycopg2
setuptools
xmlschema
swh.core[http]>=0.4
@@ -31,9 +32,9 @@ djangorestframework-stubs>=1.4
django-test-migrations
types-requests
types-pyyaml
-django<4,>=2
+django
djangorestframework
-psycopg2<2.9
+psycopg2
setuptools
xmlschema
swh.core[http]>=0.4
diff --git a/swh/deposit/api/common.py b/swh/deposit/api/common.py
index b2a24959c0f72d0a77e922c7e81592070dcf1efb..7e970d0517d396c41c368d847203905de49a3038 100644
--- a/swh/deposit/api/common.py
+++ b/swh/deposit/api/common.py
@@ -584,15 +584,24 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
"in the multipart deposit",
)
- filehandler = data["application/zip"]
- if not filehandler:
- filehandler = data["application/x-tar"]
+ filehandler = data["application/zip"] or data["application/x-tar"]
+ if filehandler is None:
+ raise DepositError(
+ BAD_REQUEST,
+ "You must provide an archive, either as application/zip or "
+ "application/x-tar",
+ )
assert isinstance(filehandler, UploadedFile), filehandler
self._check_file_length(filehandler)
self._check_file_md5sum(filehandler, headers.content_md5sum)
+ if data["application/atom+xml"] is None:
+ raise DepositError(
+ BAD_REQUEST, "You must provide an application/atom+xml entry."
+ )
+
try:
raw_metadata, metadata_tree = self._read_metadata(
data["application/atom+xml"]
@@ -620,7 +629,6 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
deposit, deposit_request_data, replace_metadata, replace_archives
)
- assert filehandler is not None
return Receipt(
deposit_id=deposit.id,
deposit_date=deposit.reception_date,
@@ -1010,9 +1018,7 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
if self._client is None:
try:
- self._client = DepositClient.objects.get( # type: ignore
- username=username
- )
+ self._client = DepositClient.objects.get(username=username)
except DepositClient.DoesNotExist:
raise DepositError(NOT_FOUND, f"Unknown client name {username}")
diff --git a/swh/deposit/api/private/urls.py b/swh/deposit/api/private/urls.py
index f9f8357a2a081e541314836204937a77a0453415..a5d29b0ed166216c7d2f71b5cfdfb552aa2e3624 100644
--- a/swh/deposit/api/private/urls.py
+++ b/swh/deposit/api/private/urls.py
@@ -1,9 +1,9 @@
-# Copyright (C) 2017-2022 The Software Heritage developers
+# Copyright (C) 2017-2023 The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information
-from django.conf.urls import url
+from django.urls import re_path as url
from ...config import (
PRIVATE_CHECK_DEPOSIT,
diff --git a/swh/deposit/api/urls.py b/swh/deposit/api/urls.py
index a8b7146baf3bd3943e953b71845a889a541c2ef6..c204c04d2f4869127665069d1c6d18b692ee3bd1 100644
--- a/swh/deposit/api/urls.py
+++ b/swh/deposit/api/urls.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2017-2021 The Software Heritage developers
+# Copyright (C) 2017-2023 The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information
@@ -7,8 +7,8 @@
"""
-from django.conf.urls import url
from django.shortcuts import render
+from django.urls import re_path as url
from ..config import COL_IRI, CONT_FILE_IRI, EDIT_IRI, EM_IRI, SD_IRI, SE_IRI, STATE_IRI
from .collection import CollectionAPI
diff --git a/swh/deposit/cli/admin.py b/swh/deposit/cli/admin.py
index 594ca17723a6859b9ce5ec6c2d4a972685f1f73f..7e0735b500191017d8a5dc62bb01d930f5963b94 100644
--- a/swh/deposit/cli/admin.py
+++ b/swh/deposit/cli/admin.py
@@ -114,7 +114,7 @@ def user_create(
# user create/update
try:
- user = DepositClient.objects.get(username=username) # type: ignore
+ user = DepositClient.objects.get(username=username)
click.echo(f"Update user '{username}'.")
action_done = "updated"
except DepositClient.DoesNotExist:
@@ -166,7 +166,7 @@ def user_exists(ctx, username: str):
from swh.deposit.models import DepositClient
try:
- DepositClient.objects.get(username=username) # type: ignore
+ DepositClient.objects.get(username=username)
click.echo(f"User {username} exists.")
ctx.exit(0)
except DepositClient.DoesNotExist:
diff --git a/swh/deposit/config.py b/swh/deposit/config.py
index 585b3be4c717aae1604412769a1fbe0fc5403aae..aaf2ebd99c7aa95d212f526554c8990f7c0b9ab7 100644
--- a/swh/deposit/config.py
+++ b/swh/deposit/config.py
@@ -4,7 +4,7 @@
# See top-level LICENSE file for more information
import os
-from typing import Any, Dict
+from typing import Any, Dict, Optional
from swh.core import config
from swh.deposit import __version__
@@ -58,36 +58,37 @@ DEFAULT_CONFIG = {
}
-def setup_django_for(platform=None, config_file=None):
- """Setup function for command line tools (swh.deposit.create_user) to
- initialize the needed db access.
+def setup_django_for(platform: Optional[str] = None, config_file: Optional[str] = None):
+ """Setup function for command line tools (e.g. swh.deposit.create_user) to
+ initialize the needed db access.
Note:
Do not import any django related module prior to this function
- call. Otherwise, this will raise an
- django.core.exceptions.ImproperlyConfigured error message.
+ call. Otherwise, this will raise a django.core.exceptions.ImproperlyConfigured
+ error message.
Args:
- platform (str): the platform the scheduling is running
- config_file (str): Extra configuration file (typically for the
- production platform)
+ platform: the platform to use when running program (e.g. cli, ...)
+ config_file: Extra configuration file (typically for the production platform)
Raises:
- ValueError in case of wrong platform inputs.
+ ValueError in case of wrong platform inputs
"""
if platform is not None:
if platform not in AUTHORIZED_PLATFORMS:
- raise ValueError("Platform should be one of %s" % AUTHORIZED_PLATFORMS)
+ raise ValueError(f"Platform should be one of {AUTHORIZED_PLATFORMS}")
if "DJANGO_SETTINGS_MODULE" not in os.environ:
- os.environ["DJANGO_SETTINGS_MODULE"] = "swh.deposit.settings.%s" % platform
+ os.environ["DJANGO_SETTINGS_MODULE"] = f"swh.deposit.settings.{platform}"
if config_file:
+ # Hack to set the environment variable which in some cases is required (e.g.
+ # production)
os.environ.setdefault("SWH_CONFIG_FILENAME", config_file)
- import django
+ from django import setup
- django.setup()
+ setup()
class APIConfig:
diff --git a/swh/deposit/tests/api/test_collection_post_multipart.py b/swh/deposit/tests/api/test_collection_post_multipart.py
index 20f1bdb4c60215427d44639860b3711b2fa75f6b..812263e9c3af6dcb28811cb973465b1eed494179 100644
--- a/swh/deposit/tests/api/test_collection_post_multipart.py
+++ b/swh/deposit/tests/api/test_collection_post_multipart.py
@@ -271,7 +271,7 @@ def test_post_deposit_multipart_put_to_replace_metadata(
# FAILURE scenarios
-def test_post_deposit_multipart_only_archive_and_atom_entry(
+def test_post_deposit_multipart_only_one_archive_and_atom_entry(
authenticated_client, deposit_collection
):
"""Multipart deposit only accepts one archive and one atom+xml"""
@@ -391,3 +391,81 @@ def test_post_deposit_multipart_if_upload_size_limit_exceeded(
with pytest.raises(Deposit.DoesNotExist):
Deposit.objects.get(external_id=external_id)
+
+
+def test_post_deposit_atom_400_multipart_no_atom(
+ authenticated_client, deposit_collection, atom_dataset, deposit_user, sample_archive
+):
+ """Posting without an atom body should return a 400 response"""
+ origin_url = "http://example.org/foo"
+
+ archive = InMemoryUploadedFile(
+ BytesIO(sample_archive["data"]),
+ field_name=sample_archive["name"],
+ name=sample_archive["name"],
+ content_type="application/x-tar",
+ size=sample_archive["length"],
+ charset=None,
+ )
+
+ atom_data = atom_dataset["entry-data0"] % origin_url
+
+ atom_entry = InMemoryUploadedFile(
+ BytesIO(atom_data.encode("utf-8")),
+ field_name="atom0",
+ name="atom0",
+ content_type="application/x-foobar", # should be application/atom+xml
+ size=len(atom_data),
+ charset="utf-8",
+ )
+
+ response = authenticated_client.post(
+ reverse(COL_IRI, args=[deposit_collection.name]),
+ format="multipart",
+ data={
+ "archive": archive,
+ "atom_entry": atom_entry,
+ },
+ )
+
+ assert response.status_code == status.HTTP_400_BAD_REQUEST
+ assert "provide an application/atom+xml entry" in response.content.decode()
+
+
+def test_post_deposit_atom_400_multipart_no_archive(
+ authenticated_client, deposit_collection, atom_dataset, deposit_user, sample_archive
+):
+ """Posting without an atom body should return a 400 response"""
+ origin_url = "http://example.org/foo"
+
+ archive = InMemoryUploadedFile(
+ BytesIO(sample_archive["data"]),
+ field_name=sample_archive["name"],
+ name=sample_archive["name"],
+ content_type="application/x-foobar", # should be application/x-tar
+ size=sample_archive["length"],
+ charset=None,
+ )
+
+ atom_data = atom_dataset["entry-data0"] % origin_url
+
+ atom_entry = InMemoryUploadedFile(
+ BytesIO(atom_data.encode("utf-8")),
+ field_name="atom0",
+ name="atom0",
+ content_type="application/atom+xml",
+ size=len(atom_data),
+ charset="utf-8",
+ )
+
+ response = authenticated_client.post(
+ reverse(COL_IRI, args=[deposit_collection.name]),
+ format="multipart",
+ data={
+ "archive": archive,
+ "atom_entry": atom_entry,
+ },
+ )
+
+ assert response.status_code == status.HTTP_400_BAD_REQUEST
+ assert "provide an archive" in response.content.decode()
diff --git a/swh/deposit/tests/conftest.py b/swh/deposit/tests/conftest.py
index e858a1214cc1967e3e34f78e8e71bccba1e22a83..5f13a6c64a8098dc864682ebb3ca36e8b45ce71d 100644
--- a/swh/deposit/tests/conftest.py
+++ b/swh/deposit/tests/conftest.py
@@ -264,7 +264,7 @@ def _create_deposit_user(
user_data_d = deepcopy(user_data)
user_data_d.pop("collection", None)
passwd = user_data_d.pop("password", None)
- user, _ = DepositClient.objects.get_or_create( # type: ignore
+ user, _ = DepositClient.objects.get_or_create(
username=user_data_d["username"],
defaults={**user_data_d, "collections": [collection.id]},
)
diff --git a/swh/deposit/tests/test_config.py b/swh/deposit/tests/test_config.py
new file mode 100644
index 0000000000000000000000000000000000000000..82fd404aa5ef6c2ae3b36600dd190f38fba761a3
--- /dev/null
+++ b/swh/deposit/tests/test_config.py
@@ -0,0 +1,38 @@
+# Copyright (C) 2022 The Software Heritage developers
+# See the AUTHORS file at the top-level directory of this distribution
+# License: GNU General Public License version 3, or any later version
+# See top-level LICENSE file for more information
+
+import os
+
+import pytest
+
+from swh.deposit.config import setup_django_for
+
+
+def test_setup_django_for_raise_unknown_platform():
+ """Unknown platform should make the function setup raise"""
+ with pytest.raises(ValueError, match="Platform should be"):
+ setup_django_for(platform="unknown")
+
+
+def test_setup_django__for_set_django_settings_module(monkeypatch, deposit_config_path):
+ monkeypatch.delenv("DJANGO_SETTINGS_MODULE")
+ platform = "testing"
+ setup_django_for(platform)
+
+ assert os.environ["DJANGO_SETTINGS_MODULE"] == f"swh.deposit.settings.{platform}"
+
+
+def test_setup_django_for_ok_set_django_settings_module(
+ monkeypatch, deposit_config_path
+):
+ monkeypatch.delenv("SWH_CONFIG_FILENAME")
+ setup_django_for("testing", deposit_config_path)
+
+ assert os.environ["SWH_CONFIG_FILENAME"] == deposit_config_path
+
+
+def test_setup_django_for_ok(deposit_config_path):
+ """Everything is fine, moving along (fixture sets environment appropriately)"""
+ setup_django_for()
diff --git a/swh/deposit/urls.py b/swh/deposit/urls.py
index 64aee5607ade9280c2867b96648c9e11056607ae..36eb3a1dff696282133624642c6656f45b5c759c 100644
--- a/swh/deposit/urls.py
+++ b/swh/deposit/urls.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2017-2021 The Software Heritage developers
+# Copyright (C) 2017-2023 The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information
@@ -10,8 +10,9 @@ from __future__ import annotations
from typing import Sequence, Union
-from django.conf.urls import include, url
+from django.conf.urls import include
from django.shortcuts import render
+from django.urls import re_path as url
from django.views.generic.base import RedirectView
from rest_framework.urlpatterns import format_suffix_patterns
diff --git a/tox.ini b/tox.ini
index dcdc31082f7ba851ebb6e763d37454664af8e860..52e042f0b4e6ec3893d2b31aa0176ad70fc9fe53 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,4 +1,6 @@
[tox]
+requires =
+ tox>4
envlist=flake8,mypy,py3-django2,py3-django3
[testenv]
@@ -22,15 +24,16 @@ commands =
[testenv:black]
skip_install = true
deps =
- black==22.3.0
+ black==22.10.0
commands =
{envpython} -m black --check swh
[testenv:flake8]
skip_install = true
deps =
- flake8==4.0.1
- flake8-bugbear==22.3.23
+ flake8==5.0.4
+ flake8-bugbear==22.9.23
+ pycodestyle==2.9.1
commands =
{envpython} -m flake8 \
--exclude=.tox,.git,__pycache__,.tox,.eggs,*.egg,swh/deposit/migrations
@@ -40,7 +43,7 @@ setenv = DJANGO_SETTINGS_MODULE=swh.deposit.settings.testing
extras =
testing
deps =
- mypy==0.942
+ mypy==1.0.1
commands =
mypy swh
@@ -48,14 +51,13 @@ commands =
# git HEAD of swh-docs, is executed on CI for each diff to prevent
# breaking doc build
[testenv:sphinx]
-whitelist_externals = make
+allowlist_externals = make
usedevelop = true
extras =
testing
deps =
# fetch and install swh-docs in develop mode
- -e git+https://forge.softwareheritage.org/source/swh-docs#egg=swh.docs
-
+ -e git+https://gitlab.softwareheritage.org/swh/devel/swh-docs.git\#egg=swh.docs
setenv =
SWH_PACKAGE_DOC_TOX_BUILD = 1
# turn warnings into errors
@@ -63,18 +65,16 @@ setenv =
commands =
make -I ../.tox/sphinx/src/swh-docs/swh/ -C docs
-
# build documentation only inside swh-environment using local state
# of swh-docs package
[testenv:sphinx-dev]
-whitelist_externals = make
+allowlist_externals = make
usedevelop = true
extras =
testing
deps =
# install swh-docs in develop mode
-e ../swh-docs
-
setenv =
SWH_PACKAGE_DOC_TOX_BUILD = 1
# turn warnings into errors