Update upstream source from tag 'debian/upstream/1.2.1'

Update to upstream version '1.2.1'
with Debian dir 67c544533f0542f34cd7863c7a47cee8cfb120f1

.pre-commit-config.yaml

+exclude: ^swh/deposit/tests/data/atom/.*$
+
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.1.0
+    rev: v4.3.0
     hooks:
       - id: trailing-whitespace
       - id: check-json
       - id: check-yaml
 
-  - repo: https://gitlab.com/pycqa/flake8
-    rev: 4.0.1
+  - repo: https://github.com/pycqa/flake8
+    rev: 5.0.4
     hooks:
       - id: flake8
-        additional_dependencies: [flake8-bugbear==22.3.23]
+        additional_dependencies: [flake8-bugbear==22.9.23]
 
   - repo: https://github.com/codespell-project/codespell
-    rev: v2.1.0
+    rev: v2.2.2
     hooks:
       - id: codespell
         name: Check source code spelling
@@ -31,11 +33,11 @@ repos:
         types: [python]
 
   - repo: https://github.com/PyCQA/isort
-    rev: 5.10.1
+    rev: 5.11.5
     hooks:
       - id: isort
 
   - repo: https://github.com/python/black
-    rev: 22.3.0
+    rev: 22.10.0
     hooks:
       - id: black

PKG-INFO

 Metadata-Version: 2.1
 Name: swh.deposit
-Version: 1.2.0
+Version: 1.2.1
 Summary: Software Heritage Deposit Server
 Home-page: https://forge.softwareheritage.org/source/swh-deposit/
 Author: Software Heritage developers
@@ -37,7 +37,7 @@ Description
 -----------
 
 Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
 origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
 the responsibility of the SWH project to gather and collect source code artifacts that
 way.

README.rst

@@ -14,7 +14,7 @@ Description
 -----------
 
 Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
 origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
 the responsibility of the SWH project to gather and collect source code artifacts that
 way.

docs/README.rst

@@ -14,7 +14,7 @@ Description
 -----------
 
 Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
 origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
 the responsibility of the SWH project to gather and collect source code artifacts that
 way.

docs/api/metadata.rst

@@ -104,7 +104,7 @@ Using Atom with CodeMeta
               <codemeta:identifier> article id </codemeta:identifier>
             </codemeta:referencePublication>
             <codemeta:isPartOf>
-                <codemeta:type> Collaboration/Projet </codemeta:type>
+                <codemeta:type> Collaboration/Project </codemeta:type>
                 <codemeta:name> project name</codemeta:name>
                 <codemeta:identifier> id </codemeta:identifier>
             </codemeta:isPartOf>

docs/api/register-account.rst

@@ -15,9 +15,9 @@ As a deposit client
 -------------------
 
 For this, as a client, you need to register an account on the swh keycloak `production
-<https://archive.softwareheritage.org/oidc/login/?next_path=https://archive.softwareheritage.org/>`_
+<https://archive.softwareheritage.org/oidc/login/>`_
 or `staging
-<https://webapp.staging.swh.network/oidc/login/?next_path=https://webapp.staging.swh.network/>`_
+<https://webapp.staging.swh.network/oidc/login/>`_
 instance.
 
 .. _swh-deposit-register-account-as-sysadm:

docs/index.rst

@@ -18,4 +18,12 @@ Reference Documentation
    :maxdepth: 2
 
    cli
-   /apidoc/swh.deposit
+
+.. only:: standalone_package_doc
+
+   Indices and tables
+   ------------------
+
+   * :ref:`genindex`
+   * :ref:`modindex`
+   * :ref:`search`

docs/specs/protocol-reference.rst

@@ -144,11 +144,30 @@ of Software Heritage.
 While CodeMeta is designed for use in JSON-LD, it is easy to reuse its vocabulary
 and embed it in an XML document, in three steps:
 
-1. use the JSON-LD compact representation of the CodeMeta document
-2. replace ``@context`` declarations with XML namespaces
-3. unfold JSON lists to sibling XML subtrees
-
-For example, this CodeMeta document:
+1. use the `JSON-LD compact representation`_ of the CodeMeta document with
+   ``@context: "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"`` and no other context;
+   which implies that:
+
+   1. Codemeta properties (whether in the ``https://codemeta.github.io/terms/``
+      or ``http://schema.org/`` namespaces) are unprefixed terms
+   2. other properties in the ``http://schema.org/`` namespace use `compact IRIs`_
+      with the ``schema`` prefix
+   3. other properties are absolute
+2. replace ``@context`` declarations with a XMLNS declaration with
+   ``https://doi.org/10.5063/SCHEMA/CODEMETA-2.0`` as namespace
+   (eg. ``xmlns="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"``
+   or ``xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"``)
+3. if using a non-default namespace, apply its prefix to any unprefixed term
+   (ie. any term defined in https://doi.org/10.5063/SCHEMA/CODEMETA-2.0 )
+4. add XMLNS declarations for any other prefix (eg. ``xmlns:schema="http://schema.org/"``
+   if any property in that namespace is used)
+5. unfold JSON lists to sibling XML subtrees
+
+.. _JSON-LD compact representation: https://www.w3.org/TR/json-ld11/#compacted-document-form
+.. _compact IRIs: https://www.w3.org/TR/json-ld11/#compact-iris
+
+Example Codemeta document
+"""""""""""""""""""""""""
 
 .. code:: json
 
@@ -201,6 +220,56 @@ Or, equivalently:
    </entry>
 
 
+Note that in both these examples, ``codemeta:name`` is used even though
+the property is actually ``http://schema.org/name``.
+
+Example generic JSON-LD document
+""""""""""""""""""""""""""""""""
+
+Another example using properties not part of Codemeta:
+
+.. code:: json
+
+   {
+      "@context": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
+      "name": "My Software",
+      "schema:sameAs": "http://example.org/my-software"
+   }
+
+which is equivalent to:
+
+.. code:: json
+
+   {
+      "@context": "https://doi.org/10.5063/SCHEMA/CODEMETA-2.0",
+      "name": "My Software",
+      "http://schema.org/sameAs": "http://example.org/my-software"
+   }
+
+becomes this XML document:
+
+.. code:: xml
+
+   <?xml version="1.0"?>
+   <atom:entry xmlns:atom="http://www.w3.org/2005/Atom"
+               xmlns="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"
+               xmlns:schema="http://schema.org/">
+     <name>My Software</name>
+     <schema:sameAs>http://example.org/my-software</schema:sameAs>
+   </atom:entry>
+
+Or, equivalently:
+
+.. code:: xml
+
+   <?xml version="1.0"?>
+   <entry xmlns="http://www.w3.org/2005/Atom"
+          xmlns:codemeta="https://doi.org/10.5063/SCHEMA/CODEMETA-2.0"
+          xmlns:schema="http://schema.org/">
+     <codemeta:name>My Software</codemeta:name>
+     <schema:sameAs>http://example.org/my-software</schema:sameAs>
+   </entry>
+
 .. _mandatory-attributes:
 
 Mandatory attributes

requirements-server.txt

-django >= 2, < 4
+django
 djangorestframework
-psycopg2 < 2.9
+psycopg2
 setuptools
 xmlschema

requirements-swh.txt

 swh.core[http] >= 0.4
 swh.model >= 4.4.0
+swh.scheduler

swh.deposit.egg-info/PKG-INFO

 Metadata-Version: 2.1
 Name: swh.deposit
-Version: 1.2.0
+Version: 1.2.1
 Summary: Software Heritage Deposit Server
 Home-page: https://forge.softwareheritage.org/source/swh-deposit/
 Author: Software Heritage developers
@@ -37,7 +37,7 @@ Description
 -----------
 
 Most of the software source code artifacts present in the SWH Archive are gathered by
-the mean of :term:`loader <loader>` workers run by the SWH project from sourve code
+the mean of :term:`loader <loader>` workers run by the SWH project from source code
 origins identified by :term:`lister <lister>` workers. This is a pull mechanism: it's
 the responsibility of the SWH project to gather and collect source code artifacts that
 way.

swh.deposit.egg-info/SOURCES.txt

@@ -185,6 +185,7 @@ swh/deposit/tests/conftest.py
 swh/deposit/tests/test_backend.py
 swh/deposit/tests/test_client_module.py
 swh/deposit/tests/test_common.py
+swh/deposit/tests/test_config.py
 swh/deposit/tests/test_gunicorn_config.py
 swh/deposit/tests/test_init.py
 swh/deposit/tests/test_migrations.py

swh.deposit.egg-info/requires.txt

@@ -4,11 +4,12 @@ requests
 sentry-sdk
 swh.core[http]>=0.4
 swh.model>=4.4.0
+swh.scheduler
 
 [server]
-django<4,>=2
+django
 djangorestframework
-psycopg2<2.9
+psycopg2
 setuptools
 xmlschema
 swh.core[http]>=0.4
@@ -31,9 +32,9 @@ djangorestframework-stubs>=1.4
 django-test-migrations
 types-requests
 types-pyyaml
-django<4,>=2
+django
 djangorestframework
-psycopg2<2.9
+psycopg2
 setuptools
 xmlschema
 swh.core[http]>=0.4

swh/deposit/api/common.py

@@ -584,15 +584,24 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
                 "in the multipart deposit",
             )
 
-        filehandler = data["application/zip"]
-        if not filehandler:
-            filehandler = data["application/x-tar"]
+        filehandler = data["application/zip"] or data["application/x-tar"]
 
+        if filehandler is None:
+            raise DepositError(
+                BAD_REQUEST,
+                "You must provide an archive, either as application/zip or "
+                "application/x-tar",
+            )
         assert isinstance(filehandler, UploadedFile), filehandler
 
         self._check_file_length(filehandler)
         self._check_file_md5sum(filehandler, headers.content_md5sum)
 
+        if data["application/atom+xml"] is None:
+            raise DepositError(
+                BAD_REQUEST, "You must provide an application/atom+xml entry."
+            )
+
         try:
             raw_metadata, metadata_tree = self._read_metadata(
                 data["application/atom+xml"]
@@ -620,7 +629,6 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
             deposit, deposit_request_data, replace_metadata, replace_archives
         )
 
-        assert filehandler is not None
         return Receipt(
             deposit_id=deposit.id,
             deposit_date=deposit.reception_date,
@@ -1010,9 +1018,7 @@ class APIBase(APIConfig, APIView, metaclass=ABCMeta):
 
         if self._client is None:
             try:
-                self._client = DepositClient.objects.get(  # type: ignore
-                    username=username
-                )
+                self._client = DepositClient.objects.get(username=username)
             except DepositClient.DoesNotExist:
                 raise DepositError(NOT_FOUND, f"Unknown client name {username}")
 

swh/deposit/api/private/urls.py

-# Copyright (C) 2017-2022  The Software Heritage developers
+# Copyright (C) 2017-2023  The Software Heritage developers
 # See the AUTHORS file at the top-level directory of this distribution
 # License: GNU General Public License version 3, or any later version
 # See top-level LICENSE file for more information
 
-from django.conf.urls import url
+from django.urls import re_path as url
 
 from ...config import (
     PRIVATE_CHECK_DEPOSIT,

swh/deposit/api/urls.py

-# Copyright (C) 2017-2021  The Software Heritage developers
+# Copyright (C) 2017-2023  The Software Heritage developers
 # See the AUTHORS file at the top-level directory of this distribution
 # License: GNU General Public License version 3, or any later version
 # See top-level LICENSE file for more information
@@ -7,8 +7,8 @@
 
 """
 
-from django.conf.urls import url
 from django.shortcuts import render
+from django.urls import re_path as url
 
 from ..config import COL_IRI, CONT_FILE_IRI, EDIT_IRI, EM_IRI, SD_IRI, SE_IRI, STATE_IRI
 from .collection import CollectionAPI

swh/deposit/cli/admin.py

@@ -114,7 +114,7 @@ def user_create(
 
     # user create/update
     try:
-        user = DepositClient.objects.get(username=username)  # type: ignore
+        user = DepositClient.objects.get(username=username)
         click.echo(f"Update user '{username}'.")
         action_done = "updated"
     except DepositClient.DoesNotExist:
@@ -166,7 +166,7 @@ def user_exists(ctx, username: str):
     from swh.deposit.models import DepositClient
 
     try:
-        DepositClient.objects.get(username=username)  # type: ignore
+        DepositClient.objects.get(username=username)
         click.echo(f"User {username} exists.")
         ctx.exit(0)
     except DepositClient.DoesNotExist:

swh/deposit/config.py

@@ -4,7 +4,7 @@
 # See top-level LICENSE file for more information
 
 import os
-from typing import Any, Dict
+from typing import Any, Dict, Optional
 
 from swh.core import config
 from swh.deposit import __version__
@@ -58,36 +58,37 @@ DEFAULT_CONFIG = {
 }
 
 
-def setup_django_for(platform=None, config_file=None):
-    """Setup function for command line tools (swh.deposit.create_user) to
-       initialize the needed db access.
+def setup_django_for(platform: Optional[str] = None, config_file: Optional[str] = None):
+    """Setup function for command line tools (e.g. swh.deposit.create_user) to
+    initialize the needed db access.
 
     Note:
         Do not import any django related module prior to this function
-        call. Otherwise, this will raise an
-        django.core.exceptions.ImproperlyConfigured error message.
+        call. Otherwise, this will raise a django.core.exceptions.ImproperlyConfigured
+        error message.
 
     Args:
-        platform (str): the platform the scheduling is running
-        config_file (str): Extra configuration file (typically for the
-                           production platform)
+        platform: the platform to use when running program (e.g. cli, ...)
+        config_file: Extra configuration file (typically for the production platform)
 
     Raises:
-        ValueError in case of wrong platform inputs.
+        ValueError in case of wrong platform inputs
 
     """
     if platform is not None:
         if platform not in AUTHORIZED_PLATFORMS:
-            raise ValueError("Platform should be one of %s" % AUTHORIZED_PLATFORMS)
+            raise ValueError(f"Platform should be one of {AUTHORIZED_PLATFORMS}")
         if "DJANGO_SETTINGS_MODULE" not in os.environ:
-            os.environ["DJANGO_SETTINGS_MODULE"] = "swh.deposit.settings.%s" % platform
+            os.environ["DJANGO_SETTINGS_MODULE"] = f"swh.deposit.settings.{platform}"
 
     if config_file:
+        # Hack to set the environment variable which in some cases is required (e.g.
+        # production)
         os.environ.setdefault("SWH_CONFIG_FILENAME", config_file)
 
-    import django
+    from django import setup
 
-    django.setup()
+    setup()
 
 
 class APIConfig:

swh/deposit/tests/api/test_collection_post_multipart.py

@@ -271,7 +271,7 @@ def test_post_deposit_multipart_put_to_replace_metadata(
 # FAILURE scenarios
 
 
-def test_post_deposit_multipart_only_archive_and_atom_entry(
+def test_post_deposit_multipart_only_one_archive_and_atom_entry(
     authenticated_client, deposit_collection
 ):
     """Multipart deposit only accepts one archive and one atom+xml"""
@@ -391,3 +391,81 @@ def test_post_deposit_multipart_if_upload_size_limit_exceeded(
 
     with pytest.raises(Deposit.DoesNotExist):
         Deposit.objects.get(external_id=external_id)
+
+
+def test_post_deposit_atom_400_multipart_no_atom(
+    authenticated_client, deposit_collection, atom_dataset, deposit_user, sample_archive
+):
+    """Posting without an atom body should return a 400 response"""
+    origin_url = "http://example.org/foo"
+
+    archive = InMemoryUploadedFile(
+        BytesIO(sample_archive["data"]),
+        field_name=sample_archive["name"],
+        name=sample_archive["name"],
+        content_type="application/x-tar",
+        size=sample_archive["length"],
+        charset=None,
+    )
+
+    atom_data = atom_dataset["entry-data0"] % origin_url
+
+    atom_entry = InMemoryUploadedFile(
+        BytesIO(atom_data.encode("utf-8")),
+        field_name="atom0",
+        name="atom0",
+        content_type="application/x-foobar",  # should be application/atom+xml
+        size=len(atom_data),
+        charset="utf-8",
+    )
+
+    response = authenticated_client.post(
+        reverse(COL_IRI, args=[deposit_collection.name]),
+        format="multipart",
+        data={
+            "archive": archive,
+            "atom_entry": atom_entry,
+        },
+    )
+
+    assert response.status_code == status.HTTP_400_BAD_REQUEST
+    assert "provide an application/atom+xml entry" in response.content.decode()
+
+
+def test_post_deposit_atom_400_multipart_no_archive(
+    authenticated_client, deposit_collection, atom_dataset, deposit_user, sample_archive
+):
+    """Posting without an atom body should return a 400 response"""
+    origin_url = "http://example.org/foo"
+
+    archive = InMemoryUploadedFile(
+        BytesIO(sample_archive["data"]),
+        field_name=sample_archive["name"],
+        name=sample_archive["name"],
+        content_type="application/x-foobar",  # should be application/x-tar
+        size=sample_archive["length"],
+        charset=None,
+    )
+
+    atom_data = atom_dataset["entry-data0"] % origin_url
+
+    atom_entry = InMemoryUploadedFile(
+        BytesIO(atom_data.encode("utf-8")),
+        field_name="atom0",
+        name="atom0",
+        content_type="application/atom+xml",
+        size=len(atom_data),
+        charset="utf-8",
+    )
+
+    response = authenticated_client.post(
+        reverse(COL_IRI, args=[deposit_collection.name]),
+        format="multipart",
+        data={
+            "archive": archive,
+            "atom_entry": atom_entry,
+        },
+    )
+
+    assert response.status_code == status.HTTP_400_BAD_REQUEST
+    assert "provide an archive" in response.content.decode()