Extend new deposit endpoint to support metadata-only deposits

mentioned in merge request !134 (closed)

mentioned in merge request !142 (closed)

mentioned in merge request !143 (closed)

mentioned in merge request !342 (closed)

mentioned in merge request !343 (closed)

marked this issue as related to #2344

marked this issue as related to swh/meta#1021 (closed)

added Roadmap 2020 SWORD deposit Scientific Community Building priority:Normal labels

marked this issue as related to #2538 (closed)

marked this issue as related to #2539 (closed)

marked this issue as related to #2540 (closed)

Although, it might be challenging to do so, since the metadata deposit requires a SWHID and the process that follows is quite different from the deposit.

Maybe not so much in the end, we could use the slug ("suggested identifier").

Quoting the linked page:

The client MAY supply a Slug header providing a suggested identifier for the deposited content

If that slug is a SWHID (and it exists), then we infer the deposit client wants a metadata only deposit and progress as such.

After a talk with Bruno and Yannick on HAL, they say that depositing metadata is: 6.5.2. Replacing the Metadata of a Resource because the resource already exists on SWH and this should be a PUT of new (maybe new from scratch) metadata on an existing identifier (SWHID).

That's true if you consider the graph sub-dag to be the Resource. I assumed at Resource to be the deposit itself; but re-reading the SWORDv2 spec, it's not obvious which one is the right interpretation.

changed title from Extend software deposit endpoint to enable only metadata deposits to Extend software deposit endpoint to support metadata-only deposits

changed title from Extend software deposit endpoint to support metadata-only deposits to Extend new deposit endpoint to support metadata-only deposits

! In T2312#47921, @moranegg wrote: After this morning's meeting with @vlorentz and @ardumont: We will keep the metadata-only deposit specs with the idea of a separate namespace swh for which we need to write the schema (not sure we have that).

This way, the xml with metadata has a section where the identified artifact is mentioned:

Reference a snapshot, revision or release:
With ${type} in {snp (snapshot), rev (revision), rel (release) }:
<swh:deposit>
  <swh:reference>
    <swh:object id="swh:1:${type}:aaaaaaaaaaaaaa..."/>
  </swh:reference>
</swh:deposit>
We need to add to the list of types: directory and content

The possibility to deposit metadata on an origin should be implemented as well, but is not suited for institutional repositories (e.g HAL). Reference an origin:
<swh:deposit>
  <swh:reference>
    <swh:origin url="https://github.com/user/repo"/>
  </swh:reference>
</swh:deposit>
This specs fits the POST of a new deposit in SWORD and is described in the SWORD v2 documentation (6.3.3. Creating a Resource with an Atom Entry)

The sequence here is:

The client sends (POST) new metadata (without content)
In the xml a reference to the object or origin MUST be provided in a deposittag
The status of the deposit is verified
The metadata is injected into the MetaData Storage in its raw form
The status of the deposit is done

@vlorentz some comments on: https://docs.softwareheritage.org/devel/swh-deposit/specs/spec-meta-deposit.html

Th example is with HAL by referencing an origin, which is an unexisting scenarion Can you change in the example to:

<swh:deposit>
      <swh:reference>
        object swhid="swh:1:dir:31b5c8cc985d190b5a7ef4878128ebfdc2358f49;
                      origin=https://hal.archives-ouvertes.fr/hal-01243573;
                      visit=swh:1:snp:4fc1e36fca86b2070204bedd51106014a614f321;
                      anchor=swh:1:rev:9c5de20cfb54682370a398fcc733e829903c8cba;
                     path=/moranegg-AffectationRO-df7f68b/"
      </swh:reference>
    </swh:deposit>

Add this in requirements:

1. The metadata-only deposit is sent via SWORD protocol with a POST request the same as a classic deposit to a client's collection (see here https://docs.softwareheritage.org/devel/swh-deposit/spec-api.html#create-deposit)
2.  The metadata-only deposit is composed from ONLY one xml file containing all metadata
3. It MUST comply to the metadata requirements in https://docs.softwareheritage.org/devel/swh-deposit/metadata.html
4. It MUST reference an **object** or  an **origin**  in a deposit tag
5. The reference SHOULD exist in the SWH archive (to verify with upper management)
6. The **object** reference MUST be a SWHID on one of the following artifact types: 
 - snapshot
 - release
 - revision
 - directory
 - content
7. The SWHID MAY be simple (core SWHID) 
8. The SWHID MAY be complex with context (adding classifiers as documented here https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html)
8. The SWHID MUST NOT reference a fragment of code with the classifier `lines`

Should we change the link https://www.softwareheritage.org/schema/2018/deposit? which is just a redirection at the moment.. but 2018 might not be a good thing to have in a namespace url.
I have some diagrams in mermaid that I can transform to PlantUML that we can add to the use cases section.

add also an example for Core SWHID:

<swh:deposit>
      <swh:reference>
        object swhid="swh:1:dir:31b5c8cc985d190b5a7ef4878128ebfdc2358f49"
      </swh:reference>
    </swh:deposit>

In the case of the contextual SWHID, it is important to check if there are white spaces inside the SWHID before resolving with the archive

The last suggested adaptations are !395 (closed) !124 (closed).

Extend new deposit endpoint to support metadata-only deposits

Designs

Child items ...

Activity