Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • anlambert/swh-model
  • lunar/swh-model
  • franckbret/swh-model
  • douardda/swh-model
  • olasd/swh-model
  • swh/devel/swh-model
  • Alphare/swh-model
  • samplet/swh-model
  • marmoute/swh-model
  • rboyer/swh-model
10 results
Show changes
Hide whitespace changes
Inline Side-by-side
Showing
with 125 additions and 665 deletions
docs/Makefile.local deleted 100644 → 0
View file @ f97d2165
sphinx/html: images
sphinx/clean: clean-images
assets: images
images:
make -C images/
clean-images:
make -C images/ clean
.PHONY: images clean-images
# Local Variables:
# mode: makefile
# End:
docs/_static/.placeholder deleted 100644 → 0
View file @ f97d2165
docs/_templates/.placeholder deleted 100644 → 0
View file @ f97d2165
docs/cli.rst deleted 100644 → 0
View file @ f97d2165
Command-line interface
======================
.. click:: swh.model.cli:identify
:prog: swh identify
:show-nested:
docs/conf.py deleted 100644 → 0
View file @ f97d2165
from swh.docs.sphinx.conf import * # NoQA
docs/data-model.rst deleted 100644 → 0
View file @ f97d2165
.. _data-model:
Data model
==========
.. note:: The text below is adapted from §7 of the article `Software Heritage:
Why and How to Preserve Software Source Code
<https://hal.archives-ouvertes.fr/hal-01590958/>`_ (in proceedings of `iPRES
2017 <https://ipres2017.jp/>`_, 14th International Conference on Digital
Preservation, by Roberto Di Cosmo and Stefano Zacchiroli), which also
provides a more general description of Software Heritage for the digital
preservation research community.
In any archival project the choice of the underlying data model—at the logical
level, independently from how data is actually stored on physical media—is
paramount. The data model adopted by Software Heritage to represent the
information that it collects is centered around the notion of *software
artifact*, described below.
It is important to notice that according to our principles, we must store with
every software artifact full information on where it has been found
(provenance), that is also captured in our data model, so we start by providing
some basic information on the nature of this provenance information.
Source code hosting places
--------------------------
Currently, Software Heritage uses of a curated list of source code hosting
places to crawl. The most common entries we expect to place in such a list are
popular collaborative development forges (e.g., GitHub, Bitbucket), package
manager repositories that host source package (e.g., CPAN, npm), and FOSS
distributions (e.g., Fedora, FreeBSD). But we may of course allow also more
niche entries, such as URLs of personal or institutional project collections
not hosted on major forges.
While currently entirely manual, the curation of such a list might easily be
semi-automatic, with entries suggested by fellow archivists and/or concerned
users that want to notify Software Heritage of the need of archiving specific
pieces of endangered source code. This approach is entirely compatible with
Web-wide crawling approaches: crawlers capable of detecting the presence of
source code might enrich the list. In both cases the list will remain curated,
with (semi-automated) review processes that will need to pass before a hosting
place starts to be used.
Software artifacts
------------------
Once the hosting places are known, they will need to be periodically looked at
in order to add to the archive missing software artifacts. Which software
artifacts will be found there?
In general, each software distribution mechanism hosts multiple releases of a
given software at any given time. For VCS (Version Control Systems), this is
the natural behaviour; for software packages, while a single version of a
package is just a snapshot of the corresponding software product, one can often
retrieve both current and past versions of the package from its distribution
site.
By reviewing and generalizing existing VCS and source package formats, we have
identified the following recurrent artifacts as commonly found at source code
hosting places. They form the basic ingredients of the Software Heritage
archive. As the terminology varies quite a bit from technology to technology,
we provide below both the canonical name used in Software Heritage and popular
synonyms.
**contents** (AKA "blobs")
the raw content of (source code) files as a sequence of bytes, without file
names or any other metadata. File contents are often recurrent, e.g., across
different versions of the same software, different directories of the same
project, or different projects all together.
**directories**
a list of named directory entries, each of which pointing to other artifacts,
usually file contents or sub-directories. Directory entries are also
associated to arbitrary metadata, which vary with technologies, but usually
includes permission bits, modification timestamps, etc.
**revisions** (AKA "commits")
software development within a specific project is essentially a time-indexed
series of copies of a single "root" directory that contains the entire
project source code. Software evolves when a developer modifies the content
of one or more files in that directory and record their changes.
Each recorded copy of the root directory is known as a "revision". It points
to a fully-determined directory and is equipped with arbitrary metadata. Some
of those are added manually by the developer (e.g., commit message), others
are automatically synthesized (timestamps, preceding commit(s), etc).
**releases** (AKA "tags")
some revisions are more equals than others and get selected by developers as
denoting important project milestones known as "releases". Each release
points to the last commit in project history corresponding to the release and
might carry arbitrary metadata—e.g., release name and version, release
message, cryptographic signatures, etc.
Additionally, the following crawling-related information are stored as
provenance information in the Software Heritage archive:
**origins**
code "hosting places" as previously described are usually large platforms
that host several unrelated software projects. For software provenance
purposes it is important to be more specific than that.
Software origins are fine grained references to where source code artifacts
archived by Software Heritage have been retrieved from. They take the form of
``(type, url)`` pairs, where ``url`` is a canonical URL (e.g., the address at
which one can ``git clone`` a repository or download a source tarball) and
``type`` the kind of software origin (e.g., git, svn, or dsc for Debian
source packages).
..
**projects**
as commonly intended are more abstract entities that precise software
origins. Projects relate together several development resources, including
websites, issue trackers, mailing lists, as well as software origins as
intended by Software Heritage.
The debate around the most apt ontologies to capture project-related
information for software hasn't settled yet, but the place projects will take
in the Software Heritage archive is fairly clear. Projects are abstract
entities, which will be arbitrarily nestable in a versioned
project/sub-project hierarchy, and that can be associated to arbitrary
metadata as well as origins where their source code can be found.
**snapshots**
any kind of software origin offers multiple pointers to the "current" state
of a development project. In the case of VCS this is reflected by branches
(e.g., master, development, but also so called feature branches dedicated to
extending the software in a specific direction); in the case of package
distributions by notions such as suites that correspond to different maturity
levels of individual packages (e.g., stable, development, etc.).
A "snapshot" of a given software origin records all entry points found there
and where each of them was pointing at the time. For example, a snapshot
object might track the commit where the master branch was pointing to at any
given time, as well as the most recent release of a given package in the
stable suite of a FOSS distribution.
**visits**
links together software origins with snapshots. Every time an origin is
consulted a new visit object is created, recording when (according to
Software Heritage clock) the visit happened and the full snapshot of the
state of the software origin at the time.
.. note::
This model currently records visits as a single point in time. However, the
actual visit process is not instantaneous. Loaders can record successive
changes to the state of the visit, as their work progresses, as updates to
the visit object.
Data structure
--------------
.. _swh-merkle-dag:
.. figure:: images/swh-merkle-dag.svg
:width: 1024px
:align: center
Software Heritage archive as a Merkle DAG, augmented with crawling
information (click to zoom).
With all the bits of what we want to archive in place, the next question is how
to organize them, i.e., which logical data structure to adopt for their
storage. A key observation for this decision is that source code artifacts are
massively duplicated. This is so for several reasons:
* code hosting diaspora (i.e., project development moving to the most
recent/cool collaborative development technology over time);
* copy/paste (AKA "vendoring") of parts or entire external FOSS software
components into other software products;
* large overlap between revisions of the same project: usually only a very
small amount of files/directories are modified by a single commit;
* emergence of DVCS (distributed version control systems), which natively work
by replicating entire repository copies around. GitHub-style pull requests
are the pinnacle of this, as they result in creating an additional repository
copy at each change done by a new developer;
* migration from one VCS to another—e.g., migrations from Subversion to Git,
which are really popular these days—resulting in additional copies, but in a
different distribution format, of the very same development histories.
These trends seem to be neither stopping nor slowing down, and it is reasonable
to expect that they will be even more prominent in the future, due to the
decreasing costs of storage and bandwidth.
For this reason we argue that any sustainable storage layout for archiving
source code in the very long term should support deduplication, allowing to pay
for the cost of storing source code artifacts that are encountered more than
once only once. For storage efficiency, deduplication should be supported for
all the software artifacts we have discussed, namely: file contents,
directories, revisions, releases, snapshots.
Realizing that principle, the Software Heritage archive is conceptually a
single (big) `Merkle Direct Acyclic Graph (DAG)
<https://en.wikipedia.org/wiki/Merkle_tree>`_, as depicted in Figure
:ref:`Software Heritage Merkle DAG <swh-merkle-dag>`. In such a graph each of
the artifacts we have described—from file contents up to entire
snapshots—correspond to a node. Edges between nodes emerge naturally:
directory entries point to other directories or file contents; revisions point
to directories and previous revisions, releases point to revisions, snapshots
point to revisions and releases. Additionally, each node contains all metadata
that are specific to the node itself rather than to pointed nodes; e.g., commit
messages, timestamps, or file names. Note that the structure is really a DAG,
and not a tree, due to the fact that the line of revisions nodes might be
forked and merged back.
..
directory: fff3cc22cb40f71d26f736c082326e77de0b7692
parent: e4feb05112588741b4764739d6da756c357e1f37
author: Stefano Zacchiroli <zack@upsilon.cc>
date: 1443617461 +0200
committer: Stefano Zacchiroli <zack@upsilon.cc>
commiter_date: 1443617461 +0200
message:
objstorage: fix tempfile race when adding objects
Before this change, two workers adding the same
object will end up racing to write <SHA1>.tmp.
[...]
revisionid: 64a783216c1ec69dcb267449c0bbf5e54f7c4d6d
A revision node in the Software Heritage DAG
In a Merkle structure each node is identified by an intrinsic identifier
computed as a cryptographic hash of the node content. In the case of Software
Heritage identifiers are computed taking into account both node-specific
metadata and the identifiers of child nodes.
Consider the revision node in the picture whose identifier starts with
`c7640e08d..`. it points to a directory (identifier starting with
`45f0c078..`), which has also been archived. That directory contains a full
copy, at a specific point in time, of a software component—in the example the
`Hello World <https://forge.softwareheritage.org/source/helloworld/>`_ software
component available on our forge. The revision node also points to the
preceding revision node (`43ef7dcd..`) in the project development history.
Finally, the node contains revision-specific metadata, such as the author and
committer of the given change, its timestamps, and the message entered by the
author at commit time.
The identifier of the revision node itself (`c7640e08d..`) is computed as a
cryptographic hash of a (canonical representation of) all the information shown
in figure. A change in any of them—metadata and/or pointed nodes—would result
in an entirely different node identifier. All other types of nodes in the
Software Heritage archive behave similarly.
The Software Heritage archive inherits useful properties from the underlying
Merkle structure. In particular, deduplication is built-in. Any software
artifacts encountered in the wild gets added to the archive only if a
corresponding node with a matching intrinsic identifier is not already
available in the graph—file content, commits, entire directories or project
snapshots are all deduplicated incurring storage costs only once.
Furthermore, as a side effect of this data model choice, the entire development
history of all the source code archived in Software Heritage—which ambitions to
match all published source code in the world—is available as a unified whole,
making emergent structures such as code reuse across different projects or
software origins, readily available. Further reinforcing the Software Heritage
use cases, this object could become a veritable "map of the stars" of our
entire software commons.
docs/images/.gitignore deleted 100644 → 0
View file @ f97d2165
swh-merkle-dag.pdf
swh-merkle-dag.svg
docs/images/Makefile deleted 100644 → 0
View file @ f97d2165
MERKLE_DAG = swh-merkle-dag.pdf swh-merkle-dag.svg
BUILD_TARGETS =
BUILD_TARGETS += $(MERKLE_DAG)
all: $(BUILD_TARGETS)
%.svg: %.dia
inkscape -l $@ $<
%.pdf: %.dia
inkscape -A $@ $<
clean:
-rm -f $(BUILD_TARGETS)
docs/images/swh-merkle-dag.dia deleted 100644 → 0
View file @ f97d2165
File deleted
docs/index.rst deleted 100644 → 0
View file @ f97d2165
.. _swh-model:
Software Heritage - Data model
==============================
Implementation of the :ref:`data-model` to archive source code artifacts.
.. toctree::
:caption: Overview:
:titlesonly:
data-model
persistent-identifiers
cli
/apidoc/swh.model
docs/persistent-identifiers.rst deleted 100644 → 0
View file @ f97d2165
.. _persistent-identifiers:
=================================================
SoftWare Heritage persistent IDentifiers (SWHIDs)
=================================================
**version 1.3, last modified 2020-04-28**
Overview
========
You can point to objects present in the Software Heritage archive by the means
of **SoftWare Heritage persistent IDentifiers**, or **SWHIDs** for short, that
are guaranteed to remain stable (persistent) over time. Their syntax, meaning,
and usage is described below. Note that they are identifiers and not URLs, even
though URL-based resolvers for SWHIDs are also available.
A SWHID can point to any software artifact (or "object") available in the
Software Heritage archive. Objects come in different types:
* contents
* directories
* revisions
* releases
* snapshots
Each object is identified by an intrinsic, type-specific object identifier that
is embedded in its SWHID as described below. The intrinsic identifiers embedded
in SWHIDs are strong cryptographic hashes computed on the entire set of object
properties. Together, these identifiers form a `Merkle structure
<https://en.wikipedia.org/wiki/Merkle_tree>`_, specifically a Merkle DAG.
See the :ref:`Software Heritage data model <data-model>` for an overview of
object types and how they are linked together. See
:py:mod:`swh.model.identifiers` for details on how the intrinsic identifiers
embedded in SWHIDs are computed.
Syntax
------
Syntactically, SWHIDs are generated by the ``<identifier>`` entry point of the
grammar:
.. code-block:: bnf
<identifier> ::= "swh" ":" <scheme_version> ":" <object_type> ":" <object_id> ;
<scheme_version> ::= "1" ;
<object_type> ::=
"snp" (* snapshot *)
| "rel" (* release *)
| "rev" (* revision *)
| "dir" (* directory *)
| "cnt" (* content *)
;
<object_id> ::= 40 * <hex_digit> ; (* intrinsic object id, as hex-encoded SHA1 *)
<dec_digit> ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
<hex_digit> ::= <dec_digit> | "a" | "b" | "c" | "d" | "e" | "f" ;
Semantics
---------
``:`` is used as separator between the logical parts of SWHIDs. The ``swh``
prefix makes explicit that these identifiers are related to *SoftWare
Heritage*. ``1`` (``<scheme_version>``) is the current version of this
identifier *scheme*; future editions will use higher version numbers, possibly
breaking backward compatibility (but without breaking the resolvability of
SWHIDs that conform to previous versions of the scheme).
A SWHID points to a single object, whose type is explicitly captured by
``<object_type>``:
* ``snp`` to **snapshots**,
* ``rel`` to **releases**,
* ``rev`` to **revisions**,
* ``dir`` to **directories**,
* ``cnt`` to **contents**.
The actual object pointed to is identified by the intrinsic identifier
``<object_id>``, which is a hex-encoded (using lowercase ASCII characters) SHA1
computed on the content and metadata of the object itself, as follows:
* for **snapshots**, intrinsic identifiers are computed as per
:py:func:`swh.model.identifiers.snapshot_identifier`
* for **releases**, as per
:py:func:`swh.model.identifiers.release_identifier`
* for **revisions**, as per
:py:func:`swh.model.identifiers.revision_identifier`
* for **directories**, as per
:py:func:`swh.model.identifiers.directory_identifier`
* for **contents**, the intrinsic identifier is the ``sha1_git`` hash of the
multiple hashes returned by
:py:func:`swh.model.identifiers.content_identifier`, i.e., the SHA1 of a byte
sequence obtained by juxtaposing the ASCII string ``"blob"`` (without
quotes), a space, the length of the content as decimal digits, a NULL byte,
and the actual content of the file.
Git compatibility
~~~~~~~~~~~~~~~~~
SWHIDs for contents, directories, revisions, and releases are, at present,
compatible with the `Git <https://git-scm.com/>`_ way of `computing identifiers
<https://git-scm.com/book/en/v2/Git-Internals-Git-Objects>`_ for its objects.
A SWHID for a content object will correspond (in its ``<object_id>`` part) to a
Git blob identifier of any file with the same content; a SWHID for a revision
will correspond to the Git commit identifier for the same revision, etc. This
is not the case for snapshot identifiers, as Git does not have a corresponding
object type.
Note that Git compatibility is incidental and is not guaranteed to be
maintained in future versions of this scheme (or Git).
Examples
--------
* ``swh:1:cnt:94a9ed024d3859793618152ea559a168bbcbb5e2`` points to the content
of a file containing the full text of the GPL3 license
* ``swh:1:dir:d198bc9d7a6bcf6db04f476d29314f157507d505`` points to a directory
containing the source code of the Darktable photography application as it was
at some point on 4 May 2017
* ``swh:1:rev:309cf2674ee7a0749978cf8265ab91a60aea0f7d`` points to a commit in
the development history of Darktable, dated 16 January 2017, that added
undo/redo supports for masks
* ``swh:1:rel:22ece559cc7cc2364edc5e5593d63ae8bd229f9f`` points to Darktable
release 2.3.0, dated 24 December 2016
* ``swh:1:snp:c7c108084bc0bf3d81436bf980b46e98bd338453`` points to a snapshot
of the entire Darktable Git repository taken on 4 May 2017 from GitHub
Contextual information
======================
The SWHIDs as described above are *intrinsic identifiers*, as they are computed
from the designated object itself, and it is often useful to provide
*contextual information* about a particular occurrence of the object, like the
origin from where the object has been found. To this end, SWHIDs can be
coupled with **qualifiers** that capture such *contextual information*.
Qualifiers come in different kinds:
* origin
* visit
* anchor
* path
* lines
Syntax
------
The full-syntax to complement SWHIDs with contextual information is given by
the ``<identifier_with_context>`` entry point of the grammar:
.. code-block:: bnf
<identifier_with_context> ::= <identifier> [ <qualifierlist> ]
<qualifierlist> := <qualifier> [ <qualifierlist> ]
<qualifier> ::= <origin_ctxt> | <visit_ctxt> | <anchor_ctxt> | <path_ctxt> |<lines_ctxt>
<origin_ctxt> ::= ";" "origin" "=" <url_escaped>
<visit_ctxt> ::= ";" "visit" "=" <identifier>
<anchor_ctxt> ::= ";" "anchor" "=" <identifier>
<path_ctxt> ::= ";" "path" "=" <path_absolute_escaped>
<lines_ctxt> ::= ";" "lines" "=" <line_number> ["-" <line_number>]
<line_number> ::= <dec_digit> +
<url_escaped> ::= (* RFC 3986 compliant URLs, percent-escaped *)
<path_absolute_escaped> ::= (* RFC 3986 compliant absolute file path, percent-escaped *)
Where:
- ``<path_absolute_escaped>`` is an ``<ipath-absolute>`` from `RFC 3987`_, and
- ``<url_escaped>`` is a `RFC 3987`_ IRI
in either case all occurrences of ``;`` (and ``%``, as required by the RFC)
have been percent-encoded (as ``%3B`` and ``%25`` respectively). Other
characters *can* be percent-encoded, e.g., to improve readability and/or
embeddability of SWHID in other contexts.
.. _RFC 3987: https://tools.ietf.org/html/rfc3987
Semantics
---------
``;`` is used as separator between SWHIDs and the optional contextual
information qualifiers. Each contextual information qualifier is specified as a
key/value pair, using ``=`` as a separator.
The following piece of contextual information are supported:
* **origin** : the *software origin* where an object has been found or observed
in the wild, as an URI;
* **visit** : persistent identifier of a *snapshot* corresponding to a specific
*visit* of a repository containing the designated object;
* **anchor** : a *designated node* in the Merkle DAG relative to which a *path
to the object* is specified, as a persistent identifier of a directory, a
revision, a release or a snapshot;
* **path** : the *absolute file path*, from the *root directory* associated to
the *anchor node*, to the object; when the anchor denotes a directory or a
revision, and almost always when it's a release, the root directory is
uniquely determined; when the anchor denotes a snapshot, the root directory
is the one pointed to by ``HEAD`` (possibly indirectly), and undefined if
such a reference is missing;
* **lines** : *line number(s)* of interest, usually within a content object
We recommend to equip identifiers meant to be shared with as many qualifiers as
possible. While qualifiers may be listed in any order, it is good practice to
present them in the order given above, i.e., ``origin``, ``visit``, ``anchor``,
``path``, ``lines``. Redundant information should be omitted: for example, if
the *visit* is present, and the *path* is relative to the snapshot indicated
there, then the *anchor* qualifier is superfluous.
Example
-------
The following `fully qualified SWHID
<https://archive.softwareheritage.org/swh:1:cnt:4d99d2d18326621ccdd70f5ea66c2e2ac236ad8b;;origin=https://gitorious.org/ocamlp3l/ocamlp3l_cvs.git;visit=swh:1:snp:d7f1b9eb7ccb596c2622c4780febaa02549830f9;anchor=swh:1:rev:2db189928c94d62a3b4757b3eec68f0a4d4113f0;path=/Examples/SimpleFarm/simplefarm.ml;lines=9-15>`_
denotes the lines 9 to 15 of a file content that can be found at absolute path
``/Examples/SimpleFarm/simplefarm.ml`` from the root directory of the revision
``swh:1:rev:2db189928c94d62a3b4757b3eec68f0a4d4113f0`` that is contained in the
snapshot ``swh:1:snp:d7f1b9eb7ccb596c2622c4780febaa02549830f9`` taken from the
origin ``https://gitorious.org/ocamlp3l/ocamlp3l_cvs.git``.
.. code-block:: url
swh:1:cnt:4d99d2d18326621ccdd70f5ea66c2e2ac236ad8b;
origin=https://gitorious.org/ocamlp3l/ocamlp3l_cvs.git;
visit=swh:1:snp:d7f1b9eb7ccb596c2622c4780febaa02549830f9;
anchor=swh:1:rev:2db189928c94d62a3b4757b3eec68f0a4d4113f0;
path=/Examples/SimpleFarm/simplefarm.ml;
lines=9-15
And this is an example of `a fully qualified SWHID with a percent escaped file
path
<https://archive.softwareheritage.org/swh:1:cnt:f10371aa7b8ccabca8479196d6cd640676fd4a04;origin=https://github.com/web-platform-tests/wpt;visit=swh:1:snp:b37d435721bbd450624165f334724e3585346499;anchor=swh:1:rev:259d0612af038d14f2cd889a14a3adb6c9e96d96;path=/html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/support/x%3Burl=foo/>`_
.. code-block:: url
swh:1:cnt:f10371aa7b8ccabca8479196d6cd640676fd4a04;
origin=https://github.com/web-platform-tests/wpt;
visit=swh:1:snp:b37d435721bbd450624165f334724e3585346499;
anchor=swh:1:rev:259d0612af038d14f2cd889a14a3adb6c9e96d96;
path=/html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/support/x%3Burl=foo/
Resolution
==========
Dedicated resolvers
-------------------
SWHIDs can be resolved using the Software Heritage Web application (see
:py:mod:`swh.web`). In particular, the **root endpoint** ``/`` can be given a
SWHID and will lead to the browsing page of the corresponding object, like
this: ``https://archive.softwareheritage.org/<identifier>``.
A **dedicated** ``/resolve`` **endpoint** of the HTTP API is also available to
explicitly request SWHID resolution; see: :http:get:`/api/1/resolve/(swh_id)/`.
Examples:
* `<https://archive.softwareheritage.org/swh:1:cnt:94a9ed024d3859793618152ea559a168bbcbb5e2>`_
* `<https://archive.softwareheritage.org/swh:1:dir:d198bc9d7a6bcf6db04f476d29314f157507d505>`_
* `<https://archive.softwareheritage.org/api/1/resolve/swh:1:rev:309cf2674ee7a0749978cf8265ab91a60aea0f7d>`_
* `<https://archive.softwareheritage.org/api/1/resolve/swh:1:rel:22ece559cc7cc2364edc5e5593d63ae8bd229f9f>`_
* `<https://archive.softwareheritage.org/api/1/resolve/swh:1:snp:c7c108084bc0bf3d81436bf980b46e98bd338453>`_
External resolvers
------------------
The following **independent resolvers** support resolution of SWHIDs:
* `Identifiers.org <https://identifiers.org>`_; see:
`<http://identifiers.org/swh/>`_ (registry identifier `MIR:00000655
<https://www.ebi.ac.uk/miriam/main/datatypes/MIR:00000655>`_).
* `Name-to-Thing (N2T) <https://n2t.net/>`_
Examples:
* `<https://identifiers.org/swh:1:cnt:94a9ed024d3859793618152ea559a168bbcbb5e2>`_
* `<https://identifiers.org/swh:1:dir:d198bc9d7a6bcf6db04f476d29314f157507d505>`_
* `<https://identifiers.org/swh:1:rev:309cf2674ee7a0749978cf8265ab91a60aea0f7d>`_
* `<https://n2t.net/swh:1:rel:22ece559cc7cc2364edc5e5593d63ae8bd229f9f>`_
* `<https://n2t.net/swh:1:snp:c7c108084bc0bf3d81436bf980b46e98bd338453>`_
Note that resolution via Identifiers.org does not support contextual
information, due to `syntactic incompatibilities
<http://identifiers.org/documentation#custom_requests>`_.
References
==========
* Roberto Di Cosmo, Morane Gruenpeter, Stefano Zacchiroli. `Identifiers for
Digital Objects: the Case of Software Source Code Preservation
<https://hal.archives-ouvertes.fr/hal-01865790v4>`_. In Proceedings of `iPRES
2018 <https://ipres2018.org/>`_: 15th International Conference on Digital
Preservation, Boston, MA, USA, September 2018, 9 pages.
* Roberto Di Cosmo, Morane Gruenpeter, Stefano Zacchiroli. `Referencing Source
Code Artifacts: a Separate Concern in Software Citation
<https://arxiv.org/abs/2001.08647>`_. In Computing in Science and
Engineering, volume 22, issue 2, pages 33-43. ISSN 1521-9615,
IEEE. March 2020.
mypy.ini deleted 100644 → 0
View file @ f97d2165
[mypy]
namespace_packages = True
warn_unused_ignores = True
# 3rd party libraries without stubs (yet)
[mypy-attrs_strict.*] # a bit sad, but...
ignore_missing_imports = True
[mypy-django.*] # false positive, only used my hypotesis' extras
ignore_missing_imports = True
[mypy-dulwich.*]
ignore_missing_imports = True
[mypy-iso8601.*]
ignore_missing_imports = True
[mypy-pkg_resources.*]
ignore_missing_imports = True
[mypy-pyblake2.*]
ignore_missing_imports = True
[mypy-pytest.*]
ignore_missing_imports = True
pytest.ini deleted 100644 → 0
View file @ f97d2165
[pytest]
addopts = --doctest-modules
norecursedirs = docs
setup.cfg
View file @ 05807020
[flake8]
# E203: whitespaces before ':' <https://github.com/psf/black/issues/315>
# E231: missing whitespace after ','
# W503: line break before binary operator <https://github.com/psf/black/issues/52>
ignore = E203,E231,W503
max-line-length = 88
[egg_info]
tag_build =
tag_date = 0
swh.model.egg-info/PKG-INFO 0 → 100644
View file @ 05807020
Metadata-Version: 2.1
Name: swh.model
Version: 0.0.69
Summary: Software Heritage data model
Home-page: https://forge.softwareheritage.org/diffusion/DMOD/
Author: Software Heritage developers
Author-email: swh-devel@inria.fr
License: UNKNOWN
Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest
Project-URL: Funding, https://www.softwareheritage.org/donate
Project-URL: Source, https://forge.softwareheritage.org/source/swh-model
Description: swh-model
=========
Implementation of the Data model of the Software Heritage project, used to
archive source code artifacts.
This module defines the notion of Persistent Identifier (PID) and provides
tools to compute them:
```sh
$ swh-identify fork.c kmod.c sched/deadline.c
swh:1:cnt:2e391c754ae730bd2d8520c2ab497c403220c6e3 fork.c
swh:1:cnt:0277d1216f80ae1adeed84a686ed34c9b2931fc2 kmod.c
swh:1:cnt:57b939c81bce5d06fa587df8915f05affbe22b82 sched/deadline.c
$ swh-identify --no-filename /usr/src/linux/kernel/
swh:1:dir:f9f858a48d663b3809c9e2f336412717496202ab
```
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: cli
Provides-Extra: testing
swh.model.egg-info/SOURCES.txt 0 → 100644
View file @ 05807020
MANIFEST.in
Makefile
README.md
pyproject.toml
requirements-cli.txt
requirements-test.txt
requirements.txt
setup.cfg
setup.py
version.txt
swh/__init__.py
swh.model.egg-info/PKG-INFO
swh.model.egg-info/SOURCES.txt
swh.model.egg-info/dependency_links.txt
swh.model.egg-info/entry_points.txt
swh.model.egg-info/requires.txt
swh.model.egg-info/top_level.txt
swh/model/__init__.py
swh/model/cli.py
swh/model/exceptions.py
swh/model/from_disk.py
swh/model/hashutil.py
swh/model/hypothesis_strategies.py
swh/model/identifiers.py
swh/model/merkle.py
swh/model/model.py
swh/model/py.typed
swh/model/toposort.py
swh/model/validators.py
swh/model/fields/__init__.py
swh/model/fields/compound.py
swh/model/fields/hashes.py
swh/model/fields/simple.py
swh/model/tests/__init__.py
swh/model/tests/generate_testdata.py
swh/model/tests/generate_testdata_from_disk.py
swh/model/tests/test_cli.py
swh/model/tests/test_from_disk.py
swh/model/tests/test_generate_testdata.py
swh/model/tests/test_hashutil.py
swh/model/tests/test_hypothesis_strategies.py
swh/model/tests/test_identifiers.py
swh/model/tests/test_merkle.py
swh/model/tests/test_model.py
swh/model/tests/test_toposort.py
swh/model/tests/test_validators.py
swh/model/tests/data/dir-folders/sample-folder.tgz
swh/model/tests/data/repos/sample-repo.tgz
swh/model/tests/fields/__init__.py
swh/model/tests/fields/test_compound.py
swh/model/tests/fields/test_hashes.py
swh/model/tests/fields/test_simple.py
\ No newline at end of file
swh.model.egg-info/dependency_links.txt 0 → 100644
View file @ 05807020
swh.model.egg-info/entry_points.txt 0 → 100644
View file @ 05807020
[console_scripts]
swh-identify=swh.model.cli:identify
[swh.cli.subcommands]
identify=swh.model.cli:identify
\ No newline at end of file
swh.model.egg-info/requires.txt 0 → 100644
View file @ 05807020
vcversioner
attrs
attrs_strict
hypothesis
python-dateutil
iso8601
[:python_version < "3.6"]
pyblake2
[cli]
swh.core
Click
dulwich
[testing]
Click
dulwich
pytest
pytz
swh.model.egg-info/top_level.txt 0 → 100644
View file @ 05807020
swh