In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Also include them the proper sphinx way.

Related to T4496

Motivation:

1. It makes it easier to visualize what is actually happening when modifying
   the graph, by working explicitly on triples instead of a JSON-LD (a tree
   serialization of the graph).

2. Remove the need for the hacky `merge_values()` function (and possibly
   `merge_documents()` in a future commit)

3. It also catches malformed data exactly where it is added in the document
   (the call to rdflib.Graph.add()) instead of at the end of the mapping
   when running compaction/expansion.

Downsides:

1. Tests are clunkier, because they relied on deterministic order of
   unordered lists; but rdflib does not guarantee it

2. Code is longer

3. Extra dependency (which we will need at some point if we want to
   import from RDF datasets, anyway)

1. indexers call themselves directly instead of going through the scheduler
2. metadata is attached to directories instead of revisions

detect_metadata_files and extrinsic_metadata_formats (respectively) are somewhat
mutually exclusive, so it does not make much sense to have them in the same
class and MAPPINGS dict

swh.storage and swh.objstorage, as well as swh.indexer itself, have deprecated
using an explicit `args` in their factories for a while; we can drop them now.

related to T1330

instead of cp it in the Makefile.

Reviewers: ardumont, #reviewers

Reviewed By: ardumont, #reviewers

Differential Revision: https://forge.softwareheritage.org/D515

resolves T1230

Related to T1230

Related T1230