Summary: provide functions to fill actual storages with them.

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: ardumont, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D739

provide functions to fill actual storages with them.

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D734

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: ardumont, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D732

Summary:
mimetype: Simplify test init

Bootstrap in-mem idx storage

Test the in-mem idx storage.

fossology_license: simplify test init.

re-fix mimetype

ctags: simplify test init.

ctags+language: simplify test init.

Simplify tests in swh/indexer/tests/test_metadata.py.

Reviewers: ardumont

Reviewed By: ardumont

Subscribers: swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D725

Summary:
origin_metadata_add's input does not look like origin_metadata_get's output.

Fix mock appropriately.

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D722

Related T818

Prior to this commit, this was passing along real instance.  This did
not bode well with the __init__ call which expects configuration dict.

Possibly there to satisfy tests so to remove.

Indexer storage expects identifier as bytes

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: ardumont, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D707

Make the OriginMetadataIndexer fetch rev metadata from the storage instead of getting them via the scheduler.

Summary: Depends on D704

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D705

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D704

related to T1330

Summary: Also fixes docstrings.

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: ardumont, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D702

Summary:
I made several choices while writing that Diff that are open to discussion:

- Using GIN instead of GiST. That seems the most appropriate choice when reading https://www.postgresql.org/docs/9.1/textsearch-indexes.html ; but these part got removed from the doc of pgsql 10: https://www.postgresql.org/docs/10/textsearch-indexes.html
- Using `pg_catalog.simple` as dictionary. Since we're dealing with any language and proper names, it seemed best to use a dictionary with no stop word. Though, arguably, most of the data will be English, and stop words usually don't appear in names.
- It only supports conjunctions of search terms. I could easily add support for arbitrary levels of nestings and disjunctions/negations. That can be done later if we deem it worth it.
- It indexes JSON keys too. It is probably possible to fix this, at the expanse of complicated SQL code, or some postprocessing in Python.

Resolves T1334 and T1335.

Test Plan: tox

Reviewers: #reviewers, ardumont, olasd

Reviewed By: #reviewers, ardumont, olasd

Subscribers: olasd, ardumont, swh-public-ci

Maniphest Tasks: T1335, T1334

Differential Revision: https://forge.softwareheritage.org/D658

Related T1326

Related T1326