They cause postgresql to crash because it does not allow them in text fields.

They are seemingly only present accidentally in source documents;
so stripping them does not really impact the quality of metadata.

When capture_exceptions=False, the indexer's caller reports the exception
to Sentry itself.

However, because tags were added by indexers within a scope internal
to the indexers; the scope was closed before returning to the caller,
so these tags were actually not sent to Sentry.

Without these tags, it is often impossible to find what object
caused a given crash without guesswork based on the object's
content and swh-graph.

Invalid URLs are a common source of crashes

Detected through T4406

Related to T4412

So future services to be deployed can match on that name.

Related to T4406

The indexer language has been no longer running in production for years (and the related
indexer code has been pruned years ago as well). Those related endpoints are not
consumed by anyone. So we can drop those.

That will ease code maintenance and make the ci gain some time when running the overall
tests.

Related to T4273

It's been no longer running in production for years. That will ease code maintenance.

Related to T4273

Related to T4273

Related to T4273

intrinsic mappings use the short name, instead of the class name.

1. indexers call themselves directly instead of going through the scheduler
2. metadata is attached to directories instead of revisions

It makes resulting documents (usually) shorter, and tests more readable.

detect_metadata_files and extrinsic_metadata_formats (respectively) are somewhat
mutually exclusive, so it does not make much sense to have them in the same
class and MAPPINGS dict

It is already set by _translate_dict itself.

Which calls the GitHub mapping, for RawExtrinsicMetadata objects coming
from github

This introduces the scaffholding for extrinsic metadata mappings

We have many of those now; and keeping them all their tests in the same file
is messy

This causes these tests to run after Celery tests, which breaks them;
so this commit also renames Celery tests to make them run last.

Extrinsic metadata indexers will not use a 'file' as input,
but will typically use RawExtrinsicMetadata containing formats
in JSON.

This also moves the call to `detect_metadata()` to
`translate_directory_intrinsic_metadata` so type annotations make more
sense; and remove a dead/broken code branch in
`DirectoryMetadataIndexer.index()` that was detected by mypy.

_STORAGE_CLASSES will be renamed to OBJSTORAGE_IMPLEMENTATIONS.