that recursively merge 2 (config) dictionaries.
This is meant to be used to simplify a bit the config system of several
systems of the swh platform (related to T1410 and T826).

and raise logging level when loading a config file to info.

This class is meant to be used not only to wrap storage-like db.

Also rewrite the test_logger using pytest-postgresql fixture so
we do not need pifpaf anymore.

Using these 2 negotiate formatters, it is possible to write Flask endpoints
as:

  from flask import Flask
  from negotiate.flask import negotiate
  from swh.core.api import JSONFormatter, MshpackFormatter

  app = Flask()

  @app.route('/somewhere')
  @negotiate(MsgpackFormatter)
  @negotiate(JSONFormatter)
  def somewhere():
      return {'some': 'content'}

See https://github.com/nickstenning/negotiate

prometheus-statsd-exporter uses the datadog format for its tags, so base our
client on their Python code, with the following modifications:

 - Removed python < 3.5 compat code
 - trimmed the imports down to be a single module
 - move to time.monotonic() instead of time.time()
 - adjust some options:
   - drop unix socket connection option
   - add environment variable support for setting the statsd host and
     port (pulled the idea from the main python statsd module)
   - only send timer metrics in milliseconds (that's what
     prometheus-statsd-exporter expects)
   - drop DataDog-specific metric types (that are unsupported in
     prometheus-statsd-exporter)
 - made the tags a dict instead of a list (prometheus-statsd-exporter only
   supports tags with a value, mirroring prometheus)
 - improve unit test coverage
 - documentation cleanup

Related D816?id=2603#inline-4587

The first time, the db will be initialized.
The next time this is called, this will do nothing.

Loader-tar defines a utils.grouper as well whose sole difference is
the need for the fillvalue definition to a tuple of (None, None).
This will allow to use that main function instead.

Related T1411

related to T1330

Summary: PendingDeprecationWarning: encoding is deprecated, Use raw=False instead.

Reviewers: #reviewers, zack

Reviewed By: #reviewers, zack

Subscribers: zack, olasd, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D628

Summary: DbTestFixture.setupClass reads `cls._DB_DUMP_LIST`, which is set by `cls.add_db`, so `cls.add_db` should be called by the child's `setupClass` beforehand

Reviewers: #reviewers, ardumont

Reviewed By: #reviewers, ardumont

Subscribers: ardumont, swh-public-ci

Differential Revision: https://forge.softwareheritage.org/D603

... and make flake8 happy that way

The repr format of datetime.timedeltas has changed in Python 3.7, and the
construct is very unsafe anyway.

silent deprecation warnings.

makes swh-vault's tests a bit easier to write.

related to T1265.

used to initialise a database for a given (swh) module, like:

  swh-db-init storage

based on sql files shipped within the python package. This require several
small refactorings of SingleDbTestFixture.setUpClass and .add_db() methods:

- replace the 2 arguments (dump, dump_type) by a single 'dumps' one; this later
  must be an iterable of couples (dump_file, dump_type)
- refactor all needed helper functions accordingly.

We also add a small helper function to sort numerical file names, and provide a
DB_DUMP_TYPES as a global variable to prepare the next commit in which a new
cli tool 'swh-db-init' is added.

Related to T1265.

it's at best useless.