Related to T2367

I noticed this because my virtualenv ended up with the djangorestframework-stubs
package installed, which made the mypy pre-commit hook fail.

This commit adds drf-stubs to the tox.ini, and does the relevant updates to our
type annotations/signatures.

Turns out that upstream DRF has all `request` parameters named that way, so we
rename all our `req` parameters to match, which appeases mypy's signature
checks.

- blackify all the python files,
- enable black in pre-commit,
- add a black tox environment.

New latest release on swh.core added the environment parameter.
This will fix the current broken jenkins build [1]

[1] https://jenkins.softwareheritage.org/job/DDEP/job/tests/658/console

Related to T2188

Those tests were broken since rDCORE2255767b9763cefe0841a8d0231f568adcec87ff.

For this we:

- ensure we create the DepositRequest with str as raw_metadata (instead
  of a bytes), and
- make atom_dataset return a str instead of a bytes

  Every time this chunk of data is sent on the wire (http req to the django app)
  this needs to be an str, otherwise we have things like:

    raw_metadata = "b'[...]'"

  especially when testing against django 2, which is the only django version
  available on debian sid, so this is needed to make it possible to build the
  deb file on sid *with* tests enabled.
- add a py3-django2 environment in tox
- bump the dependency to django <3 in requirements-server.txt (otherwise it
  will take precendence on the deps: field of the tox file, thus execute tests
  in py3-django2 with django 1.11).

Note that no other tests than executing pytest have been done with django2, so
do not expect it works flawlessly.

This later no longer exists.

for the sake of consistency.

this code was the only user for the swh.core.tarball.compress function. Doing
so allows to reduce a bit the complexity and coupligin in the code. It will also
allow to get rid of the swh.core.tarball.compress function.

No need for this to be a subpackage.

This reflects what is done in practice and what the code expects.
Using an email is invalid because the value of --author is used
in a `http://schema.org/name` property, not `http://schema.org/email`

This new option allows these cli commands (upload and status) to produce
JSON or YAML output, so it's easier to test.

so the scheduler interaction code is executed.

Note that this does not test for correctness in these interactions yet.

also move tests/__init__.py content in tests/conftest.py and adapt test code
accordingly.

This also ensures retries_left is set otherwise tests may fail when using
the local sheduler.

Replace the (non-working) django init hook by overloaded django_db_setup
and make this django_db_setup use pytest-postgresql so we do not depend on
external test environment (running postgresql server, tox or hand managed
pifpaf).

even if there is only one of those, otherwise the loader will fail since
this later does expect the archive to be a zip file.

Also replace utils.origin_url_from() by a Deposit.origin_url property, and
ensure the archive file format of the uploaded file is supported.

The rename of invalid.tar.gz as invalid.gz is required for tests to pass
which should not be the case. It will be investigated in a later revision,
but for now we want tests to be green.

This requires to make config dict available in SWHPrivateAPIView so we can
have access to a scheduler from there.

ensuring these generalted URLs are correct and resolvable (eg. when the WSGI
app is mounted on a different path than /).

This needed to move the post_deposit_save code from a signal directly
within the view, so we have a request from which we can forge absolute URLs.

Mostly, the code used to allow having either --author or --name,
while it should use either both (when generating metadata) or
neither (when using a metadata file).

This code fixes this, and also catches some more invalid cases.

This uses a context-manager instead of manually handling
removing the file.

'return' statements were ignored, and log level was too low for an error.

Related to 0a9ec2a9

The deposit service only sends message for the current new deposit loader.

Related to T2140

Current djangos-stubs version (1.2.0) is not compatible with latest mypy one (0.750).
So pin their version until the issue is fixed.

Related T2024

so pre-commit is happy.