Enable to check package documentation can be built without producing
sphinx warnings.

The sphinx environment is designed to be used in continuous integration
in order to prevent breaking documentation build when committing changes.

The sphinx-dev environment is designed to be used inside a full swh
development environment.

Related to T3258

Clean scan outputs for ndjson, json, and text formats

Reviewers: #reviewers

Differential Revision: https://forge.softwareheritage.org/D5211

pytest-flask removed the url method of its LiveServer class wrapped by
the live_server fixture.

So use recommended way to get an URL when using Flask, this requires
adding a dummy root endpoint to the API in order to get its base URL.

This makes DB import much more rosbut, at the price of ~15% performance
impact (which is still very much bearable, in light of recent DB import
improvements).  If needed, this cost can be mitigated in the future by
pipelining SWHID validation with SQLite insertion, using either a thread or
asyncio.

Closes T2813

- avoid loading into memory the entire input file, and rely on DB unicity
  constraints to spot duplicated

- tune sqlite disabling sync writes and journal; this is unsafe, but insertion
  is all or nothing anyway

- minor: improve exception handling, propagating SQLite errors up the stack

With this chagen import time for 30M SWHIDs went down from ~6m30s to ~55s, and
memory usage down from 5 GiB to a few tens MiB

Closes T2836
Closes T2812

- preserve error message from DBError exception
- do *not* remove DB if it exists, as it might help in debugging
- fail using click's fail() instead of bypassing it with sys.exit

thanks Alexios Zavras for the bug report and fix suggestion

Closes T2680

Serve the local sqlite database generated with the "db import" option.

The API endpoint is compatible with the one used in the Web API [1].

[1] https://archive.softwareheritage.org/api/1/known/doc/

in particular:
- make both -i and -o required, as they have no sane defaults ("-" can still be
  passed to -i to read from stdin)
- improve help message for that command

- import SWHIDs from an input file
- new database class to store local SWHIDs (extracted from the input file)

This fixes the build [1]

[1] https://jenkins.softwareheritage.org/job/DTSCN/job/tests/297/console

- add smoke and unit tests
- mock scanner to not run actual scan
- spy on config loading to verify how config is loaded
- use a sample test config file and add it to MANIFEST

Conflicts arised between multiple users of the same session-scoped
fixtures based on tmp_path_factory.
Fix both the fixture itself to avoid side-effects and tests based
on its inner workings.

It makes cli more testable and helps avoid ambiguity between uses of 'scan' and
'scanner' for the reader.

since the model already provide a function to iterate the child nodes,
_iter_nodes_attr is not useful

function to_dict updated to reflect changes

Closes T2690

Also rename private methods from leading 2 underscores to 1 to match
project coding style

Remove reliance on default arg child_nodes which is a dict.

It is unused in client code, breaks tests, and is not needed to build
the dict representation of the tree.

This also refines types on related impacted methods which helps
reasoning about them.

The docstring was moved out of scanner docstring spot to work around a
bug in our current Python version. See https://bugs.python.org/issue28739
Its formatting was also improved.

- make config_file optional
- fail if file does not exist instead of using empty config dict

Related to T2610

Related to T2610

flake8 hook has been removed from https://github.com/pre-commit/pre-commit-hooks
so now use the one from https://gitlab.com/pycqa/flake8

Replace PosixPath with Path, which astracts away environment detail, in the whole package