Jérémy Bobbio (Lunar) authored 2 years ago and vlorentz committed 2 years ago

protobuf earlier 4.12.* versions crashed when `swhgraph_pb2.py`
was discovered by pytest. This has been fixed in the 4.12.11 release.
See: https://github.com/protocolbuffers/protobuf/issues/10151

Preventing protobuf 4.12.* to be used makes `pip` install grpcio-tools
version 1.49.0 which fails to build on Debian bookworm. In order to
allow more fixed versions of grpcio-tools to be used, we bump the
dependency on protobuf to version 4.12.11 or later.

vlorentz authored 2 years ago and vlorentz committed 2 years ago

to help debugging

This will be used as a metric for 'popularity' of directories,
which will be used to weigh results of PopularContents (which
counts the most popular names used to refer to each content)

vlorentz authored 2 years ago and vlorentz committed 2 years ago

It caused NodeProperties to OOM because it tried to allocate array 22 times
larger than needed.

vlorentz authored 2 years ago and vlorentz committed 2 years ago

It is 5 times faster

This fixes python 3.7 support due to poetry, a dependency of isort, that
removed support for that Python version in a recent release.

vlorentz authored 2 years ago and vlorentz committed 2 years ago

To be consistent with other CSV files in the license dataset.

Because the main table already contains (origin_id, person_id) pairs,
it is redundant to repeat every pair in this file.

They tend to be forgotten; this will make sure that running 'tox' will
run Java tests as well as Python tests.

It will also make it easier to run Java tests on the CI

two paths + reimport tasks at the package level so they are automatically
picked up by luigi without passing all module names on the CLI

whitelist_externals is deprecated since v3.18, and removed since v4.0

RPCServerApp inherits from aiohttp.web.Application, and GraphServerApp
does not use any of the features added by RPCServerApp

It's slightly more accurate

Origin pseudo-SWHIDs are kind of useless on their own, as there is no public API
to resolve them yet; so we might as well produce integers instead.

A future commit will add a mapping table to URLs, which are actually
useful.

In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Also include them the proper sphinx way.

Related to T4496

It is easier to keep track of it in Grafana.

RunAll is not practical, because we need to mount the graph on a tmpfs
at some point. Let's keep that step manual for now, which means
RunExportCompressUpload and DeanonymizeOriginContributors will be
called separately.