Generate loader and lister tables from a common .yml

It is more compact, easier to edit, parseable, and I am planning to use it to generate other tables (eg. to track completion of sub-goals, such as collection of extrinsic metadata and indexation of intrinsic+extrinsic metadata)

As a side-effect, I updated Phabricator links to GitLab links

requested review from @anlambert

forge-support.yml seems to be missing (?)

gg me

Jenkins job DDOC/builds #250 failed .
See Console Output and Coverage Report for more details.

added 1 commit

• f75e8306 - Add missing forge-support.yml

Compare with previous version

Jenkins job DDOC/builds #251 failed .
See Console Output and Coverage Report for more details.

17:55:52  /var/lib/jenkins/workspace/DDOC/builds/swh-docs/docs/user/listers/index.rst:12: CRITICAL: Problems with "include" directive path:
17:55:52  InputError: [Errno 2] No such file or directory: 'user/listers/table.inc'.

added 1 commit

• 6149fd07 - wrangle the Makefiles to actually generate tables

Compare with previous version

Jenkins job DDOC/builds #253 failed .
See Console Output and Coverage Report for more details.

table content should be indented, I got this warnings when compiling the doc:

listers/table.inc:6: WARNING: Content block expected for the "table" directive; none found.

.. table::
  :align: center
loaders/table.inc:6: WARNING: Content block expected for the "table" directive; none found.

.. table::
  :align: center

added 3 commits

• 35112ed9 - fix py 3.7
• f92f8b6b - indent table
• a1d696f2 - Make dynamic-rst PHONY

Compare with previous version

Jenkins job DDOC/builds #254 succeeded .
See Console Output and Coverage Report for more details.

In order to generate the rst table, you should use the tabulate package, see diff below.

diff --git a/requirements.txt b/requirements.txt
index c2e9ddd..d42a251 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -11,3 +11,4 @@ pydata-sphinx-theme
 sphinx-click
 myst-parser
 sphinx-celery
+tabulate
diff --git a/swh/docs/generate_forge_list.py b/swh/docs/generate_forge_list.py
index a0ce862..78ca383 100644
--- a/swh/docs/generate_forge_list.py
+++ b/swh/docs/generate_forge_list.py
@@ -8,8 +8,10 @@ summarizing the status of loaders ands listers."""
 
 from pathlib import Path
 import sys
+import textwrap
 from typing import Literal, Tuple
 
+from tabulate import tabulate
 import yaml
 
 LISTERS_PRELUDE = """
@@ -36,15 +38,13 @@ LOADERS_PRELUDE = """
 
 def write_table(data, lister_or_loader: Literal["lister", "loader"], file) -> None:
     links = []
-    table = []
-    table.append(
-        (
-            f"{lister_or_loader.capitalize()} name",
-            "Related links",
-            "Current status",
-            "Related `grants <https://www.softwareheritage.org/grants/>`_",
-        )
+    headers = (
+        f"{lister_or_loader.capitalize()} name",
+        "Related links",
+        "Current status",
+        "Related `grants <https://www.softwareheritage.org/grants/>`_",
     )
+    table = []
     for (forge_id, forge) in sorted(data["forges"].items()):
 
         if lister_or_loader == "loader":
@@ -154,44 +154,13 @@ def write_table(data, lister_or_loader: Literal["lister", "loader"], file) -> No
 
         table.append((logo_cell, links_cell, status_cell, grant_cell))
 
-    cell_widths = (0, 0, 0, 0)
-    for row in table:
-        cell_widths = tuple(  # type: ignore[assignment]
-            max(cell_width, *map(len, cell.split("\n")))
-            for (cell_width, cell) in zip(cell_widths, row)
+    file.write(
+        textwrap.indent(
+            tabulate(table, headers=headers, tablefmt="grid"),
+            prefix="  ",
         )
-
-    def write_row(row: Tuple[str, str, str, str]) -> None:
-        nb_lines = max(cell.count("\n") + 1 for cell in row)
-        for subrow in range(nb_lines):
-            file.write(
-                "| "
-                + " | ".join(
-                    f"{{:{cell_width}}}".format(
-                        (cell + "\n" * nb_lines).split("\n")[subrow]
-                    )
-                    for (cell_width, cell) in zip(cell_widths, row)
-                )
-                + " |\n"
-            )
-
-    row_separator = (
-        "+-"
-        + "-" * cell_widths[0]
-        + "-+-"
-        + "-" * cell_widths[1]
-        + "-+-"
-        + "-" * cell_widths[2]
-        + "-+-"
-        + "-" * cell_widths[3]
-        + "-+\n"
+        + "\n"
     )
-    file.write(row_separator)
-    write_row(table[0])  # header
-    file.write(row_separator.replace("-", "="))
-    for row in table[1:]:
-        write_row(row)
-        file.write(row_separator)
 
     file.write("\n")

done, thx

There is something weird between listers table and loaders one: icons do not have the same size while previously it was the case. My guess is that it is related to some links that are now inlined.

it turned out to be a docutils version mismatch; old versions added this at the beginning of the table:

<colgroup>
<col style="width: 12%" />
<col style="width: 48%" />
<col style="width: 9%" />
<col style="width: 32%" />
</colgroup>

added 3 commits

• a417ffa0 - Use tabulate instead of manually formatting the table
• a533f552 - Fix broken refs
• 58211f48 - Fix typo

Compare with previous version

Jenkins job DDOC/builds #255 succeeded .
See Console Output and Coverage Report for more details.

approved this merge request

added 10 commits

• a67845ad - 1 commit from branch master
• 82d1eae9 - Generate loader and lister tables from a common .yml
• 3a4a9503 - Add missing forge-support.yml
• 88c8877c - wrangle the Makefiles to actually generate tables
• db0c4149 - fix py 3.7
• c614c7bb - indent table
• 47a3ef37 - Make dynamic-rst PHONY
• 6197d056 - Use tabulate instead of manually formatting the table
• 75d284a9 - Fix broken refs
• f8e037e9 - Fix typo

Compare with previous version

enabled an automatic merge when the pipeline for f8e037e9 succeeds

Jenkins job DDOC/builds #256 succeeded .
See Console Output and Coverage Report for more details.

merged