devel: apidoc generated ref documentation is not accessible from index pages

added activity::Documentation activity::MRO labels

assigned to @lunar

This indeed used to be the case at least on 2022-09-28 (see it on the Wayback Machine).

This is the result of recent changes by myself on the documentation structure. I moved the full swh module indices to the landing page in 0e55968d and remove the inclusion of a module index when building the full swh documentation doc in commits like swh-auth@4c595580 (to prevent warnings about rst file being multiple included).

I agree we should restore previous behavior, module index should be found from the index page of the swh package doc.

To do so, we should add a new sphinx flag when building the full swh documentation:

(swh) ✔ ~/swh/swh-environment/swh-docs/docs [master ↓·8|✚ 2⚑ 1] 
11:29 $ git diff
diff --git a/docs/Makefile b/docs/Makefile
index f034553..9f65121 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -3,7 +3,7 @@
 
 # You can set these variables from the command line, and also
 # from the environment for the first two.
-SPHINXOPTS  ?=
+SPHINXOPTS  ?= -t full_swh_doc
 SPHINXBUILD ?= sphinx-build
 SOURCEDIR    = .
 BUILDDIR     = _build

And modify the docs/index.rst file of each swh package the following way to include the module index only when building full swh doc:

(swh) ✔ ~/swh/swh-environment/swh-auth [mr-49 L|✚ 2] 
11:30 $ git diff
diff --git a/docs/index.rst b/docs/index.rst
index 9223441..ee4756f 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -11,6 +11,16 @@ Reference Documentation
    cli
    django
 
+.. only:: full_swh_doc
+
+   Module index
+   ------------
+
+   .. toctree::
+      :maxdepth: 2
+
+      /devel/apidoc/swh.auth
+
 
 .. only:: standalone_package_doc

Nervertheless, this breaks the standalone package doc build on Jenkins as sphinx still parses the content of the only directive even when the flag is not set (sigh) and emits a warning about the non existing /devel/apidoc/* file.

Fortunately, we can add a log filter when building standalone package doc to discard the warning and makes the doc build succeed.

(swh) ✔ ~/swh/swh-environment/swh-docs/docs [master ↓·8|✚ 2⚑ 1] 
11:29 $ git diff
diff --git a/swh/docs/sphinx/conf.py b/swh/docs/sphinx/conf.py
index 6f956f7..a301547 100755
--- a/swh/docs/sphinx/conf.py
+++ b/swh/docs/sphinx/conf.py
@@ -331,3 +331,11 @@ def setup(app):
 
             # insert a custom filter in the warning log handler of sphinx
             logger.handlers[1].filters.insert(0, HttpDomainRefWarningFilter())
+
+        # filter out warning about nonexisting '/devel/apidoc/*.rst file in the toc
+        # as it is only available when building the full swh documentation
+        class ApiDocNonExistingFilter(logging.Filter):
+            def filter(self, record: logging.LogRecord) -> bool:
+                return "/devel/apidoc/" in record.msg
+
+        logger.handlers[1].filters.insert(0, ApiDocNonExistingFilter())

unassigned @lunar

As the file included in the package index is predictable, would there be a way to modify that index on the fly instead?

We could try to implement a callback for the doctree-resolved event and modify the TOC tree on the fly indeed.

Or use a post_transform hook, see examples.

So I managed to include the apidoc reference in each swh package doc index dynamically when building the full swh documentation using this simple sphinx post-transform:

diff --git a/swh/docs/sphinx/conf.py b/swh/docs/sphinx/conf.py
index 6f956f7..f8d6350 100755
--- a/swh/docs/sphinx/conf.py
+++ b/swh/docs/sphinx/conf.py
@@ -5,9 +5,12 @@
 
 import logging
 import os
+import re
 from typing import Dict
 
+from sphinx import addnodes
 from sphinx.ext import autodoc
+from sphinx.transforms.post_transforms import SphinxPostTransform
 
 from swh.docs.django_settings import force_django_settings
 
@@ -331,3 +334,22 @@ def setup(app):
 
             # insert a custom filter in the warning log handler of sphinx
             logger.handlers[1].filters.insert(0, HttpDomainRefWarningFilter())
+
+    else:
+
+        class SwhPackageTocTreeAddApidoc(SphinxPostTransform):
+            default_priority = 700
+
+            def run(self, **kwargs):
+                match = re.match(r"^devel/(swh-[^/]+)/index$", self.env.docname)
+                if not match:
+                    return
+                swh_package = match.group(1).replace("-", ".")
+                swh_package_apidoc = f"devel/apidoc/{swh_package}"
+                nodes = list(self.document.findall(addnodes.toctree))
+                if nodes:
+                    main_toc = nodes[0]
+                    main_toc["entries"].append((None, swh_package_apidoc))
+                    main_toc["includefiles"].append(swh_package_apidoc)
+
+        app.add_post_transform(SwhPackageTocTreeAddApidoc)

I love sphinx for this kind of hack.

Great!

Any reason you didn't chose the source-read signal to do something similar as add_glossary_to_index?

source-read enables to modify a rst file content before its parsing by docutils while a post-transform enables to modify the docutils nodes before writing the final HTML.

mentioned in merge request !322 (merged)

closed with commit 26e47563

closed with merge request !322 (merged)

mentioned in commit 26e47563

devel: apidoc generated ref documentation is not accessible from index pages

Designs

Child items ...

Activity