Skip to content
Snippets Groups Projects
New look: Off

Fix documentation dev build (again)

    • Closed Issue created by Antoine R. Dumont @ardumont

    This prevents having diff (which builds green).

    Migrated from T4120 (view on Phabricator)

    Designs

      Child items ...

      2. Activity

      • Antoine R. Dumont mentioned in merge request !212 (closed)

        mentioned in merge request !212 (closed)

      • Antoine R. Dumont mentioned in issue #4122

        mentioned in issue #4122

      • Antoine R. Dumont added Documentation priority:High labels

        added Documentation priority:High labels

      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        Tentatively trying to reproduce the ci issue:

        $ cd $SWH_CI_ENVIRONMENT_HOME/swh-jenkins-dockerfile
# build the image
$ make swh-jenkins/sphinx
$ cd $SWH_ENVIRONMENT_HOME/swh-docs
$ docker run -ti -v $PWD:/src/swh-docs/ --entrypoint bash swh-jenkins/sphinx
-> $ git clone https://forge.softwareheritage.org/source/swh-environment.git
-> $ cd swh-environment
# copy out the local diffs (or git checkout from staging as you wish)
-> $ crudini --del .mrconfig snippets
-> $ crudini --del .mrconfig swh-py-template
-> $ mr -j 8 -t update
-> $ cd swh-docs
# install the locally mount the swh-docs "changes" to debug
-> $ rsync -avz --exclude .tox /src/swh-docs/ ./
-> $ git clean -dfx
-> $ SPHINXOPTS='-W -q --keep-going -w errors.log' SPHINXOPTCOLOR='--no-color' tox -e sphinx-dev
...

        To no avail ^ Not reproduced exactly the same issue. And it's taking forever like the ci build...

        Legend:

        • '->': In docker container
      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        Step to tentatively reproduce the issue:

        • create the docker image jenkins run locally
        • connect to it
        • setup the container to run as jenkins does

        as follows:

        -> $ SPHINXOPTS='-W -q --keep-going -w errors.log' SPHINXOPTCOLOR='--no-color' tox -e sphinx-dev
...
touch rec-build-stamp
sphinx-build -b html "." "_build/html" -W -q --keep-going -w errors.log --no-color
/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/scanner/dashboard/dashboard.py:11: UserWarning:
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/scanner/dashboard/dashboard.py:12: UserWarning:
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
WARNING: autodoc: failed to import module 'checks' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'collection' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'common' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'content' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'edit' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'edit_media' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_check' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_read' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_update_status' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'service_document' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'state' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'sword_edit' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit'; the following exception was raised:
No module named 'xmlschema'
make: *** [Makefile:72: sphinx/html] Error 1
make: Leaving directory '/home/jenkins/swh-environment/swh-docs/docs'
WARNING [pifpaf.util] `psutil.Popen(pid=3706, status='terminated', exitcode=0, started='15:30:26')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3689, status='terminated', exitcode=0, started='15:30:25')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3684, status='terminated', exitcode=0, started='15:30:25')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3717, status='terminated', exitcode=2, started='15:30:26')` is already gone, sending SIGKILL to its process group
ERROR: InvocationError for command '/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/bin/python -m pifpaf run postgresql -- make -C docs html' (exited with code 2)
___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR:   sphinx-dev: commands failed

# the last 2 commits correspond to the diffs i opened to "fix" that build
-> $ git log
commit 39d9a83813743ca265d65cd70105bc0bbb59d8ea (HEAD -> master)
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Mar 30 12:49:11 2022 +0200

    add-new-package: Update with necessary documentation instructions

    Without this, the actual documentation build will fail once a new package is
    incompletely installed.

commit f6cbf42ad1d961b35fa13d33ee07ef3c346dcf1b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Mar 29 09:33:46 2022 +0200

    Reference the new swh.scrubber module
->

        The error message in there ^ is not actually what i expected. I wanted the same error message as the build failures [1] [2].

        Whose gist of it is "solely" a problem in rendering the cli docstring [3] (swh.model.cli:identify).

        13:48:09  /var/lib/jenkins/workspace/DDOC/build-on-diff@2/swh-docs/docs/swh-model/cli.rst:14: WARNING: Literal block expected; none found

        As @vlorentz suggested, it might be an upstream bug in sphinx-click (used for rendering sphinx out of a click docstring). As I failed to reproduce for now, i'll put this on the side for now.

        FWIW, I've looked into their issue tracker and found no related issues [4]

      • vlorentz
        vlorentz @vlorentz ·
        Maintainer

        try copying this syntax for the code block:

        https://forge.softwareheritage.org/source/swh-storage/browse/master/swh/storage/cli.py$213-223

      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        try copying this syntax for the code block:

        https://forge.softwareheritage.org/source/swh-storage/browse/master/swh/storage/cli.py$213-223

        Thanks, i tentatively tried it [1] and triggered a jenkins build [2]. We'll see if that works.

      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        It seems to have fixed it! Great. Thanks.

        Now it's the (new and) same error as in docker ^, the new xmlschema introduced in swh.deposit[server] deps.

      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        !212 (closed) should fix it?

      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        !212 (closed) should fix it?

        nope... well, yeah... the warnings about xmlschema disappeared... but the initial problem supposedly fixed by [1] came back...

        • [1] rDMODba7af7447e5aa7fcf05b327fe1a35419446849a9
      • Antoine R. Dumont
        Antoine R. Dumont @ardumont ·
        Author Maintainer

        That last diff fixed it in the end... My docker tryout agreed [1] (contrary to jenkins diff for some reason). So i pushed and jenkins dev build got green again [2] (finally).

        I can finally close that one.

        • [1]
        ___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
  sphinx-dev: commands succeeded
  congratulations :)
      • Antoine R. Dumont assigned to @ardumont

        assigned to @ardumont

      • Antoine R. Dumont closed

        closed

      • Phabricator Migration user mentioned in merge request !209 (closed)

        mentioned in merge request !209 (closed)

      Please register or sign in to reply