Skip to content

check icinga alert for svn save-code-now

There is something fishy going on.

Worker logs report ingestion ok [1].

Still the save code data model [2] (the icinga alert is in sync with those at least ;).

We need to investigate and fix. The communication with the scheduler might be in cause

  • [3] This might be another case of T3458.

  • [1]

Sep 16 05:21:38 worker0 python3[495]: [2021-09-16 05:21:38,904: INFO/MainProcess] Received task: swh.loader.git.tasks.UpdateGitRepository[d70a3434-ec27-48f6-a946-6831b868393f]
Sep 16 05:21:40 worker0 python3[900]: [2021-09-16 05:21:40,404: INFO/ForkPoolWorker-1] Load origin 'https://github.com/rdicosmo/parmap' with type 'git'
Sep 16 05:21:40 worker0 python3[900]: [2021-09-16 05:21:40,782: INFO/ForkPoolWorker-1] Listed 108 refs for repo https://github.com/rdicosmo/parmap
Sep 16 05:21:40 worker0 python3[900]: [2021-09-16 05:21:40,903: INFO/ForkPoolWorker-1] Task swh.loader.git.tasks.UpdateGitRepository[d70a3434-ec27-48f6-a946-6831b868393f] succeeded in 1.995606476906687s: {'status': 'uneventful'}
Sep 16 18:57:40 worker0 python3[495]: [2021-09-16 18:57:40,502: INFO/MainProcess] Received task: swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[2e2f3066-c505-445b-9684-8c45126a8c90]
Sep 16 18:57:41 worker0 python3[900]: [2021-09-16 18:57:41,385: INFO/ForkPoolWorker-1] Load origin 'https://subversion.renater.fr/anonscm/svn/panda' with type 'svn'
Sep 16 18:57:46 worker0 python3[900]: [2021-09-16 18:57:46,935: WARNING/ForkPoolWorker-1] file:///tmp/tmp10ai0_k_/swh.loader.svn.rrk8eqeo-900/tmp92h37np6@5 already injected.
Sep 16 18:57:48 worker0 python3[900]: [2021-09-16 18:57:48,814: INFO/ForkPoolWorker-1] Task swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[2e2f3066-c505-445b-9684-8c45126a8c90] succeeded in 8.307126371655613s: {'status': 'uneventful'}
  • [2]
09:26:31 swh-web@belvedere:5432=> select * from save_origin_request where origin_url = 'https://subversion.renater.fr/anonscm/svn/panda' order by request_date desc limit 6;
+-------+-------------------------------+------------+-------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
|  id   |         request_date          | visit_type |                   origin_url                    |  status  | loading_task_id |          visit_date           | loading_task_status | visit_status | user_ids |
+-------+-------------------------------+------------+-------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
| 89952 | 2021-08-25 02:50:06.604702+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398244749 | (null)                        | failed              | (null)       | (null)   |
| 89951 | 2021-08-25 02:49:26.638799+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398244749 | (null)                        | failed              | (null)       | (null)   |
| 89950 | 2021-08-25 02:48:46.117809+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398244749 | (null)                        | failed              | (null)       | (null)   |
| 89949 | 2021-08-25 02:48:06.564125+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398244749 | (null)                        | failed              | (null)       | (null)   |
| 89948 | 2021-08-25 02:40:56.739116+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398244749 | (null)                        | failed              | (null)       | (null)   |
| 89748 | 2021-08-24 02:40:56.642982+00 | svn        | https://subversion.renater.fr/anonscm/svn/panda | accepted |       398187435 | 2021-08-24 02:41:07.153813+00 | succeeded           | full         | (null)   |
+-------+-------------------------------+------------+-------------------------------------------------+----------+-----------------+-------------------------------+---------------------+--------------+----------+
  • [3] The task is stuck...
09:27:56 softwareheritage-scheduler@belvedere:5432=> select * from task where id =398244749;
+-----------+----------+------------------------------------------------------------------------------------+-------------------------------+------------------+--------------------+---------+--------------+----------+
|    id     |   type   |                                     arguments                                      |           next_run            | current_interval |       status       | policy  | retries_left | priority |
+-----------+----------+------------------------------------------------------------------------------------+-------------------------------+------------------+--------------------+---------+--------------+----------+
| 398244749 | load-svn | {"args": [], "kwargs": {"url": "https://subversion.renater.fr/anonscm/svn/panda"}} | 2021-08-25 06:02:46.140666+00 | 1 day            | next_run_scheduled | oneshot |            0 | high     |
+-----------+----------+------------------------------------------------------------------------------------+-------------------------------+------------------+--------------------+---------+--------------+----------+
(1 row)

Time: 36.006 ms

Migrated from T3583 (view on Phabricator)

Edited by Phabricator Migration user