[Add Forge Now] process https://git.juef.net

https://webapp.staging.swh.network/admin/add-forge/request/63/
Type: stagit

changed milestone to %Extend archive coverage [Roadmap - Collect]

added AddForgeNow label

Request completly messed up in staging environment.
For the pipeline details, see:
https://gitlab.softwareheritage.org/swh/infra/add-forge-now-requests/-/pipelines/5674.

Request completly messed up in production environment.
For the pipeline details, see:
https://gitlab.softwareheritage.org/swh/infra/add-forge-now-requests/-/pipelines/5674.

The stagit lister pod creation took more time than the listing delay (300 seconds):

ᐅ kbs describe pod $(kbs get po -n swh | awk '/stagit/{print $1}') -n swh | tail -4
  Normal   Pulling           10m    kubelet            Pulling image "container-registry.softwareheritage.org/swh/infra/swh-apps/lister:20231127.1"
  Normal   Pulled            3m29s  kubelet            Successfully pulled image "container-registry.softwareheritage.org/swh/infra/swh-apps/lister:20231127.1" in 6m51.387271808s (6m51.387309979s including waiting)
  Normal   Created           3m3s   kubelet            Created container listers
  Normal   Started           3m3s   kubelet            Started container listers

Furthermore the stagit lister failed to get origins from this forge:

ᐅ kbs logs pods/$(kbs get po -n swh | awk '/stagit/{print $1}') -n swh  | tail -1 | jq
Defaulted container "listers" out of: listers, prepare-configuration (init)
{
  "asctime": "2023-11-28 08:44:50,189",
  "threadName": "MainThread",
  "pathname": "/opt/swh/.local/lib/python3.10/site-packages/celery/app/trace.py",
  "lineno": 270,
  "funcName": "_log_error",
  "task_name": null,
  "task_id": null,
  "name": "celery.app.trace",
  "levelname": "ERROR",
  "message": "Task swh.lister.stagit.tasks.StagitListerTask[790dfe52-cfd2-4438-a767-8be0e30b6b60] raised unexpected: AttributeError(\"'NoneType' object has no attribute 'find_all'\")",
  "exc_info": "Traceback (most recent call last):\n  File \"/opt/swh/.local/lib/python3.10/site-packages/celery/app/trace.py\", line 477, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/scheduler/task.py\", line 61, in __call__\n    result = super().__call__(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/celery/app/trace.py\", line 760, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/stagit/tasks.py\", line 16, in list_stagit\n    return lister.run().dict()\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/pattern.py\", line 219, in run\n    for page in self.get_pages():\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/stagit/lister.py\", line 76, in get_pages\n    for tr in bs_idx.find(\"table\", {\"id\": re.compile(\"index\")}).find_all(\"tr\"):\nAttributeError: 'NoneType' object has no attribute 'find_all'",
  "data": {
    "hostname": "lister@lister-stagit-9d6b7cf95-6vgrs",
    "id": "790dfe52-cfd2-4438-a767-8be0e30b6b60",
    "name": "swh.lister.stagit.tasks.StagitListerTask",
    "exc": "AttributeError(\"'NoneType' object has no attribute 'find_all'\")",
    "traceback": "Traceback (most recent call last):\n  File \"/opt/swh/.local/lib/python3.10/site-packages/celery/app/trace.py\", line 477, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/scheduler/task.py\", line 61, in __call__\n    result = super().__call__(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/celery/app/trace.py\", line 760, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/stagit/tasks.py\", line 16, in list_stagit\n    return lister.run().dict()\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/pattern.py\", line 219, in run\n    for page in self.get_pages():\n  File \"/opt/swh/.local/lib/python3.10/site-packages/swh/lister/stagit/lister.py\", line 76, in get_pages\n    for tr in bs_idx.find(\"table\", {\"id\": re.compile(\"index\")}).find_all(\"tr\"):\nAttributeError: 'NoneType' object has no attribute 'find_all'\n",
    "args": "()",
    "kwargs": "{'instance': 'git.juef.net', 'max_pages': 3, 'enable_origins': False, 'max_origins_per_page': 10}",
    "description": "raised unexpected",
    "internal": false
  }
}

I try to process this forge manually.

closed