Deploy swh.lister v4.4

Contains:

• 5d0f35a v4.4.0 bitbucket: Skip buggy page when listing
• 62b0835 maven: Discard templated origin URLs

staging:

• Tag
• Build new image (swh.lister)
• Deploy
• Checks

production:

• Bypass debian unstable build (so buster builds) (it worked ;)
• Upgrade package python3-swh.lister
• Restart swh-worker@lister
• (ongoing) Trigger back lister bitbucket
• Checks

[1]

2023-03-13 12:07:30 softwareheritage-scheduler@belvedere:5432 λ select * from task where type='list-bitbucket-incremental';
+-[ RECORD 1 ]-----+-------------------------------+
| id               | 167882276                     |
| type             | list-bitbucket-incremental    |
| arguments        | {"args": [], "kwargs": {}}    |
| next_run         | 2022-04-23 19:15:51.826171+00 |
| current_interval | 1 day                         |
| status           | next_run_not_scheduled        |
| policy           | recurring                     |
| retries_left     | 1                             |
| priority         | (null)                        |
+------------------+-------------------------------+
Time: 4.258 ms

2023-03-13 12:07:32 softwareheritage-scheduler@belvedere:5432 λ select * from task where type='list-bitbucket-incremental';
+-[ RECORD 1 ]-----+-------------------------------+
| id               | 167882276                     |
| type             | list-bitbucket-incremental    |
| arguments        | {"args": [], "kwargs": {}}    |
| next_run         | 2022-04-23 19:15:51.826171+00 |
| current_interval | 1 day                         |
| status           | next_run_scheduled            |
| policy           | recurring                     |
| retries_left     | 1                             |
| priority         | (null)                        |
+------------------+-------------------------------+
Time: 4.265 ms

marked the checklist item Tag as completed

changed the description

marked the checklist item Bypass debian unstable build (so buster builds) (it worked ;) as completed

marked the checklist item Build new image (swh.lister) as completed

mentioned in commit ardumont/swh-charts@e389ce41

marked the checklist item Deploy as completed

marked the checklist item Checks as completed

marked the checklist item Upgrade package python3-swh.lister as completed

changed the description

marked the checklist item (ongoing) Restart swh-worker@lister as completed

changed the description

Running:

root@worker13:~# dpkg -l python3-swh.lister
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name               Version              Architecture Description
+++-==================-====================-============-=================================================================
ii  python3-swh.lister 4.4.0-1~swh1~bpo10+1 all          Software Heritage Listers (bitbucket, git(lab|hub), pypi, etc...)
Mar 13 11:07:34 worker13 python3[2178363]: [2023-03-13 11:07:34,308: INFO/MainProcess] Received task: swh.lister.bitbucket.tasks.IncrementalBitBucketLister[dd4ce559-2be1-4933-a2ea-1436d0fa6e85]
Mar 13 11:07:34 worker13 python3[2178371]: [2023-03-13 11:07:34,355: WARNING/ForkPoolWorker-4] No credentials set in configuration, using anonymous mode
Mar 13 11:07:35 worker13 python3[2178371]: [2023-03-13 11:07:35,135: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 500 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00: b'{"type": "error", "error": {"message": "Something went wrong", "id": "bdad46c206074799b243361b520dfe7a"}}'
Mar 13 11:07:35 worker13 python3[2178371]: [2023-03-13 11:07:35,136: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 1.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00.
Mar 13 11:07:36 worker13 python3[2178371]: [2023-03-13 11:07:36,940: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 500 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00: b'{"type": "error", "error": {"message": "Something went wrong", "id": "a9e4332ead24401e89de616ab09d4387"}}'
Mar 13 11:07:36 worker13 python3[2178371]: [2023-03-13 11:07:36,941: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 10.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00.
Mar 13 11:07:47 worker13 python3[2178371]: [2023-03-13 11:07:47,775: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 500 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00: b'{"type": "error", "error": {"message": "Something went wrong", "id": "c771a831459f47e18bdafa6b12c331e4"}}'
Mar 13 11:07:47 worker13 python3[2178371]: [2023-03-13 11:07:47,775: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 100.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00.
Mar 13 11:09:29 worker13 python3[2178371]: [2023-03-13 11:09:29,019: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 500 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00: b'{"type": "error", "error": {"message": "Something went wrong", "id": "0255edb3427c4f269928bbf166ce8292"}}'
Mar 13 11:09:29 worker13 python3[2178371]: [2023-03-13 11:09:29,020: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 1000.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2022-04-22T17%3A14%3A21.817530%2B00%3A00.

...
Mar 13 13:02:27 worker13 python3[2178371]: [2023-03-13 13:02:27,229: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 429 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2021-02-05T20%3A33%3A15.658311%2B00%3A00: b'Rate limit for this resource has been exceeded'
Mar 13 13:02:27 worker13 python3[2178371]: [2023-03-13 13:02:27,230: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 1.0 seconds as it raised HTTPError: 429 Client Error: Too Many Requests for url: https://api.
bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2021-02-05T20%3A33%3A15.658311%2B00%3A00.
Mar 13 13:02:29 worker13 python3[2178371]: [2023-03-13 13:02:29,840: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 429 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2C
values.updated_on%2Cvalues.created_on&after=2021-02-06T05%3A45%3A19.217785%2B00%3A00: b'Rate limit for this resource has been exceeded'
Mar 13 13:02:29 worker13 python3[2178371]: [2023-03-13 13:02:29,841: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 1.0 seconds as it raised HTTPError: 429 Client Error: Too Many Requests for url: https://api.
bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2021-02-06T05%3A45%3A19.217785%2B00%3A00.
Mar 13 13:02:30 worker13 python3[2178371]: [2023-03-13 13:02:30,976: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 429 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2C
values.updated_on%2Cvalues.created_on&after=2021-02-06T05%3A45%3A19.217785%2B00%3A00: b'Rate limit for this resource has been exceeded'
Mar 13 13:02:30 worker13 python3[2178371]: [2023-03-13 13:02:30,976: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 10.0 seconds as it raised HTTPError: 429 Client Error: Too Many Requests for url: https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2021-02-06T05%3A45%3A19.217785%2B00%3A00.
Mar 13 13:02:48 worker13 python3[2178371]: [2023-03-13 13:02:48,292: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 429 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2C
values.updated_on%2Cvalues.created_on&after=2021-02-07T16%3A48%3A09.728067%2B00%3A00: b'Rate limit for this resource has been exceeded'
Mar 13 13:02:48 worker13 python3[2178371]: [2023-03-13 13:02:48,293: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 1.0 seconds as it raised HTTPError: 429 Client Error: Too Many Requests for url: https://api.
bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2Cvalues.updated_on%2Cvalues.created_on&after=2021-02-07T16%3A48%3A09.728067%2B00%3A00.
Mar 13 13:02:49 worker13 python3[2178371]: [2023-03-13 13:02:49,419: WARNING/ForkPoolWorker-4] Unexpected HTTP status code 429 on https://api.bitbucket.org/2.0/repositories?pagelen=1000&fields=next%2Cvalues.links.clone.href%2Cvalues.scm%2C
values.updated_on%2Cvalues.created_on&after=2021-02-07T16%3A48%3A09.728067%2B00%3A00: b'Rate limit for this resource has been exceeded'
Mar 13 13:02:49 worker13 python3[2178371]: [2023-03-13 13:02:49,420: WARNING/ForkPoolWorker-4] Retrying swh.lister.pattern.Lister.http_request in 10.0 seconds as it raised HTTPError: 429 Client Error: Too Many Requests for url: https://api

marked the checklist item (ongoing) Trigger back lister bitbucket as completed

marked the checklist item Checks as completed

It seems to no longer stop, so it's all good to me. I'm closing this.

closed

After upgrading the python3-swh.lister for this issue (and the remaining other swh packages, including python3-swh.storage), most of the systemd services ("static" infra) discussing with storage were failing due to "breaking" change inside that package (import change). Including the save-code-now issues (swh-worker@loader_high_priority).

All impacted worker services (swh-worker@loader*, swh-worker@lister) needed a 'systemctl restart'.

Hence the sentry issue opened too [1]

[1] #4807 (closed)

added 1d of time spent