Deploy swh.loader.core v2.1

It includes a fix to again unstuck the nixguix loader. Previous fixes (T3763) only unstuck it once and then fails on subsequent ones.

• Tag (@vlorentz did)

• release package ok

• Deploy on workers

• Set tag on grafana about the deployment

• Restart swh-worker@loader_nixguix service [1]

• Trigger the loader nixguix again [2]

• Restart other swh-workers@loader_$package for package in {deposit, debian, ...} [3]

• Is the nixguix unstuck? -> No.

• Report issue

• D6816: Review fix

• loop over the first instructions for v2.1.1 (when tagged)

• Are nixguix loading visits ok?

• [1]

root@pergamon:~# clush -b -w @staging-loader-workers -w @swh-workers "systemctl restart swh-worker@loader_nixguix"

• [2]

09:16:33 softwareheritage-scheduler@belvedere:5432=> update task set next_run=now(), status='next_run_not_scheduled' where type='load-nixguix' and status!='disabled';
UPDATE 2
Time: 261.256 ms
09:17:12 softwareheritage-scheduler@belvedere:5432=> select * from task where type='load-nixguix' and status!='disabled';
+-[ RECORD 1 ]-----+------------------------------------------------------------------------------------------------------+
| id               | 334411727                                                                                            |
| type             | load-nixguix                                                                                         |
| arguments        | {"args": [], "kwargs": {"url": "https://nix-community.github.io/nixpkgs-swh/sources-unstable.json"}} |
| next_run         | 2021-12-10 08:17:12.764649+00                                                                        |
| current_interval | 1 day                                                                                                |
| status           | next_run_scheduled                                                                                   |
| policy           | recurring                                                                                            |
| retries_left     | 3                                                                                                    |
| priority         | (null)                                                                                               |
+-[ RECORD 2 ]-----+------------------------------------------------------------------------------------------------------+
| id               | 337282717                                                                                            |
| type             | load-nixguix                                                                                         |
| arguments        | {"args": [], "kwargs": {"url": "https://guix.gnu.org/sources.json"}}                                 |
| next_run         | 2021-12-10 08:17:12.764649+00                                                                        |
| current_interval | 1 day                                                                                                |
| status           | next_run_scheduled                                                                                   |
| policy           | recurring                                                                                            |
| retries_left     | 3                                                                                                    |
| priority         | (null)                                                                                               |
+------------------+------------------------------------------------------------------------------------------------------+

Time: 15.690 ms

• [3]

root@pergamon:~# clush -b -w @swh-workers -w @staging-loader-workers 'for type in debian archive cran opam pypi npm deposit; do systemctl restart swh-worker@loader_${type}; done'

---

Migrated from T3791 (view on Phabricator)

added Nixguix loader Package Loader System administration priority:High labels

added state:wip label

changed the description

marked the checklist item Deploy on workers as completed

marked the checklist item Restart swh-worker@loader_nixguix service as completed

Now the loader nixguix fails with [1]

• [1] https://sentry.softwareheritage.org/share/issue/1ff035857ecf490192aec7e76096c40a/

marked the checklist item Restart other swh-workers@loader_$package for package in {deposit, debian, ...} as completed

changed the description

changed the description

marked the checklist item loop over the first instructions for v2.1.1 (when tagged) as completed

changed the description

Nixguix loader is ok.

root@pergamon:~# date; clush -b -w @swh-workers systemctl status swh-worker@loader_nixguix | grep succeeded
Sun 12 Dec 2021 04:14:37 PM UTC
Dec 11 19:10:47 worker13 python3[2780]: [2021-12-11 19:10:47,371: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[2541de72-6471-4766-960c-aba768aa6274] succeeded in 2493.3735876400024s: {'status': 'eventful', 'snapshot_id': 'be11862d047bcd7b6f8ebd8f417ffa6e545b7bc9'}
Dec 12 00:48:04 worker14 python3[2810]: [2021-12-12 00:48:04,783: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[c835fdd9-29e4-4584-a890-e38d1d7e845e] succeeded in 1039.6907857869955s: {'status': 'eventful', 'snapshot_id': '2893d092115f3d9b535d64c583f95b94e82f87f1'}
Dec 11 00:28:07 worker15 python3[2801]: [2021-12-11 00:28:07,621: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[eaa2e617-7855-40f7-a80b-456356f89e6c] succeeded in 24255.011169397003s: {'status': 'eventful', 'snapshot_id': '6f79b0c18dda0459b154003bbe899d50f183a8f8'}
Dec 10 18:27:12 worker16 python3[2797]: [2021-12-10 18:27:12,959: INFO/ForkPoolWorker-1] Task swh.loader.package.nixguix.tasks.LoadNixguix[faf7a510-8548-443b-8d80-d27798a65a46] succeeded in 2600.335359801s: {'status': 'eventful', 'snapshot_id': '389ee393260c375915652389ce9ff4fd5712b9d7'}

changed the description

closed