Deploy loader.svn 1.7

includes:

• Performance improvment on the export_temporary method

staging:

• Build docker images
  • swh-loader-svn [1]
  • swh-loader-savecodenow [2]
• swh-charts: Bump images' version
• Deploy
• Checks

production:

• Build debian package (thx @anlambert which did the necessary changes so it builds correctly)
• Upgrade package on worker nodes
• swh-charts: merge staging branch in production and push [3]
• (ongoing) Restart swh-worker@loader_svn
• Checks

common:

• Tag in grafana

[1] https://jenkins.softwareheritage.org/job/swh-apps/job/build-docker-image/107/console

[2] https://jenkins.softwareheritage.org/job/swh-apps/job/build-docker-image/108/console

[3] That's taking care of deploying the loader save-code-now (production)

changed milestone to %MRO 2023

added activity::Deployment label

changed the description

mentioned in merge request swh/infra/ci-cd/swh-jenkins-jobs!172 (merged)

mentioned in commit swh-apps@f5a453ad

changed the description

marked the checklist item swh-loader-savecodenow [2] as completed

marked the checklist item Build docker images as completed

mentioned in commit swh/infra/ci-cd/swh-charts@8171adb7

marked the checklist item swh-charts: Bump images' version as completed

mentioned in merge request swh/infra/ci-cd/swh-charts!60 (merged)

marked the checklist item Deploy as completed

marked the checklist item Checks as completed

mentioned in commit swh/infra/ci-cd/swh-charts@847c7215

changed the description

mentioned in commit swh/infra/ci-cd/swh-charts@9bd9a938

marked the checklist item (ongoing) Restart swh-worker@loader_svn as completed

marked the checklist item Checks as completed

All is good.

Loaders restarted [1] and are ingesting again [2].

[1]

root@pergamon:~# clush -b -w @swh-workers systemctl status swh-worker@loader_svn | grep -i "active"
     Active: active (running) since Wed 2023-05-31 09:31:15 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:15 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:15 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago
     Active: active (running) since Wed 2023-05-31 09:31:16 UTC; 7min ago

[2]

May 31 09:31:17 worker16 python3[766374]: [2023-05-31 09:31:17,384: INFO/ForkPoolWorker-1] Load origin 'https://svn.code.sf.net/p/legato/code/legato-wps/tags/2.0.0/wps-parent' with type 'svn'
May 31 09:37:16 worker16 python3[766374]: [2023-05-31 09:37:16,699: INFO/ForkPoolWorker-1] Processing revisions [1202-1459] for {'swh-origin': 'https://svn.code.sf.net/p/legato/code/legato-wps/tags/2.0.0/wps-parent', 'remote_url': 'file:///tmp/swh.loader.svn.fno7svh7-766374/swh.loader.svn.qf0r6gxs-766374/tmpahhbmx0r', 'local_url': b'/tmp/swh.loader.svn.lpdipb2c-766374/tmpahhbmx0r', 'uuid': b'ee9ff2f3-9ad2-4b87-9088-cc2b5668db61'}
May 31 09:37:17 worker16 python3[766374]: [2023-05-31 09:37:17,700: INFO/ForkPoolWorker-1] Task swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[91c39448-6971-4509-83e3-9ee12f7fbdb2] succeeded in 360.3885820899159s: {'status': 'uneventful'}
May 31 09:37:17 worker16 python3[766362]: [2023-05-31 09:37:17,895: INFO/MainProcess] Received task: swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[013fb05c-1d93-4d6a-a5a4-1dcde51d8a49]
May 31 09:37:17 worker16 python3[766640]: [2023-05-31 09:37:17,997: INFO/ForkPoolWorker-2] Load origin 'svn://andy_jefferson@svn.code.sf.net/p/datanucleus/code/platform/accessplatform-jdo-rdbms/tags/datanucleus-accessplatform-jdo-rdbms-3.2.6' with type 'svn'
May 31 09:37:39 worker16 svnrdump[766644]: DIGEST-MD5 common mech free
May 31 09:37:47 worker16 python3[766640]: [2023-05-31 09:37:47,558: INFO/ForkPoolWorker-2] Processing revisions [18200-18919] for {'swh-origin': 'svn://svn.code.sf.net/p/datanucleus/code/platform/accessplatform-jdo-rdbms/tags/datanucleus-accessplatform-jdo-rdbms-3.2.6', 'remote_url': 'file:///tmp/swh.loader.svn.n87dmhcf-766640/swh.loader.svn.uiwlz7vc-766640/tmp4hfvcioh', 'local_url': b'/tmp/swh.loader.svn.rtrf0dd1-766640/tmp4hfvcioh', 'uuid': b'63e128b0-7949-0410-9533-d5af397be969'}
May 31 09:37:49 worker16 python3[766640]: [2023-05-31 09:37:49,507: INFO/ForkPoolWorker-2] Task swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[0ef14930-9477-4e91-98a6-d17135546eb6] succeeded in 31.612987933680415s: {'status': 'uneventful'}
May 31 09:37:49 worker16 python3[766362]: [2023-05-31 09:37:49,679: INFO/MainProcess] Received task: swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[b29fb727-d278-447f-9d3e-6d97aa4549e1]

changed the description

closed

assigned to @ardumont