Skip to content

vault.server: Introduce typed VaultInterface

Antoine R. Dumont requested to merge generated-differential-D4329-source into generated-differential-D4329-target

Technically, this allows to drop redundant code between rpc server and rpc client using the base class from swh.core.api.asynchronous.RPCServerApp (which now hides the detail of aiohttp there). This now matches how other rpc servers are declared, reducing friction on reading.

Ultimately, the goal is to make this maintenance a bit less heavy than it currently is. This also adds types so it aligns with other modules.

Related to T2223

Test Plan

tox

then with some docker love:

$ docker-compose up -d
$ cat docker-compose.override.yaml
...
services:
...
  swh-storage-db:
    ports:
      - "5434:5432"

  swh-vault-db:
    ports:
      - "5435:5432"

  swh-vault:
    volumes:
      - "$SWH_ENVIRONMENT_HOME/swh-vault:/src/swh-vault"  # <- contains this diff

  swh-vault-worker:
    volumes:
      - "$SWH_ENVIRONMENT_HOME/swh-vault:/src/swh-vault"
...
$ time doco exec swh-loader swh loader run git https://github.com/ardumont/home-manager
INFO:swh.loader.git.BulkLoader:Load origin 'https://github.com/ardumont/home-manager' with type 'git'
Enumerating objects: 8451, done.
Total 8451 (delta 0), reused 0 (delta 0), pack-reused 8451
INFO:swh.loader.git.BulkLoader:Listed 14 refs for repo https://github.com/ardumont/home-manager
{'status': 'eventful'}
docker-compose exec swh-loader swh loader run git   0.59s user 0.45s system 2% cpu 37.315 total
$ unset PAGER; psql service=swh-dev -c 'select id from directory limit 5'
                     id
--------------------------------------------
 \xa4a3a2a858eb84f2e77b44081c0a00cae7200c37
 \xade428bf6618bc66ac725bad3d14efd92d3f6e82
 \x8077bb0a78ddf494e7adb88ea87b8f9d9eeba15c
 \x336b371f1c7c8b9cdaa1a71b0922e18e9ef7e684
 \x838bc7ca7541fd96ed6e665c1485a0f8b6d4c3f3
$ doco exec swh-scheduler  \
    swh scheduler task add cook-vault-bundle \
    obj_type=directory obj_id=838bc7ca7541fd96ed6e665c1485a0f8b6d4c3f3

Created 1 tasks

Task 4
  Next run: just now (2020-10-22 12:50:04+00:00)
  Interval: 1 day, 0:00:00
  Type: cook-vault-bundle
  Policy: recurring
  Args:
  Keyword args:
    obj_id: '838bc7ca7541fd96ed6e665c1485a0f8b6d4c3f3'
    obj_type: 'directory'
$ doco logs -f swh-vault
...
swh-vault_1                     | ======== Running on http://0.0.0.0:5005 ========
swh-vault_1                     | (Press CTRL+C to quit)
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_status HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_status HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_progress HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_progress HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /put_bundle HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /put_bundle HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_status HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_status HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_progress HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /set_progress HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | INFO:aiohttp.access:172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /send_notif HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
swh-vault_1                     | [INFO] aiohttp.access -- 172.29.0.30 [22/Oct/2020:12:50:08 +0000] "POST /send_notif HTTP/1.1" 200 147 "-" "python-requests/2.24.0"
...

Migrated from D4329 (view on Phabricator)

Merge request reports