Skip to content

scrubber does not comply to what's expected by the swh db tooling

After crafting of a "proper" config.yml [1] because otherwise it complains [2].

It still complains about missing implementations [3] [4].

This currently prevents migration with the proper tooling as per documentation [5].

Current version of the used scrubber is the latest v0.0.3 [6]

  • [1] config.yml:
swhworker@scrubber0:~$ cat config.yml
---
scrubber:
  cls: postgresql
  db: host=<host> port=<port> dbname=swh-scrubber user=swh-scrubber password=<password>
  • [2]
swhworker@scrubber0:~$ swh db --config-file /etc/softwareheritage/scrubber/postgres.yml upgrade scrubber --to-version=2
Usage: swh db upgrade [OPTIONS] MODULE

Error: Invalid value: Configuration cls must be set to 'postgresql' for this command.
  • [3]
swhworker@scrubber0:~$ swh db --config-file config.yml version scrubber
module: scrubber
Traceback (most recent call last):
  File "/usr/bin/swh", line 33, in <module>
    sys.exit(load_entry_point('swh.core==2.8.0', 'console_scripts', 'swh')())
  File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 184, in main
    return swh(auto_envvar_prefix="SWH")
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/core/cli/db.py", line 302, in db_version
    code_version = datastore.get_current_version()
AttributeError: 'ScrubberDb' object has no attribute 'get_current_version'
  • [4]
swhworker@scrubber0:~$ swh db --config-file config.yml upgrade scrubber --to-version=2
Traceback (most recent call last):
  File "/usr/bin/swh", line 33, in <module>
    sys.exit(load_entry_point('swh.core==2.8.0', 'console_scripts', 'swh')())
  File "/usr/lib/python3/dist-packages/swh/core/cli/__init__.py", line 184, in main
    return swh(auto_envvar_prefix="SWH")
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3/dist-packages/swh/core/cli/db.py", line 390, in db_upgrade
    ds_version = datastore.get_current_version()
AttributeError: 'ScrubberDb' object has no attribute 'get_current_version'
swhworker@scrubber0:~$ dpkg -l python3-swh.scrubber | grep ii
ii  python3-swh.scrubber 0.0.3-1~swh1~bpo10+1 all          Software Heritage Datastore Scrubber
ii  python3-swh.core 2.8.0-1~swh1~bpo10+1 all          Software Heritage core utilities

Migrated from T4284 (view on Phabricator)

Edited by Phabricator Migration user
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information