Skip to content

bitbucket lister does not work

It has been discussed on irc to deploy it if it works or open a task as fallback (as per my proposition ;). From current swh-lister (tag ~0.0.27 or so), the current bitbucket lister fails to execute properly.

Expected behavior

  • listing properly
  • no error in logs
  • new cache entries in lister's bitbucket_repo table
  • new scheduling tasks (load-hg, load-git) in scheduler db

What really happens

After task scheduling for that lister, the task fails as show below.

Details

Scheduling the task within the docker-env:

SCHEDULER_API_URL=http://localhost:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-bitbucket-full --policy recurring api_baseurl='https://api.bitbucket.org/2.0'
Created 1 tasks

Task 2276
  Next run: just now (2019-06-18 12:46:34+00:00)
  Interval: 90 days, 0:00:00
  Type: list-bitbucket-full
  Policy: recurring
  Args:
  Keyword args:
    api_baseurl: 'https://api.bitbucket.org/2.0'

Letting it run:

swh-lister_1                  | [2019-06-18 12:46:36,547: ERROR/ForkPoolWorker-1] Task swh.lister.bitbucket.tasks.FullBitBucketRelister[32a85cb7-eb3b-41b8-b663-dce60e4fbaba] raised unexpected: ValueError("Can't partition an empty range")
swh-lister_1                  | Traceback (most recent call last):
swh-lister_1                  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/celery/app/trace.py", line 385, in trace_task
swh-lister_1                  |     R = retval = fun(*args, **kwargs)
swh-lister_1                  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/scheduler/task.py", line 45, in __call__
swh-lister_1                  |     return super().__call__(*args, **kwargs)
swh-lister_1                  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/celery/app/trace.py", line 648, in __protected_call__
swh-lister_1                  |     return self.run(*args, **kwargs)
swh-lister_1                  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/lister/bitbucket/tasks.py", line 34, in full_bitbucket_relister
swh-lister_1                  |     ranges = lister.db_partition_indices(split or GROUP_SPLIT)
swh-lister_1                  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/lister/core/indexing_lister.py", line 116, in db_partition_indices
swh-lister_1                  |     raise ValueError("Can't partition an empty range")
swh-lister_1                  | ValueError: Can't partition an empty range

Don't know the reason (could be an api change?), the task's goal is to analyze and fix.

Migrated from T1826 (view on Phabricator)