scheduler: Add scheduler-schedule-recurrent service to schedule origins regularly
Loading
Contains chart services:
helm template and minikube are happy about it:
$ chart=swh; helm template $chart $chart/ --values values-swh-application-versions.yaml --values $chart/values.yaml --values $chart/values/default.yaml --values $chart/values/minikube.yaml --debug
...
# Source: swh/templates/scheduler/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: swh
name: scheduler-configuration-template
data:
config.yml.template: |
scheduler:
cls: remote
url: http://fake-scheduler.i.s.s.n:5008
celery:
task_broker: amqp://guest:guest@fake-scheduler.i.s.s.n:5672/%2f
scheduling_policy:
default:
- policy: never_visited_oldest_update_first
weight: 100
opam:
- policy: origins_without_last_update
weight: 100
...
...[1] Starting (and crashing as expected as there ain't any scheduler db behind).
Running swh command scheduler schedule-recurrent
Traceback (most recent call last):
File "/opt/swh/.local/bin/swh", line 8, in <module>
sys.exit(main())
File "/opt/swh/.local/lib/python3.10/site-packages/swh/core/cli/__init__.py", line 174, in main
return swh(auto_envvar_prefix="SWH")
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/swh/.local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/opt/swh/.local/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/opt/swh/.local/lib/python3.10/site-packages/swh/scheduler/cli/admin.py", line 168, in schedule_recurrent
all_task_types = scheduler.get_task_types()
AttributeError: 'NoneType' object has no attribute 'get_task_types'
Stream closed EOF for swh/scheduler-schedule-recurrent-b95d646cd-2cqj4 (scheduler-schedule-recurrent)[2] rpc happy (not doing anything in minikube though as i don't have the rest configured):
│ Starting the swh-scheduler API server │
│ [2023-09-06 13:46:41 +0000] [1] [INFO] Starting gunicorn 21.2.0 │
│ [2023-09-06 13:46:41 +0000] [1] [INFO] Listening at: http://0.0.0.0:5008 (1) │
│ [2023-09-06 13:46:41 +0000] [1] [INFO] Using worker: sync │
│ [2023-09-06 13:46:41 +0000] [7] [INFO] Booting worker with pid: 7