Add support for the rdkafka 'stats_cb' config option in JournalClient
this options allows to define a callback which will be called once every 'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a json string).
See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md
This allows to define this callback as a string of the form:
"path.to.module:function"
Depends on !212 (closed)
Test Plan
WIP because needs tests
Migrated from D6884 (view on Phabricator)
Merge request reports
Activity
Build is green
Patch application report for D6884 (id=24961)
Could not rebase; Attempt merge onto 4e5e0098...
Updating 4e5e009..d877377 Fast-forward swh/journal/client.py | 54 +++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 19 deletions(-)
Changes applied before test
commit d87737790bc067a30510e87c20b402d5d310f2dc Author: David Douard <david.douard@sdfa3.org> Date: Thu Jan 6 12:07:50 2022 +0100 Add support for the rdkafka 'stats_cb' config option in JournalClient this options allows to define a callback which will be called once every 'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a json string). See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md This allows to define this callback as a string of the form: "path.to.module:function" commit 35bb07d74b2bc19353280770f825648c5d053bc4 Author: David Douard <david.douard@sdfa3.org> Date: Tue Jan 4 17:25:02 2022 +0100 Remove 'process_timeout' from JournalClient's arguments it was not used anywhere, so simplify the code a bit.
See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/189/ for more details.
Build is green
Patch application report for D6884 (id=24966)
Could not rebase; Attempt merge onto 4e5e0098...
Updating 4e5e009..5b17c50 Fast-forward swh/journal/client.py | 53 +++++++++++++++++++++++++++++++++------------------ 1 file changed, 34 insertions(+), 19 deletions(-)
Changes applied before test
commit 5b17c50d32809c1cb4f3505846d23572c5650496 Author: David Douard <david.douard@sdfa3.org> Date: Thu Jan 6 12:07:50 2022 +0100 Add support for the rdkafka 'stats_cb' config option in get_journal_client this options allows to define a callback which will be called once every 'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a json string). See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md This allows to define this callback as a string of the form: "path.to.module:function" commit 0d115993e0f1bd5e049ba07a1185ae29fcb6e654 Author: David Douard <david.douard@sdfa3.org> Date: Tue Jan 4 17:25:02 2022 +0100 Remove 'process_timeout' from JournalClient's arguments it was not used anywhere, so simplify the code a bit.
See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/191/ for more details.
mentioned in merge request !214 (closed)
! In !213 (closed), @olasd wrote: I'm kinda wondering if this import stuff should move to a common module - I think we do kind of the same thing with entrypoints?
yeah we might think of it eventually.