Skip to content
Snippets Groups Projects

Add support for the rdkafka 'stats_cb' config option in JournalClient

this options allows to define a callback which will be called once every 'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a json string).

See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md

This allows to define this callback as a string of the form:

"path.to.module:function"

Depends on !212 (closed)

Test Plan

WIP because needs tests


Migrated from D6884 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Build is green

    Patch application report for D6884 (id=24961)

    Could not rebase; Attempt merge onto 4e5e0098...

    Updating 4e5e009..d877377
    Fast-forward
     swh/journal/client.py | 54 +++++++++++++++++++++++++++++++++------------------
     1 file changed, 35 insertions(+), 19 deletions(-)
    Changes applied before test
    commit d87737790bc067a30510e87c20b402d5d310f2dc
    Author: David Douard <david.douard@sdfa3.org>
    Date:   Thu Jan 6 12:07:50 2022 +0100
    
        Add support for the rdkafka 'stats_cb' config option in JournalClient
        
        this options allows to define a callback which will be called once every
        'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a
        json string).
        
        See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md
        
        This allows to define this callback as a string of the form:
        
          "path.to.module:function"
    
    commit 35bb07d74b2bc19353280770f825648c5d053bc4
    Author: David Douard <david.douard@sdfa3.org>
    Date:   Tue Jan 4 17:25:02 2022 +0100
    
        Remove 'process_timeout' from JournalClient's arguments
        
        it was not used anywhere, so simplify the code a bit.

    See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/189/ for more details.

  • Author Maintainer

    move the code in get_journal_client

  • Build is green

    Patch application report for D6884 (id=24966)

    Could not rebase; Attempt merge onto 4e5e0098...

    Updating 4e5e009..5b17c50
    Fast-forward
     swh/journal/client.py | 53 +++++++++++++++++++++++++++++++++------------------
     1 file changed, 34 insertions(+), 19 deletions(-)
    Changes applied before test
    commit 5b17c50d32809c1cb4f3505846d23572c5650496
    Author: David Douard <david.douard@sdfa3.org>
    Date:   Thu Jan 6 12:07:50 2022 +0100
    
        Add support for the rdkafka 'stats_cb' config option in get_journal_client
        
        this options allows to define a callback which will be called once every
        'statistics.interval.ms' ms by rdkafka with a bunch of statistics (as a
        json string).
        
        See https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md
        
        This allows to define this callback as a string of the form:
        
          "path.to.module:function"
    
    commit 0d115993e0f1bd5e049ba07a1185ae29fcb6e654
    Author: David Douard <david.douard@sdfa3.org>
    Date:   Tue Jan 4 17:25:02 2022 +0100
    
        Remove 'process_timeout' from JournalClient's arguments
        
        it was not used anywhere, so simplify the code a bit.

    See https://jenkins.softwareheritage.org/job/DJNL/job/tests-on-diff/191/ for more details.

  • Nicolas Dandrimont mentioned in merge request !214 (closed)

    mentioned in merge request !214 (closed)

  • I'm kinda wondering if this import stuff should move to a common module - I think we do kind of the same thing with entrypoints?

  • Merge request was accepted

  • Nicolas Dandrimont approved this merge request

    approved this merge request

  • Author Maintainer

    ! In !213 (closed), @olasd wrote: I'm kinda wondering if this import stuff should move to a common module - I think we do kind of the same thing with entrypoints?

    yeah we might think of it eventually.

  • Author Maintainer

    Merge request was merged

  • closed

Please register or sign in to reply
Loading