Skip to content
Snippets Groups Projects

Handle BufferErrors in writer.kafka.Producer

When the local rdkafka producer queue is full, the producer would crash with a BufferError. While this condition is rare, it can happen when running a backfill operation.

When such a condition arises, retry sending the message a few times with an increasing backoff. When giving up, use the common KafkaDeliveryError mechanism rather than the plain BufferError.

Related to swh-scheduler#3000 (closed)

Test Plan

new tests added for retry and DeliveryError


Migrated from D5144 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading