docs/README.rst
Forked from
Platform / Development / swh-export
156 commits behind the upstream repository.

vlorentz
authored
This caused `JournalClientOffsetRanges` to ignore the last batch of messages in each assignment, because `JournalClient.handle_messages` deserializes all messages in the batch before calling the worker function; and raising `EOFError` from `deserialize_message` makes it exit early (before calling the worker fn). Additionally, it doesn't make much sense for a `deserialize_message` function to raise this kind of exception. Instead, this commit removes the explicit `raise EOFError`, and tells `JournalClient` to stop on EOF. `deserialize_message` calls `handle_offset`, which updates the assignment of the Kafka consumer to be the empty set, which causes it to be EOF (since there are no more partitions to read from).