Journal error in production

Each time I submit a save request for the cpython repository in production, the following error is reported:

[2019-12-10 14:24:24,047: ERROR/ForkPoolWorker-12505] Loading failure, updating to `partial` status
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 872, in load
    self.store_data()
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 979, in store_data
    self.send_snapshot(self.get_snapshot())
  File "/usr/lib/python3/dist-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/usr/lib/python3/dist-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/usr/lib/python3/dist-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 490, in send_snapshot
    self.storage.snapshot_add([snapshot])
  File "/usr/lib/python3/dist-packages/swh/storage/api/client.py", line 97, in snapshot_add
    return self.post('snapshot/add', {'snapshots': snapshots})
  File "/usr/lib/python3/dist-packages/swh/core/api/__init__.py", line 206, in post
    return self._decode_response(response)
  File "/usr/lib/python3/dist-packages/swh/core/api/__init__.py", line 238, in _decode_response
    raise pickle.loads(decode_response(response))
_pickle.PicklingError: Can't pickle : import of module 'cimpl' failed

This seems related to the journal.

The real error seems not correctly reported due to that pickling issue in confluent-kafka: https://github.com/confluentinc/confluent-kafka-python/issues/641

---

Migrated from T2143 (view on Phabricator)