journal publisher may crash on HTTP timeouts
On my docker env, I often see the journal publisher die because of:
swh-journal-publisher_1 | Traceback (most recent call last):
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
swh-journal-publisher_1 | "__main__", mod_spec)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
swh-journal-publisher_1 | exec(code, run_globals)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 239, in <module>
swh-journal-publisher_1 | main()
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
swh-journal-publisher_1 | return self.main(*args, **kwargs)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
swh-journal-publisher_1 | rv = self.invoke(ctx)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
swh-journal-publisher_1 | return ctx.invoke(self.callback, **ctx.params)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
swh-journal-publisher_1 | return callback(*args, **kwargs)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 237, in main
swh-journal-publisher_1 | publisher.poll()
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 121, in poll
swh-journal-publisher_1 | new_objects = self.process_objects(messages)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 151, in process_objects
swh-journal-publisher_1 | for key, value in messages.items()
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 151, in <dictcomp>
swh-journal-publisher_1 | for key, value in messages.items()
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/journal/publisher.py", line 176, in process_contents
swh-journal-publisher_1 | (c[b'sha1'] for c in content_objs))
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/storage/api/client.py", line 40, in content_get_metadata
swh-journal-publisher_1 | return self.post('content/metadata', {'content': content})
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/core/api/__init__.py", line 185, in post
swh-journal-publisher_1 | return self._decode_response(response)
swh-journal-publisher_1 | File "/usr/local/lib/python3.6/site-packages/swh/core/api/__init__.py", line 220, in _decode_response
swh-journal-publisher_1 | raise pickle.loads(decode_response(response))
swh-journal-publisher_1 | psycopg2.extensions.QueryCanceledError: canceling statement due to statement timeout
Migrated from T1541 (view on Phabricator)