Skip to content

docker/nginx: Align objstorage proxy_read_timeout value with production

Antoine Lambert requested to merge generated-differential-D8791-source into generated-differential-D8791-target

As in production settings for the objstorage reverse proxy, set HTTP read timeout to 1h in order to avoid 504 error (Gateway Time-out) when posting a large amount of contents to the objects storage.

I got hit by that issue when attempting to load the huge subversion repository for TexLive (svn://tug.org/texlive).

docker-swh-loader-1  | [2022-10-28 01:58:32,371: DEBUG/ForkPoolWorker-1] rev: 90, swhrev: 041c10306f3efb9ad37e77eea5cca2e9772ec5ef, dir: b3e98625596f907e969400f1f4c133a2a397e68c
docker-swh-loader-1  | [2022-10-28 01:58:32,401: DEBUG/ForkPoolWorker-1] Flushing 17021 objects of type content (223127418 bytes)
docker-swh-loader-1  | [2022-10-28 01:59:33,719: ERROR/ForkPoolWorker-1] Loading failure, updating to `failed` status
docker-swh-loader-1  | Traceback (most recent call last):
docker-swh-loader-1  |   File "/tmp/tmp.E2SMlP3pbi/swh-loader-core/swh/loader/core/loader.py", line 414, in load
docker-swh-loader-1  |     self.store_data()
docker-swh-loader-1  |   File "/tmp/tmp.E2SMlP3pbi/swh-loader-svn/swh/loader/svn/loader.py", line 471, in store_data
docker-swh-loader-1  |     self.storage.content_add(self._contents)
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 159, in content_add
docker-swh-loader-1  |     keys=["sha1", "sha1_git", "sha256", "blake2s256"],
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 230, in object_add
docker-swh-loader-1  |     return self.flush()
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 292, in flush
docker-swh-loader-1  |     stats = add_fn(list(batch))
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/filter.py", line 58, in content_add
docker-swh-loader-1  |     [x for x in content if x.sha256 in contents_to_add]
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/client.py", line 45, in content_add
docker-swh-loader-1  |     return self._post("content/add", {"content": content})
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 283, in _post
docker-swh-loader-1  |     return self._decode_response(response)
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 365, in _decode_response
docker-swh-loader-1  |     self.raise_for_status(response)
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/client.py", line 29, in raise_for_status
docker-swh-loader-1  |     super().raise_for_status(response)
docker-swh-loader-1  |   File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 355, in raise_for_status
docker-swh-loader-1  |     raise exception from None
docker-swh-loader-1  | swh.core.api.RemoteException: <html>
docker-swh-loader-1  | <head><title>504 Gateway Time-out</title></head>
docker-swh-loader-1  | <body>
docker-swh-loader-1  | <center><h1>504 Gateway Time-out</h1></center>
docker-swh-loader-1  | <hr><center>nginx/1.23.2</center>
docker-swh-loader-1  | </body>
docker-swh-loader-1  | </html>

Migrated from D8791 (view on Phabricator)

Merge request reports