Skip to content

Work around issue on frozen set generation

Antoine R. Dumont requested to merge ardumont/swh-apps:workaround-wrong-pkg-resources-generated-working into master

This is the sole solution i've managed to make build successfully images.

That falsifies the frozen set definition... But I'd argue for it as a temporary solution which unstucks the build docker image pipeline development...

I've tried other implementations to no avail (pushed in another branch in my fork)

  • docker (during the image build that runs the script) [1]
  • venv (with app_manager.py) [2]
frozen set generation and build commands 
$ docker build -t app-manager scripts/
Sending build context to Docker daemon  19.97kB
Step 1/7 : FROM python:3.10-bullseye
 ---> 109421b4b8cd
Step 2/7 : RUN apt-get -y update &&     apt-get -y upgrade &&     apt-get install -y       libcmph-dev       librdkafka-dev       libsvn-dev libapr1-dev libaprutil1-dev       libpython3-dev       libpq-dev       python3-click       python3-venv       python3-yaml       python3-dulwich     &&     apt clean
 ---> Using cache
 ---> b3fb8e0051cf
Step 3/7 : RUN mkdir -p /opt/scripts/
 ---> Using cache
 ---> aa645b2ebd30
Step 4/7 : COPY app_manager.py /opt/scripts/
 ---> f7e9348f4437
Step 5/7 : RUN chmod +x /opt/scripts/app_manager.py
 ---> Running in 590749456dab
Removing intermediate container 590749456dab
 ---> 99f3875e7fe0
Step 6/7 : ENV SWH_APPS_DIR=/src/apps
 ---> Running in ce8ba9d4b9a0
Removing intermediate container ce8ba9d4b9a0
 ---> b0344df6431c
Step 7/7 : ENTRYPOINT [ "/opt/scripts/app_manager.py" ]
 ---> Running in e7267e1b1233
Removing intermediate container e7267e1b1233
 ---> 8d0c5050739f
Successfully built 8d0c5050739f
Successfully tagged app-manager:latest

$ git status



master...upstream/master


M scripts/app_manager.py


$ docker run -v $PWD:/src --user 1000 -it app-manager generate-frozen-requirements swh-loader-git
Requirement already satisfied: pip in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (20.3.4)
Collecting pip
Downloading pip-22.3.1-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 16.2 MB/s
Requirement already satisfied: setuptools in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (44.1.1)
Collecting setuptools
Downloading setuptools-66.1.1-py3-none-any.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 110.7 MB/s
Collecting wheel
Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
Installing collected packages: wheel, setuptools, pip
Attempting uninstall: setuptools
Found existing installation: setuptools 44.1.1
Uninstalling setuptools-44.1.1:
Successfully uninstalled setuptools-44.1.1
Attempting uninstall: pip
Found existing installation: pip 20.3.4
Uninstalling pip-20.3.4:
Successfully uninstalled pip-20.3.4
Successfully installed pip-22.3.1 setuptools-66.1.1 wheel-0.38.4
Collecting swh-loader-git
Downloading swh.loader.git-2.1.0-py3-none-any.whl (73 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.8/73.8 kB 20.5 MB/s eta 0:00:00
Collecting swh-loader-metadata
Downloading swh.loader.metadata-0.0.3-py3-none-any.whl (34 kB)
Collecting swh.scheduler>=0.0.39
Downloading swh.scheduler-1.5.0-py3-none-any.whl (166 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 kB 89.6 MB/s eta 0:00:00
Collecting retrying
Downloading retrying-1.3.4-py3-none-any.whl (11 kB)
Collecting dulwich>=0.20.43
Downloading dulwich-0.21.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (505 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 505.7/505.7 kB 183.1 MB/s eta 0:00:00
Collecting swh.storage>=0.22.0
Downloading swh.storage-1.7.3-py3-none-any.whl (449 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 449.7/449.7 kB 368.5 MB/s eta 0:00:00
Collecting click
Downloading click-8.1.3-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 358.4 MB/s eta 0:00:00
Collecting swh.model>=4.3.0
Downloading swh.model-6.6.1-py3-none-any.whl (136 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.2/136.2 kB 257.2 MB/s eta 0:00:00
Collecting swh.loader.core>=5.0.0
Downloading swh.loader.core-5.1.0-py3-none-any.whl (10.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 109.9 MB/s eta 0:00:00
Collecting swh.core>=0.0.7
Downloading swh.core-2.20.0-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 130.0 MB/s eta 0:00:00
Collecting swh.lister>=2.9.0
Downloading swh.lister-4.3.3-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 141.9 MB/s eta 0:00:00
Collecting urllib3>=1.25
Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 253.4 MB/s eta 0:00:00
Collecting pyyaml
Downloading PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 661.8/661.8 kB 182.6 MB/s eta 0:00:00
Collecting python-magic
Downloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB)
Collecting deprecated
Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting sentry-sdk
Downloading sentry_sdk-1.14.0-py2.py3-none-any.whl (178 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 390.3 MB/s eta 0:00:00
Collecting aiohttp
Downloading aiohttp-3.8.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 166.5 MB/s eta 0:00:00
Collecting requests
Downloading requests-2.28.2-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 283.0 MB/s eta 0:00:00
Collecting flask
Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.5/101.5 kB 365.0 MB/s eta 0:00:00
Collecting blinker
Downloading blinker-1.5-py2.py3-none-any.whl (12 kB)
Collecting msgpack>=1.0.0
Downloading msgpack-1.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 322.4/322.4 kB 367.2 MB/s eta 0:00:00
Collecting iso8601
Downloading iso8601-1.1.0-py3-none-any.whl (9.9 kB)
Collecting aiohttp-utils>=3.1.1
Downloading aiohttp_utils-3.1.1-py2.py3-none-any.whl (10 kB)
Collecting launchpadlib
Downloading launchpadlib-1.11.0-py2.py3-none-any.whl (217 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 217.7/217.7 kB 256.6 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /tmp/swh-loader-gitu15uqnzc/lib/python3.9/site-packages (from swh.lister>=2.9.0->swh-loader-metadata->-r /src/apps/swh-loader-git/requirements.txt (line 2)) (66.1.1)
Collecting beautifulsoup4
Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.2/128.2 kB 218.1 MB/s eta 0:00:00
Collecting lxml
Downloading lxml-4.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 113.4 MB/s eta 0:00:00
Collecting testing.postgresql
Downloading testing.postgresql-1.3.0-py2.py3-none-any.whl (8.9 kB)
Collecting repomd
Downloading repomd-0.2.1-py3-none-any.whl (3.9 kB)
Collecting tenacity>=6.2
Downloading tenacity-8.1.0-py3-none-any.whl (23 kB)
Collecting python-debian
Downloading python_debian-0.1.49-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 212.4 MB/s eta 0:00:00
Collecting psycopg2
Downloading psycopg2-2.9.5.tar.gz (384 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.3/384.3 kB 229.2 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting python-dateutil
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 388.7 MB/s eta 0:00:00
Collecting typing-extensions
Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting toml
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting pkginfo
Downloading pkginfo-1.9.6-py3-none-any.whl (30 kB)
Collecting swh.objstorage>=0.2.2
Downloading swh.objstorage-2.0.1-py3-none-any.whl (93 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.7/93.7 kB 375.8 MB/s eta 0:00:00
Collecting psutil
Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 391.3 MB/s eta 0:00:00
Collecting packaging
Downloading packaging-23.0-py3-none-any.whl (42 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 kB 266.1 MB/s eta 0:00:00
Collecting hypothesis
Downloading hypothesis-6.65.0-py3-none-any.whl (402 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 402.2/402.2 kB 304.2 MB/s eta 0:00:00
Collecting attrs-strict>=0.0.7
Downloading attrs_strict-1.0.0-py3-none-any.whl (14 kB)
Collecting attrs!=21.1.0
Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 kB 223.5 MB/s eta 0:00:00
Collecting celery!=5.0.3,>=4.3
Downloading celery-5.2.7-py3-none-any.whl (405 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 405.6/405.6 kB 270.5 MB/s eta 0:00:00
Collecting humanize
Downloading humanize-4.4.0-py3-none-any.whl (106 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 281.0 MB/s eta 0:00:00
Collecting importlib-metadata<5.0
Downloading importlib_metadata-4.13.0-py3-none-any.whl (23 kB)
Collecting pika>=1.1.0
Downloading pika-1.3.1-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 338.0 MB/s eta 0:00:00
Collecting cassandra-driver!=3.21.0,>=3.19.0
Downloading cassandra_driver-3.25.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 119.0 MB/s eta 0:00:00
Collecting redis
Downloading redis-4.4.2-py3-none-any.whl (237 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.8/237.8 kB 295.8 MB/s eta 0:00:00
Collecting swh.counters>=v0.8.0
Downloading swh.counters-0.9.1-py3-none-any.whl (37 kB)
Collecting mypy-extensions
Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting six>=1.7.0
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting gunicorn
Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 375.1 MB/s eta 0:00:00
Collecting python-mimeparse
Downloading python_mimeparse-1.6.0-py2.py3-none-any.whl (6.1 kB)
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting yarl<2.0,>=1.0
Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.6/264.6 kB 368.1 MB/s eta 0:00:00
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB 370.5 MB/s eta 0:00:00
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.8/158.8 kB 368.4 MB/s eta 0:00:00
Collecting charset-normalizer<3.0,>=2.0
Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting geomet<0.3,>=0.1
Downloading geomet-0.2.1.post1-py3-none-any.whl (18 kB)
Collecting billiard<4.0,>=3.6.4.0
Downloading billiard-3.6.4.0-py3-none-any.whl (89 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB 362.2 MB/s eta 0:00:00
Collecting vine<6.0,>=5.0.0
Downloading vine-5.0.0-py2.py3-none-any.whl (9.4 kB)
Collecting click-didyoumean>=0.0.3
Downloading click_didyoumean-0.3.0-py3-none-any.whl (2.7 kB)
Collecting click-plugins>=1.1.1
Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
Collecting kombu<6.0,>=5.2.3
Downloading kombu-5.2.4-py3-none-any.whl (189 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 189.2/189.2 kB 392.7 MB/s eta 0:00:00
Collecting click-repl>=0.2.0
Downloading click_repl-0.2.0-py3-none-any.whl (5.2 kB)
Collecting pytz>=2021.3
Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 499.4/499.4 kB 301.9 MB/s eta 0:00:00
Collecting zipp>=0.5
Downloading zipp-3.11.0-py3-none-any.whl (6.6 kB)
Collecting pytest-postgresql<4.0.0,>=3
Downloading pytest_postgresql-3.1.3-py3-none-any.whl (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 kB 247.6 MB/s eta 0:00:00
Collecting swh.journal
Downloading swh.journal-1.2.1-py3-none-any.whl (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 235.1 MB/s eta 0:00:00
Collecting swh.perfecthash>=0.1.2
Downloading swh.perfecthash-0.1.2.tar.gz (27 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting soupsieve>1.2
Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting wrapt<2,>=1.10
Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.8/77.8 kB 261.6 MB/s eta 0:00:00
Collecting Werkzeug>=2.2.2
Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 403.2 MB/s eta 0:00:00
Collecting itsdangerous>=2.0
Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting Jinja2>=3.0
Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 252.1 MB/s eta 0:00:00
Collecting sortedcontainers<3.0.0,>=2.1.0
Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting exceptiongroup>=1.0.0
Downloading exceptiongroup-1.1.0-py3-none-any.whl (14 kB)
Collecting httplib2
Downloading httplib2-0.21.0-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 358.6 MB/s eta 0:00:00
Collecting lazr.uri
Downloading lazr.uri-1.0.6.tar.gz (18 kB)
Preparing metadata (setup.py) ... done
Collecting lazr.restfulclient>=0.14.2
Downloading lazr.restfulclient-0.14.5-py3-none-any.whl (70 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 324.6 MB/s eta 0:00:00
Collecting chardet
Downloading chardet-5.1.0-py3-none-any.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.1/199.1 kB 261.3 MB/s eta 0:00:00
Collecting defusedxml
Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 249.9 MB/s eta 0:00:00
Collecting idna<4,>=2.5
Downloading idna-3.4-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 320.1 MB/s eta 0:00:00
Collecting pg8000>=1.10
Downloading pg8000-1.29.4-py3-none-any.whl (51 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 250.0 MB/s eta 0:00:00
Collecting testing.common.database
Downloading testing.common.database-2.0.3-py2.py3-none-any.whl (10 kB)
Collecting prompt-toolkit
Downloading prompt_toolkit-3.0.36-py3-none-any.whl (386 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.4/386.4 kB 286.8 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0
Downloading MarkupSafe-2.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting amqp<6.0.0,>=5.0.9
Downloading amqp-5.1.1-py3-none-any.whl (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.8/50.8 kB 234.7 MB/s eta 0:00:00
Collecting wadllib>=1.1.4
Downloading wadllib-1.3.6.tar.gz (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.2/62.2 kB 262.7 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting oauthlib
Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 327.6 MB/s eta 0:00:00
Collecting distro
Downloading distro-1.8.0-py3-none-any.whl (20 kB)
Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 258.7 MB/s eta 0:00:00
Collecting scramp>=1.4.3
Downloading scramp-1.4.4-py3-none-any.whl (13 kB)
Collecting mirakuru>=2.3.0
Downloading mirakuru-2.4.2-py3-none-any.whl (29 kB)
Collecting port-for
Downloading port_for-0.6.3-py3-none-any.whl (21 kB)
Collecting pytest>=6.2.0
Downloading pytest-7.2.1-py3-none-any.whl (317 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 kB 274.7 MB/s eta 0:00:00
Collecting cffi
Downloading cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.2/441.2 kB 228.3 MB/s eta 0:00:00
Collecting confluent-kafka
Downloading confluent_kafka-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 116.4 MB/s eta 0:00:00
Collecting iniconfig
Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting pluggy<2.0,>=0.12
Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting tomli>=1.0.0
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting asn1crypto>=1.5.1
Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.0/105.0 kB 337.2 MB/s eta 0:00:00
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 266.7 MB/s eta 0:00:00
Collecting wcwidth
Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Building wheels for collected packages: psycopg2, swh.perfecthash, lazr.uri, wadllib
Building wheel for psycopg2 (setup.py) ... done
Created wheel for psycopg2: filename=psycopg2-2.9.5-cp39-cp39-linux_x86_64.whl size=456363 sha256=bd081f655ff0cd28a74633466d2f81ec9d7eb87b3a57d3ce648ca5e534151fa4
Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/7e/ad/8b/26873413259428280c0894640ff3c475221f3b10282796e9d6
Building wheel for swh.perfecthash (pyproject.toml) ... done
Created wheel for swh.perfecthash: filename=swh.perfecthash-0.1.2-cp39-cp39-linux_x86_64.whl size=44342 sha256=ba7966aed650d51975d106c30a3c05d9f2abe6102913b3b24c68c62a9825b669
Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/07/49/c1/7c53ed0ea6208320a2d9ff6faf2fc1922304fccba86bdcc4e7
Building wheel for lazr.uri (setup.py) ... done
Created wheel for lazr.uri: filename=lazr.uri-1.0.6-py3-none-any.whl size=20359 sha256=4dc47e7cf8c6297b71869dff150830b8952c4636b1334f66b53c76643579405c
Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/65/f1/1c/708e52900a3dfdc83b80cd112643516468d4671e40e0c906ad
Building wheel for wadllib (setup.py) ... done
Created wheel for wadllib: filename=wadllib-1.3.6-py3-none-any.whl size=58279 sha256=83c806fc6fce39f079bf99bd50f7c470b39eaafb4df5272928c87dbdb34fe114
Stored in directory: /tmp/pip-ephem-wheel-cache-_8op_6pk/wheels/15/8c/a2/fc4313c0655453812b92189cedd07b0bad20935f29306ae54d
Successfully built psycopg2 swh.perfecthash lazr.uri wadllib
Installing collected packages: wcwidth, testing.common.database, sortedcontainers, pytz, python-mimeparse, mypy-extensions, msgpack, confluent-kafka, billiard, asn1crypto, zipp, wrapt, vine, urllib3, typing-extensions, tomli, toml, tenacity, soupsieve, six, scramp, pyyaml, python-magic, pyparsing, pycparser, psycopg2, psutil, prompt-toolkit, port-for, pluggy, pkginfo, pika, packaging, oauthlib, multidict, MarkupSafe, lxml, lazr.uri, itsdangerous, iso8601, iniconfig, idna, humanize, gunicorn, frozenlist, exceptiongroup, distro, defusedxml, click, charset-normalizer, chardet, certifi, blinker, attrs, async-timeout, yarl, Werkzeug, wadllib, sentry-sdk, retrying, requests, repomd, redis, python-debian, python-dateutil, pytest, mirakuru, Jinja2, importlib-metadata, hypothesis, httplib2, geomet, dulwich, deprecated, click-repl, click-plugins, click-didyoumean, cffi, beautifulsoup4, attrs-strict, amqp, aiosignal, swh.perfecthash, swh.model, swh.core, pytest-postgresql, pg8000, lazr.restfulclient, kombu, flask, cassandra-driver, aiohttp, testing.postgresql, swh.journal, launchpadlib, celery, aiohttp-utils, swh.objstorage, swh.counters, swh.storage, swh.scheduler, swh.loader.core, swh.lister, swh-loader-metadata, swh-loader-git
Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.2 Werkzeug-2.2.2 aiohttp-3.8.3 aiohttp-utils-3.1.1 aiosignal-1.3.1 amqp-5.1.1 asn1crypto-1.5.1 async-timeout-4.0.2 attrs-22.2.0 attrs-strict-1.0.0 beautifulsoup4-4.11.1 billiard-3.6.4.0 blinker-1.5 cassandra-driver-3.25.0 celery-5.2.7 certifi-2022.12.7 cffi-1.15.1 chardet-5.1.0 charset-normalizer-2.1.1 click-8.1.3 click-didyoumean-0.3.0 click-plugins-1.1.1 click-repl-0.2.0 confluent-kafka-2.0.2 defusedxml-0.7.1 deprecated-1.2.13 distro-1.8.0 dulwich-0.21.2 exceptiongroup-1.1.0 flask-2.2.2 frozenlist-1.3.3 geomet-0.2.1.post1 gunicorn-20.1.0 httplib2-0.21.0 humanize-4.4.0 hypothesis-6.65.0 idna-3.4 importlib-metadata-4.13.0 iniconfig-2.0.0 iso8601-1.1.0 itsdangerous-2.1.2 kombu-5.2.4 launchpadlib-1.11.0 lazr.restfulclient-0.14.5 lazr.uri-1.0.6 lxml-4.9.2 mirakuru-2.4.2 msgpack-1.0.4 multidict-6.0.4 mypy-extensions-0.4.3 oauthlib-3.2.2 packaging-23.0 pg8000-1.29.4 pika-1.3.1 pkginfo-1.9.6 pluggy-1.0.0 port-for-0.6.3 prompt-toolkit-3.0.36 psutil-5.9.4 psycopg2-2.9.5 pycparser-2.21 pyparsing-3.0.9 pytest-7.2.1 pytest-postgresql-3.1.3 python-dateutil-2.8.2 python-debian-0.1.49 python-magic-0.4.27 python-mimeparse-1.6.0 pytz-2022.7.1 pyyaml-6.0 redis-4.4.2 repomd-0.2.1 requests-2.28.2 retrying-1.3.4 scramp-1.4.4 sentry-sdk-1.14.0 six-1.16.0 sortedcontainers-2.4.0 soupsieve-2.3.2.post1 swh-loader-git-2.1.0 swh-loader-metadata-0.0.3 swh.core-2.20.0 swh.counters-0.9.1 swh.journal-1.2.1 swh.lister-4.3.3 swh.loader.core-5.1.0 swh.model-6.6.1 swh.objstorage-2.0.1 swh.perfecthash-0.1.2 swh.scheduler-1.5.0 swh.storage-1.7.3 tenacity-8.1.0 testing.common.database-2.0.3 testing.postgresql-1.3.0 toml-0.10.2 tomli-2.0.1 typing-extensions-4.4.0 urllib3-1.26.14 vine-5.0.0 wadllib-1.3.6 wcwidth-0.2.6 wrapt-1.14.1 yarl-1.8.2 zipp-3.11.0


$ gdf | grep "pkg_resources"  # prior to this workaround, that'd display the dreaded pkg_resources==0.0.0 line
{+        """Work around issue with wheel generating wrong version for pkg_resources."""+}
{+        return requirements_content.replace(b'pkg_resources==0.0.0\n', b'\n')+}


$ docker build --tag container-registry.softwareheritage.org/swh/infra/swh-apps/loader_git:20230124.1 apps/swh-loader-git
Sending build context to Docker daemon  8.192kB
Step 1/15 : FROM python:3.10-bullseye
---> 109421b4b8cd
Step 2/15 : RUN apt-get -y update &&     apt-get -y upgrade &&     apt-get install -y libcmph-dev librdkafka-dev &&     apt clean &&     addgroup --gid 1000 swh &&     useradd --gid 1000 --uid 1000 -m -d /opt/swh swh &&     mkdir /etc/swh
---> Using cache
---> 5a6d2e1289bb
Step 3/15 : USER swh
---> Using cache
---> 4c1fca783f9d
Step 4/15 : WORKDIR /opt/swh
---> Using cache
---> bd167973351e
Step 5/15 : COPY --chown=swh:swh requirements-frozen.txt /opt/swh
---> 1348926b5a97
Step 6/15 : COPY --chown=swh:swh entrypoint.sh /opt/swh
---> 9e415498f881
Step 7/15 : ENV PYTHONPATH=/opt/swh
---> Running in b70a8792d09c
Removing intermediate container b70a8792d09c
---> 2f3ad33139c9
Step 8/15 : ENV PATH=/opt/swh/.local/bin:$PATH
---> Running in c51adb747254
Removing intermediate container c51adb747254
---> f80645dd4b71
Step 9/15 : RUN chmod u+x /opt/swh/entrypoint.sh &&     /usr/local/bin/python -m pip install --upgrade pip &&     pip install --no-cache-dir -r requirements-frozen.txt &&     pip install gunicorn
---> Running in 7c09ab0c3117
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (22.3.1)
Defaulting to user installation because normal site-packages is not writeable
Collecting aiohttp==3.8.3
Downloading aiohttp-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 56.1 MB/s eta 0:00:00
Collecting aiohttp-utils==3.1.1
Downloading aiohttp_utils-3.1.1-py2.py3-none-any.whl (10 kB)
Collecting aiosignal==1.3.1
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting amqp==5.1.1
Downloading amqp-5.1.1-py3-none-any.whl (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.8/50.8 kB 382.1 MB/s eta 0:00:00
Collecting asn1crypto==1.5.1
Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.0/105.0 kB 439.9 MB/s eta 0:00:00
Collecting async-timeout==4.0.2
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting attrs==22.2.0
Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 kB 257.1 MB/s eta 0:00:00
Collecting attrs-strict==1.0.0
Downloading attrs_strict-1.0.0-py3-none-any.whl (14 kB)
Collecting beautifulsoup4==4.11.1
Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.2/128.2 kB 302.2 MB/s eta 0:00:00
Collecting billiard==3.6.4.0
Downloading billiard-3.6.4.0-py3-none-any.whl (89 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB 445.1 MB/s eta 0:00:00
Collecting blinker==1.5
Downloading blinker-1.5-py2.py3-none-any.whl (12 kB)
Collecting cassandra-driver==3.25.0
Downloading cassandra_driver-3.25.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 113.8 MB/s eta 0:00:00
Collecting celery==5.2.7
Downloading celery-5.2.7-py3-none-any.whl (405 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 405.6/405.6 kB 202.8 MB/s eta 0:00:00
Collecting certifi==2022.12.7
Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 446.2 MB/s eta 0:00:00
Collecting cffi==1.15.1
Downloading cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.8/441.8 kB 224.6 MB/s eta 0:00:00
Collecting chardet==5.1.0
Downloading chardet-5.1.0-py3-none-any.whl (199 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.1/199.1 kB 453.5 MB/s eta 0:00:00
Collecting charset-normalizer==2.1.1
Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting click==8.1.3
Downloading click-8.1.3-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 433.4 MB/s eta 0:00:00
Collecting click-didyoumean==0.3.0
Downloading click_didyoumean-0.3.0-py3-none-any.whl (2.7 kB)
Collecting click-plugins==1.1.1
Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
Collecting click-repl==0.2.0
Downloading click_repl-0.2.0-py3-none-any.whl (5.2 kB)
Collecting confluent-kafka==2.0.2
Downloading confluent_kafka-2.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 62.5 MB/s eta 0:00:00
Collecting defusedxml==0.7.1
Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting Deprecated==1.2.13
Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting distro==1.8.0
Downloading distro-1.8.0-py3-none-any.whl (20 kB)
Collecting dulwich==0.21.2
Downloading dulwich-0.21.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (506 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 506.2/506.2 kB 154.4 MB/s eta 0:00:00
Collecting exceptiongroup==1.1.0
Downloading exceptiongroup-1.1.0-py3-none-any.whl (14 kB)
Collecting Flask==2.2.2
Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.5/101.5 kB 445.3 MB/s eta 0:00:00
Collecting frozenlist==1.3.3
Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.6/149.6 kB 331.1 MB/s eta 0:00:00
Collecting geomet==0.2.1.post1
Downloading geomet-0.2.1.post1-py3-none-any.whl (18 kB)
Collecting gunicorn==20.1.0
Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 439.6 MB/s eta 0:00:00
Collecting httplib2==0.21.0
Downloading httplib2-0.21.0-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 467.7 MB/s eta 0:00:00
Collecting humanize==4.4.0
Downloading humanize-4.4.0-py3-none-any.whl (106 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 468.4 MB/s eta 0:00:00
Collecting hypothesis==6.65.0
Downloading hypothesis-6.65.0-py3-none-any.whl (402 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 402.2/402.2 kB 220.2 MB/s eta 0:00:00
Collecting idna==3.4
Downloading idna-3.4-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 392.3 MB/s eta 0:00:00
Collecting importlib-metadata==4.13.0
Downloading importlib_metadata-4.13.0-py3-none-any.whl (23 kB)
Collecting iniconfig==2.0.0
Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting iso8601==1.1.0
Downloading iso8601-1.1.0-py3-none-any.whl (9.9 kB)
Collecting itsdangerous==2.1.2
Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting Jinja2==3.1.2
Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 401.7 MB/s eta 0:00:00
Collecting kombu==5.2.4
Downloading kombu-5.2.4-py3-none-any.whl (189 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 189.2/189.2 kB 273.5 MB/s eta 0:00:00
Collecting launchpadlib==1.11.0
Downloading launchpadlib-1.11.0-py2.py3-none-any.whl (217 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 217.7/217.7 kB 292.6 MB/s eta 0:00:00
Collecting lazr.restfulclient==0.14.5
Downloading lazr.restfulclient-0.14.5-py3-none-any.whl (70 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 292.4 MB/s eta 0:00:00
Collecting lazr.uri==1.0.6
Downloading lazr.uri-1.0.6.tar.gz (18 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting lxml==4.9.2
Downloading lxml-4.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 87.3 MB/s eta 0:00:00
Collecting MarkupSafe==2.1.2
Downloading MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting mirakuru==2.4.2
Downloading mirakuru-2.4.2-py3-none-any.whl (29 kB)
Collecting msgpack==1.0.4
Downloading msgpack-1.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (316 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.0/317.0 kB 275.4 MB/s eta 0:00:00
Collecting multidict==6.0.4
Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.5/114.5 kB 472.5 MB/s eta 0:00:00
Collecting mypy-extensions==0.4.3
Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting oauthlib==3.2.2
Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 460.3 MB/s eta 0:00:00
Collecting packaging==23.0
Downloading packaging-23.0-py3-none-any.whl (42 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.7/42.7 kB 277.0 MB/s eta 0:00:00
Collecting pg8000==1.29.4
Downloading pg8000-1.29.4-py3-none-any.whl (51 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 358.5 MB/s eta 0:00:00
Collecting pika==1.3.1
Downloading pika-1.3.1-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 460.7 MB/s eta 0:00:00
Collecting pkginfo==1.9.6
Downloading pkginfo-1.9.6-py3-none-any.whl (30 kB)
Collecting pluggy==1.0.0
Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting port-for==0.6.3
Downloading port_for-0.6.3-py3-none-any.whl (21 kB)
Collecting prompt-toolkit==3.0.36
Downloading prompt_toolkit-3.0.36-py3-none-any.whl (386 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.4/386.4 kB 224.1 MB/s eta 0:00:00
Collecting psutil==5.9.4
Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 375.0 MB/s eta 0:00:00
Collecting psycopg2==2.9.5
Downloading psycopg2-2.9.5.tar.gz (384 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.3/384.3 kB 176.9 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting pycparser==2.21
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 391.8 MB/s eta 0:00:00
Collecting pyparsing==3.0.9
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 466.1 MB/s eta 0:00:00
Collecting pytest==7.2.1
Downloading pytest-7.2.1-py3-none-any.whl (317 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 kB 289.6 MB/s eta 0:00:00
Collecting pytest-postgresql==3.1.3
Downloading pytest_postgresql-3.1.3-py3-none-any.whl (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 kB 327.9 MB/s eta 0:00:00
Collecting python-dateutil==2.8.2
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 259.8 MB/s eta 0:00:00
Collecting python-debian==0.1.49
Downloading python_debian-0.1.49-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 435.2 MB/s eta 0:00:00
Collecting python-magic==0.4.27
Downloading python_magic-0.4.27-py2.py3-none-any.whl (13 kB)
Collecting python-mimeparse==1.6.0
Downloading python_mimeparse-1.6.0-py2.py3-none-any.whl (6.1 kB)
Collecting pytz==2022.7.1
Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 499.4/499.4 kB 193.2 MB/s eta 0:00:00
Collecting PyYAML==6.0
Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 130.4 MB/s eta 0:00:00
Collecting redis==4.4.2
Downloading redis-4.4.2-py3-none-any.whl (237 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.8/237.8 kB 232.8 MB/s eta 0:00:00
Collecting repomd==0.2.1
Downloading repomd-0.2.1-py3-none-any.whl (3.9 kB)
Collecting requests==2.28.2
Downloading requests-2.28.2-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 414.2 MB/s eta 0:00:00
Collecting retrying==1.3.4
Downloading retrying-1.3.4-py3-none-any.whl (11 kB)
Collecting scramp==1.4.4
Downloading scramp-1.4.4-py3-none-any.whl (13 kB)
Collecting sentry-sdk==1.14.0
Downloading sentry_sdk-1.14.0-py2.py3-none-any.whl (178 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 482.8 MB/s eta 0:00:00
Collecting six==1.16.0
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting sortedcontainers==2.4.0
Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting soupsieve==2.3.2.post1
Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting swh.core==2.20.0
Downloading swh.core-2.20.0-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 119.2 MB/s eta 0:00:00
Collecting swh.counters==0.9.1
Downloading swh.counters-0.9.1-py3-none-any.whl (37 kB)
Collecting swh.journal==1.2.1
Downloading swh.journal-1.2.1-py3-none-any.whl (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 362.3 MB/s eta 0:00:00
Collecting swh.lister==4.3.3
Downloading swh.lister-4.3.3-py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 154.4 MB/s eta 0:00:00
Collecting swh.loader.core==5.1.0
Downloading swh.loader.core-5.1.0-py3-none-any.whl (10.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.0/10.0 MB 110.0 MB/s eta 0:00:00
Collecting swh.loader.git==2.1.0
Downloading swh.loader.git-2.1.0-py3-none-any.whl (73 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.8/73.8 kB 422.7 MB/s eta 0:00:00
Collecting swh.loader.metadata==0.0.3
Downloading swh.loader.metadata-0.0.3-py3-none-any.whl (34 kB)
Collecting swh.model==6.6.1
Downloading swh.model-6.6.1-py3-none-any.whl (136 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 136.2/136.2 kB 456.0 MB/s eta 0:00:00
Collecting swh.objstorage==2.0.1
Downloading swh.objstorage-2.0.1-py3-none-any.whl (93 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.7/93.7 kB 447.2 MB/s eta 0:00:00
Collecting swh.perfecthash==0.1.2
Downloading swh.perfecthash-0.1.2.tar.gz (27 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting swh.scheduler==1.5.0
Downloading swh.scheduler-1.5.0-py3-none-any.whl (166 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 kB 460.9 MB/s eta 0:00:00
Collecting swh.storage==1.7.3
Downloading swh.storage-1.7.3-py3-none-any.whl (449 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 449.7/449.7 kB 178.0 MB/s eta 0:00:00
Collecting tenacity==8.1.0
Downloading tenacity-8.1.0-py3-none-any.whl (23 kB)
Collecting testing.common.database==2.0.3
Downloading testing.common.database-2.0.3-py2.py3-none-any.whl (10 kB)
Collecting testing.postgresql==1.3.0
Downloading testing.postgresql-1.3.0-py2.py3-none-any.whl (8.9 kB)
Collecting toml==0.10.2
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting tomli==2.0.1
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting typing_extensions==4.4.0
Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting urllib3==1.26.14
Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 461.8 MB/s eta 0:00:00
Collecting vine==5.0.0
Downloading vine-5.0.0-py2.py3-none-any.whl (9.4 kB)
Collecting wadllib==1.3.6
Downloading wadllib-1.3.6.tar.gz (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.2/62.2 kB 357.8 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting wcwidth==0.2.6
Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Collecting Werkzeug==2.2.2
Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 286.9 MB/s eta 0:00:00
Collecting wrapt==1.14.1
Downloading wrapt-1.14.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 412.5 MB/s eta 0:00:00
Collecting yarl==1.8.2
Downloading yarl-1.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 264.0/264.0 kB 248.9 MB/s eta 0:00:00
Collecting zipp==3.11.0
Downloading zipp-3.11.0-py3-none-any.whl (6.6 kB)
Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.10/site-packages (from gunicorn==20.1.0->-r requirements-frozen.txt (line 31)) (65.5.1)
Building wheels for collected packages: lazr.uri, psycopg2, swh.perfecthash, wadllib
Building wheel for lazr.uri (setup.py): started
Building wheel for lazr.uri (setup.py): finished with status 'done'
Created wheel for lazr.uri: filename=lazr.uri-1.0.6-py3-none-any.whl size=20362 sha256=95d28b423a964c31846e2703d34644cc65c08c48f7288a7b396764e2362d4678
Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/a3/09/68/5456b2eb028e206bba5f703f3d1a34e88f51e76f6249e24d94
Building wheel for psycopg2 (setup.py): started
Building wheel for psycopg2 (setup.py): finished with status 'done'
Created wheel for psycopg2: filename=psycopg2-2.9.5-cp310-cp310-linux_x86_64.whl size=479238 sha256=28ffeef3ac10b3f352559f642eac1978aff82dec22319433988a70912ac48843
Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/07/2f/1d/00dd98e4de351e52b04ad341a254fe6bebf7c6edb9523ad9a1
Building wheel for swh.perfecthash (pyproject.toml): started
Building wheel for swh.perfecthash (pyproject.toml): finished with status 'done'
Created wheel for swh.perfecthash: filename=swh.perfecthash-0.1.2-cp310-cp310-linux_x86_64.whl size=47880 sha256=464c7e8844afa32e993401475c71ed7223524b69749f775bb037b3c90ba4bc8a
Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/21/bb/54/e5b93f41dce533d4b7cd25effc1171b986b5dab0f2403f6a7e
Building wheel for wadllib (setup.py): started
Building wheel for wadllib (setup.py): finished with status 'done'
Created wheel for wadllib: filename=wadllib-1.3.6-py3-none-any.whl size=58279 sha256=e58938b592dc63f1aabd67db8c9c9271e79798d9310d39063efec712c7d054d3
Stored in directory: /tmp/pip-ephem-wheel-cache-2q9b809_/wheels/48/01/fd/2f6448c6149fca1c0606c084d83e98d44a1830316af92bfef4
Successfully built lazr.uri psycopg2 swh.perfecthash wadllib
Installing collected packages: wcwidth, testing.common.database, sortedcontainers, pytz, python-mimeparse, mypy-extensions, msgpack, confluent-kafka, billiard, asn1crypto, zipp, wrapt, vine, urllib3, typing_extensions, tomli, toml, tenacity, soupsieve, six, scramp, PyYAML, python-magic, pyparsing, pycparser, psycopg2, psutil, prompt-toolkit, port-for, pluggy, pkginfo, pika, packaging, oauthlib, multidict, MarkupSafe, lxml, lazr.uri, itsdangerous, iso8601, iniconfig, idna, humanize, gunicorn, frozenlist, exceptiongroup, distro, defusedxml, click, charset-normalizer, chardet, certifi, blinker, attrs, async-timeout, yarl, Werkzeug, wadllib, sentry-sdk, retrying, requests, repomd, redis, python-debian, python-dateutil, pytest, mirakuru, Jinja2, importlib-metadata, hypothesis, httplib2, geomet, dulwich, Deprecated, click-repl, click-plugins, click-didyoumean, cffi, beautifulsoup4, attrs-strict, amqp, aiosignal, swh.perfecthash, swh.model, swh.core, pytest-postgresql, pg8000, lazr.restfulclient, kombu, Flask, cassandra-driver, aiohttp, testing.postgresql, swh.journal, launchpadlib, celery, aiohttp-utils, swh.objstorage, swh.counters, swh.storage, swh.scheduler, swh.loader.core, swh.lister, swh.loader.metadata, swh.loader.git
Successfully installed Deprecated-1.2.13 Flask-2.2.2 Jinja2-3.1.2 MarkupSafe-2.1.2 PyYAML-6.0 Werkzeug-2.2.2 aiohttp-3.8.3 aiohttp-utils-3.1.1 aiosignal-1.3.1 amqp-5.1.1 asn1crypto-1.5.1 async-timeout-4.0.2 attrs-22.2.0 attrs-strict-1.0.0 beautifulsoup4-4.11.1 billiard-3.6.4.0 blinker-1.5 cassandra-driver-3.25.0 celery-5.2.7 certifi-2022.12.7 cffi-1.15.1 chardet-5.1.0 charset-normalizer-2.1.1 click-8.1.3 click-didyoumean-0.3.0 click-plugins-1.1.1 click-repl-0.2.0 confluent-kafka-2.0.2 defusedxml-0.7.1 distro-1.8.0 dulwich-0.21.2 exceptiongroup-1.1.0 frozenlist-1.3.3 geomet-0.2.1.post1 gunicorn-20.1.0 httplib2-0.21.0 humanize-4.4.0 hypothesis-6.65.0 idna-3.4 importlib-metadata-4.13.0 iniconfig-2.0.0 iso8601-1.1.0 itsdangerous-2.1.2 kombu-5.2.4 launchpadlib-1.11.0 lazr.restfulclient-0.14.5 lazr.uri-1.0.6 lxml-4.9.2 mirakuru-2.4.2 msgpack-1.0.4 multidict-6.0.4 mypy-extensions-0.4.3 oauthlib-3.2.2 packaging-23.0 pg8000-1.29.4 pika-1.3.1 pkginfo-1.9.6 pluggy-1.0.0 port-for-0.6.3 prompt-toolkit-3.0.36 psutil-5.9.4 psycopg2-2.9.5 pycparser-2.21 pyparsing-3.0.9 pytest-7.2.1 pytest-postgresql-3.1.3 python-dateutil-2.8.2 python-debian-0.1.49 python-magic-0.4.27 python-mimeparse-1.6.0 pytz-2022.7.1 redis-4.4.2 repomd-0.2.1 requests-2.28.2 retrying-1.3.4 scramp-1.4.4 sentry-sdk-1.14.0 six-1.16.0 sortedcontainers-2.4.0 soupsieve-2.3.2.post1 swh.core-2.20.0 swh.counters-0.9.1 swh.journal-1.2.1 swh.lister-4.3.3 swh.loader.core-5.1.0 swh.loader.git-2.1.0 swh.loader.metadata-0.0.3 swh.model-6.6.1 swh.objstorage-2.0.1 swh.perfecthash-0.1.2 swh.scheduler-1.5.0 swh.storage-1.7.3 tenacity-8.1.0 testing.common.database-2.0.3 testing.postgresql-1.3.0 toml-0.10.2 tomli-2.0.1 typing_extensions-4.4.0 urllib3-1.26.14 vine-5.0.0 wadllib-1.3.6 wcwidth-0.2.6 wrapt-1.14.1 yarl-1.8.2 zipp-3.11.0
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: gunicorn in ./.local/lib/python3.10/site-packages (20.1.0)
Requirement already satisfied: setuptools>=3.0 in /usr/local/lib/python3.10/site-packages (from gunicorn) (65.5.1)
Removing intermediate container 7c09ab0c3117
---> 9d94f6f5877e
Step 10/15 : ENV SWH_CONFIG_FILENAME=/etc/swh/config.yml
---> Running in b91e68bbedc9
Removing intermediate container b91e68bbedc9
---> e1e2a98179f6
Step 11/15 : ENV SWH_WORKER_INSTANCE=loader
---> Running in 6c3fd3d1d57c
Removing intermediate container 6c3fd3d1d57c
---> ef351305264c
Step 12/15 : ENV CONCURRENCY=1
---> Running in 5e00a47dae31
Removing intermediate container 5e00a47dae31
---> 9ab4288e1edf
Step 13/15 : ENV MAX_TASKS_PER_CHILD=1
---> Running in be8cf81adede
Removing intermediate container be8cf81adede
---> ee53c0429f3a
Step 14/15 : ENV LOGLEVEL=INFO
---> Running in 4cc3be6e6963
Removing intermediate container 4cc3be6e6963
---> 28ecf4b825b1
Step 15/15 : ENTRYPOINT "/opt/swh/entrypoint.sh"
---> Running in de9df6e215ba
Removing intermediate container de9df6e215ba
---> da151a471177
Successfully built da151a471177
Successfully tagged container-registry.softwareheritage.org/swh/infra/swh-apps/loader_git:20230124.1

Refs. sysadm-environment#4724 (closed)

[1] ardumont/swh-apps@63e9cf33

[2] ardumont/swh-apps@d8917828

Edited by Antoine R. Dumont

Merge request reports