docker: add the swh-winery-db and swh-winery services
Related to: T3533
It depends on swh-objstorage!127 (closed) and will fail until it is merged.
Test Plan
It can be tested from sources (assuming swh-objstorage is checked out with https://forge.softwareheritage.org/swh/devel/swh-objstorage!127) with an override like this:
version: "2.1"
services:
swh-winery:
volumes:
- "../swh-objstorage:/src/swh-objstorage"
and
cd docker ; docker-compose up swh-winery-db swh-winery
Migrated from D6834 (view on Phabricator)
Merge request reports
Activity
It depends on https://forge.softwareheritage.org/swh/devel/swh-objstorage!127 and will fail until it is merged. It can be tested from sources with an override like this:
version: "2.1" services: swh-winery: volumes: - "../swh-objstorage:/src/swh-objstorage"
Instead of defining a new service, could you provide an alternative docker-compose config file? This way, it can be used as to switch all services to use it, just by adding a CLI parameter. eg. we do this to replace the postgres storage backend with cassandra: https://docs.softwareheritage.org/devel/getting-started/using-docker.html#cassandra
! In !226 (closed), @vlorentz wrote: Instead of defining a new service, could you provide an alternative docker-compose config file? This way, it can be used as to switch all services to use it, just by adding a CLI parameter. eg. we do this to replace the postgres storage backend with cassandra: https://docs.softwareheritage.org/devel/getting-started/using-docker.html#cassandra
The winery is not intended as a replacement but as an additional service that will run along the existing swh-objstorage. The swh-objstorage may be configured to use swh-winery (or not) using the existing "remote" cls. Does your suggestion still stands knowing this? I'm not against it on principle (that's my first patch here, I have no experience or opinion really). I'm just making sure you got the context right before doing what you suggest.
I don't understand why. Currently, we have:
client --> objstorage pathslicing backend (port 5003) --> disk
. What you want to do is:client --> objstorage proxy (port 5003) --> objstorage winery backend (port 5012) --> winery --> ceph
, right? (objstorage winery backend
is what is launched bydocker/services/swh-winery/entrypoint.sh
in this diff)objstorage winery backend
must implement exactly the same interface asobjstorage pathslicing backend
, otherwise the proxy won't work. So why have the proxy in the first place instead ofclient --> objstorage winery backend --> winery --> ceph
?
Anyway, assuming we use an objstorage proxy, then we will need a dockerfile that changes the objstorage config from
{cls: "pathslicing", path: ...}
(the pathslicing backend) to{cls: "remote", url: "http://swh-winery:5012"}
(a proxy to the winery backend)Ah wait I got it,
docker/services/swh-winery/entrypoint.sh
launches winery itself, not an actual objstorage backend; you're only reusing the scafholding.Alright, let's do this and we'll remove the proxy later if it turns out to be unnecessary. Please allocate port 5012 in https://forge.softwareheritage.org/source/swh-docs/browse/master/docs/services.txt though.
! In !226 (closed), @vlorentz wrote: Ah wait I got it,
docker/services/swh-winery/entrypoint.sh
launches winery itself, not an actual objstorage backend; you're only reusing the scafholding.Exactly :-)
Alright, let's do this and we'll remove the proxy later if it turns out to be unnecessary. Please allocate port 5012 in https://forge.softwareheritage.org/source/swh-docs/browse/master/docs/services.txt though.
Oh, thanks for the hint. Done in swh-docs!279 (closed)
assigned to @vsellier
Closing, it's superseded by !281 (closed) to get rid of old fabricator references