GraphQL: staging - Deploy version v0.0.10
Create a new staging release. Tag created: v0.0.10
New GraphiQL client is added with minimal SWH branding in the GraphQL server directly.
With this, swh-explorer is not a hard dependency for the GraphQL production.
No Config changes
Activity
changed milestone to %GraphQL API
added activity::Deployment environment: staging labels
mentioned in issue swh/devel/swh-graphql#4648 (closed)
assigned to @vsellier
added state:wip label
marked this issue as related to #4804 (closed)
mentioned in commit swh/infra/ci-cd/swh-charts@5e711f4c
mentioned in commit swh/infra/ci-cd/swh-charts@736f42b8
After deploying the new version, there is an issue with the client:
swh/graphql-5d65f69d49-4qc44[graphql]: [2023-03-13 12:45:50 +0000] [8] [ERROR] Exception in ASGI application swh/graphql-5d65f69d49-4qc44[graphql]: Traceback (most recent call last): swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi swh/graphql-5d65f69d49-4qc44[graphql]: result = await app( # type: ignore[func-returns-value] swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: return await self.app(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 333, in _sentry_patched_asgi_app swh/graphql-5d65f69d49-4qc44[graphql]: return await middleware(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 139, in _run_asgi3 swh/graphql-5d65f69d49-4qc44[graphql]: return await self._run_app(scope, lambda: self.app(scope, receive, send)) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 188, in _run_app swh/graphql-5d65f69d49-4qc44[graphql]: raise exc from None swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py", line 183, in _run_app swh/graphql-5d65f69d49-4qc44[graphql]: return await callback() swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/applications.py", line 120, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await self.middleware_stack(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call swh/graphql-5d65f69d49-4qc44[graphql]: return await old_call(app, scope, new_receive, new_send, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: raise exc swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive, _send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call swh/graphql-5d65f69d49-4qc44[graphql]: return await old_call(app, scope, new_receive, new_send, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 280, in _sentry_authenticationmiddleware_call swh/graphql-5d65f69d49-4qc44[graphql]: await old_call(self, scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call swh/graphql-5d65f69d49-4qc44[graphql]: return await old_call(app, scope, new_receive, new_send, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/authentication.py", line 48, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call swh/graphql-5d65f69d49-4qc44[graphql]: return await old_call(app, scope, new_receive, new_send, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 108, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: response = await self.dispatch_func(request, call_next) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/swh/graphql/middlewares/logger.py", line 26, in dispatch swh/graphql-5d65f69d49-4qc44[graphql]: response = await call_next(request) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next swh/graphql-5d65f69d49-4qc44[graphql]: raise app_exc swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive_or_disconnect, send_no_error) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 227, in _sentry_exceptionmiddleware_call swh/graphql-5d65f69d49-4qc44[graphql]: await old_call(self, scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 130, in _create_span_call swh/graphql-5d65f69d49-4qc44[graphql]: return await old_call(app, scope, new_receive, new_send, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: raise exc swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive, sender) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/routing.py", line 716, in __call__ swh/graphql-5d65f69d49-4qc44[graphql]: await route.handle(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle swh/graphql-5d65f69d49-4qc44[graphql]: await self.app(scope, receive, send) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app swh/graphql-5d65f69d49-4qc44[graphql]: response = await func(request) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py", line 401, in _sentry_async_func swh/graphql-5d65f69d49-4qc44[graphql]: return await old_func(*args, **kwargs) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/swh/graphql/client/view.py", line 6, in explorer_page swh/graphql-5d65f69d49-4qc44[graphql]: return templates.TemplateResponse("explorer.html", {"request": request}) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/templating.py", line 112, in TemplateResponse swh/graphql-5d65f69d49-4qc44[graphql]: template = self.get_template(name) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/starlette/templating.py", line 94, in get_template swh/graphql-5d65f69d49-4qc44[graphql]: return self.env.get_template(name) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/jinja2/environment.py", line 1010, in get_template swh/graphql-5d65f69d49-4qc44[graphql]: return self._load_template(name, globals) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/jinja2/environment.py", line 969, in _load_template swh/graphql-5d65f69d49-4qc44[graphql]: template = self.loader.load(self, name, self.make_globals(globals)) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/jinja2/loaders.py", line 126, in load swh/graphql-5d65f69d49-4qc44[graphql]: source, filename, uptodate = self.get_source(environment, name) swh/graphql-5d65f69d49-4qc44[graphql]: File "/opt/graphql/.local/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source swh/graphql-5d65f69d49-4qc44[graphql]: raise TemplateNotFound(template) swh/graphql-5d65f69d49-4qc44[graphql]: jinja2.exceptions.TemplateNotFound: explorer.htmlA rollback to the previous version was made until further investigations
This is an issue with a relative path to the HTML template. Let me find a solution
This diff should fix the issue:
diff --git a/swh/graphql/client/view.py b/swh/graphql/client/view.py index eb8b2e5..845123c 100644 --- a/swh/graphql/client/view.py +++ b/swh/graphql/client/view.py @@ -1,6 +1,13 @@ +# Copyright (C) 2023 The Software Heritage developers +# See the AUTHORS file at the top-level directory of this distribution +# License: GNU General Public License version 3, or any later version +# See top-level LICENSE file for more information + +import os + from starlette.templating import Jinja2Templates async def explorer_page(request): - templates = Jinja2Templates(directory="swh/graphql/client") + templates = Jinja2Templates(directory=os.path.dirname(__file__)) return templates.TemplateResponse("explorer.html", {"request": request})-
@anlambert thanks, trying that
fwiw I've updated the swh-environment configuration to be able to test in the docker environment (swh/devel/swh-environment!257 (merged))
-
@vsellier Please use the tag v0.0.91 for deployment
mentioned in commit vsellier/swh-environment@e186d843
mentioned in merge request swh/devel/swh-environment!257 (merged)
I've tested the v0.0.91 version in docker.
It seems the client ignores the uri part and it's sending the request to the webapp (/) instead of (/graphql)
swh-web_1 | [13/Mar/2023 13:42:09] [WARNING] Forbidden (CSRF token missing or incorrect.): / nginx_1 | 172.20.0.1 - - [13/Mar/2023:13:42:09 +0000] "POST / HTTP/1.1" 403 1206 "http://localhost:5080/graphql/" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/111.0" 0.022 swh-web_1 | 172.20.0.7 - - [13/Mar/2023:13:42:09 +0000] "POST / HTTP/1.0" 403 1206 "http://localhost:5080/graphql/" "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/111.0"I suppose it's due to this: https://gitlab.softwareheritage.org/swh/devel/swh-graphql/-/blob/v0.0.91/swh/graphql/client/explorer.html#L60
Edited by Vincent Sellier-
Starlette does some routing here. GET will serve the client template.
I suppose it's due to this: https://gitlab.softwareheritage.org/swh/devel/swh-graphql/-/blob/v0.0.91/swh/graphql/client/explorer.html#L60The client is posting back to /, it will be re-directed to /graphql in nginx, correct?
nope as the post is done on the client side, there is no way to know if it must go to
/or/graphql.The target should be computed according to the url on the client side.
Perhaps there is something interesting you can use in this documentation: https://www.starlette.io/requests/
-
ok What about moving the top level routing to starlette?
ie:
application = Starlette( routes=[ Route("/graphql", ariadne_app, methods=["POST"]), Route("/graphql", explorer_page, methods=["GET"]), ], middleware=middleware, )and in client
fetcher: GraphiQL.createFetcher({ url: '/graphql', }),You can still redirect any GET request to '/' to /graphql in nginx
-
swh/devel/swh-graphql!126 (merged)
I hope this will solve the probelm as the enitre routing is happeing inside the application. Will this complicate things at your end? @vsellier
You may still have to redirect any request to / (preferably only GET) to /graphql.
-
This will not work if you try to access the graphql service from the
http://localhost:5013URL when using the docker environment. You should rather get the graphql service URL using starlette Reverse URL lookups, which will detect the reverse proxy from the HTTP headers, and inject the service URL in the jinja2 template. -
This will not work if you try to access the graphql service from the http://localhost:5013 URL when using the docker environment.True, but why should that be avaialbe in http://localhost:5013?, it will be avaialbe in http://localhost:5013/graphql instead. With the current setup, we have routing in both ops and in the application. Isn't it better to keep this only in the application? (we have decided to host the app in /graphql anyway)
You should rather get the graphql service URL using starlette Reverse URL lookups, which will detect the reverse proxy from the HTTP headers, and inject the service URL in the jinja2 templateI can use the lookup to find the route for /graphql
-
True, but why should that be avaialbe in http://localhost:5013
All our internal services are rooted to http://localhost:50XX so we should do the same for consistency, the issue we are facing in the docker environment is related to the reverse proxy badly configured in the nginx configuration.
With the current setup, we have routing in both ops and in the application. Isn't it better to keep this only in the application? (we have decided to host the app in /graphql anyway)
The GraphQL service will be deployed behind a reverse proxy in production I guess so we need to properly handle that case in the GraphQL server in a generic way, meaning if we change the public mount point no change will be required in the
swh-graphqlsource code. -
I have updated the MR with url_for POST. But I think this will also fail to find the right POST URL.
Maybe the only dynamic option is something like suggested by @vsellier (window.location)
-
I manage to make it work the following way by hacking on
swh-graphql:diff --git a/swh/graphql/client/explorer.html b/swh/graphql/client/explorer.html index f07b897..e6a2d23 100644 --- a/swh/graphql/client/explorer.html +++ b/swh/graphql/client/explorer.html @@ -57,7 +57,7 @@ ReactDOM.render( React.createElement(GraphiQL, { fetcher: GraphiQL.createFetcher({ - url: '/', + url: "{{ graphql_url }}", }), query: DEFAULT_QUERY, defaultEditorToolsVisibility: true, diff --git a/swh/graphql/client/view.py b/swh/graphql/client/view.py index 845123c..bc721d9 100644 --- a/swh/graphql/client/view.py +++ b/swh/graphql/client/view.py @@ -3,11 +3,18 @@ # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information +from urllib.parse import urljoin + import os from starlette.templating import Jinja2Templates async def explorer_page(request): + url = str(request.url_for("graphql")) + if "x-script-name" in request.headers: + url = urljoin(url, request.headers["x-script-name"].lstrip("/")) templates = Jinja2Templates(directory=os.path.dirname(__file__)) - return templates.TemplateResponse("explorer.html", {"request": request}) + return templates.TemplateResponse( + "explorer.html", {"request": request, "graphql_url": url} + ) diff --git a/swh/graphql/server.py b/swh/graphql/server.py index 5a84f90..5fa1147 100644 --- a/swh/graphql/server.py +++ b/swh/graphql/server.py @@ -116,7 +132,7 @@ def make_app_from_configfile(): # Mount under a starlette application application = Starlette( routes=[ - Route("/", ariadne_app, methods=["POST"]), + Route("/", ariadne_app, methods=["POST"], name="graphql"), Route("/", explorer_page, methods=["GET"]), ], middleware=middlewareBut it seems
uvicorndoes not handle all proxy related headers properly so I had to concatenate the path stored in thex-script-nameheader myself.Maybe the only dynamic option is something like suggested by @vsellier (window.location)
Yes I think this is the right fix, much simpler !
Edited by Antoine Lambert -
ariadne is also using window.location
I will use the same
mentioned in merge request swh/devel/swh-graphql!126 (merged)
mentioned in commit swh/infra/ci-cd/swh-charts@7a561b7d
My bad, pypi continue to use
v0.0.91which make sense as it suppose to be more recent.
I've released a v0.0.92 version to be able to upgrade the docker image.Edited by Vincent Sellier
