Use keycloak authentication for the deposit

mentioned in issue #2714

mentioned in merge request !219 (closed)

mentioned in merge request swh-auth!1 (closed)

mentioned in merge request swh-auth!2 (closed)

mentioned in merge request swh-auth!3 (closed)

mentioned in merge request snippets!10 (closed)

mentioned in merge request !223 (closed)

mentioned in merge request swh-auth!6 (closed)

mentioned in merge request !231 (closed)

mentioned in merge request !232 (closed)

mentioned in merge request !399 (closed)

mentioned in merge request swh-environment!239 (closed)

mentioned in merge request swh-auth!45 (closed)

added Authentication SWORD deposit priority:Normal labels

Discussion on deposit sprint with @ardumont and @anlambert

login / mdp classic (not most secured)
authentication by token
create account for each application and give permissions depending on the account
to give permission for the deposit functionality an admin needs to change the account permission
we don't want to break the existing usage (login /mdp) => use login/mdp

you get the login/mdp in headers from clients,
the server will send it to keycloak authentication backend (which is used in swh-web
direct grant (Oauth) open ID connect

First step: change the existing clients and create their accounts (in keycloak) Test in Docker Second step: transfer to client's account in staging

direct grant (Oauth) open ID connect

Also known as "Resource Owner Password Credentials" [1]

(RFC [2] has an ascii graph detailing the mechanism)

Details:

deposit client provides credentials to the deposit server
deposit server forwards credentials to check the client authentication against keycloak
keycloak accepts connection and provides a token to the deposit server. The client is now connected.
deposit drops the client's credentials (no longer uses them).
As the deposit client is now connected, the deposit can proceed as usual.
Alternatively if the authentication fails (keycloak refuses connection), deposit forwards a 401 as before.
[1] https://www.keycloak.org/docs/latest/securing_apps/index.html#_resource_owner_password_credentials_flow
[2] https://tools.ietf.org/html/rfc6749#section-4.3

mentioned in commit 27efcccb

mentioned in commit swh-environment@11b01889

added state:wip label

mentioned in commit swh-environment@dabe1c91

assigned to @ardumont

mentioned in commit 4353a323

Currently, the deposit migration scripts are not tested. Which would have been hard to be serene about the future keycloak change.

So a new dependency grew in the test requirements and some tests got started to actually test some migrations. Dependency added in !219 (closed).

That module was not packaged for debian. It is now in the swh debian repository. And, it's our jenkins instance which does the build [1] (v1.1.0 [2]).

To ensure it works as expected with the deposit, an intermediary deposit release got specifically built to ensure the tests are now fine with that dependency (v0.11.1 [3]).

And it is (in one round for both, no less \o/).

unstable:

12:58:48  swh/deposit/tests_migration/test_migrations.py::test_migrations_20_rename_swhid_column_in_deposit_model PASSED [ 60%]
12:58:48  swh/deposit/tests_migration/test_migrations.py::test_migrations_21_add_origin_url_column_to_deposit_model PASSED [ 60%]

stable (backport):

13:00:40  swh/deposit/tests_migration/test_migrations.py::test_migrations_20_rename_swhid_column_in_deposit_model PASSED [ 99%]
13:00:41  swh/deposit/tests_migration/test_migrations.py::test_migrations_21_add_origin_url_column_to_deposit_model PASSED [100%]

So now, we shall be able to test our schema migrations in the deposit!

Starting with the new keycloak changes. So some delay to deliver this but for the good cause!