[POC] Add keycloak service and sample SoftwareHeritage realm

That diff enables to test Keycloak integration in swh-web using the docker environment.

A sample configuration for a Keycloak realm named SoftwareHeritage will be loaded when the new keycloak service will start.

In that realm, a client named swh-web-api has been created whose purpose is to protect the access to the Software Heritage Web API.

For that client, three roles have been created that can be associated to users:

• normal-user
• partner-user
• staff-user

Users with role partner-user or staff-user have the permission throttling-exempted associated while users with role normal-user do not have any permissions.

In order to test users authentication and permissions in swh-web, a Python script will be executed when the associated docker service will start. That script takes care of creating the following users in the realm:

• admin (password: admin): the realm administrator with role staff-user
• johndoe (password: johndoe-swh): a user with role partner-user
• janedoe (password: janedoe-swh): a user with role normal-user

In order to authenticate a user when making calls to the swh web api, proceed as follow:

1. Get an access token by requesting the new endpoint /auth/token/access/ of the web api (let's test for the janedoe user)

$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=janedoe" -d "password=janedoe-swh" | jq '.'
{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiI0Y2U4NjRjNS00ZTI4LTQ3YjAtODdhYy1hNDUwZWUxYTdmYmIiLCJleHAiOjE1NzA4MDc0OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI2ZjExMWEwMy0zNTRiLTQ0NTctYmQyNS01MjAxZGUyMzlkMmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkphbmUgRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiamFuZWRvZSIsImdpdmVuX25hbWUiOiJKYW5lIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImphbmUuZG9lQGV4YW1wbGUub3JnIn0.fZCwPauPdNZeWrKxSgC9D8Chcxl2HSN_WaIrysQ7XH_ipePX04Skbscfd-uZsEKPYhR585Td9-5Eek-JkMxlbYTvEZi1wd1VuFk9xEH_dErs9Lh0paTztsCbhK9tOowl8TdcSWNpCG0E4p8eL9oXRHA07kb2P23cG0PAbXsg87B7f7NHB9ttKZCMAK4SERfh926UH2zTdSHAfIFqLyUqJ59i8mvdrB3W7yiDiMYYlmn7vsZ2uDtObYCtp35QZZlOuGp7ynRxbvSVHPOiJTjBOXoN74R4XEzSBbl3DGgtdWFRnmXv9VfEmn3VR9tl3rQMIq1IMRQ0q8Dl1IJK-iGw0A",
  "expires_in": 300,
  "refresh_expires_in": 1800,
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJhYjdiYjEzNi04NTg2LTQwOWUtYWUyNi02OWY4OTllMjRkNWIiLCJleHAiOjE1NzA4MDg5OTcsIm5iZiI6MCwiaWF0IjoxNTcwODA3MTk3LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjZmMTExYTAzLTM1NGItNDQ1Ny1iZDI1LTUyMDFkZTIzOWQyYiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6ImY0ZmUxMzRkLWZiNWQtNDc1NC04ZGE4LWU2NjZhYzU4ZGZiMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.xLSHCv2zCkDd3u_AYtYX6tZbvzEkB7N2dK82tTnS9kE"
}

2. Then use the access token to perform authenticated calls to the web api

$ export TOKEN=<access_token>
$ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Fri, 11 Oct 2019 15:22:15 GMT
Connection: keep-alive
Content-Type: application/json
Vary: Accept, Cookie
Allow: OPTIONS, GET, HEAD, OPTIONS
X-RateLimit-Limit: 120
X-RateLimit-Remaining: 119
X-RateLimit-Reset: 1570807365
X-Frame-Options: SAMEORIGIN
Content-Length: 2

{}

We can see that the rate limiting headers are present in the api response as janedoe does not have any specific permission.

Let's perform the same operations with the johndoe user.

$ curl -X POST http://localhost:5004/api/1/auth/token/access/ -d "username=johndoe" -d "password=johndoe-swh" | jq '.'  
{
  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiIwOTc3N2Q3MS0xMzdmLTQyZjktYmJiZi1iYTkzNmQ5M2I1YTUiLCJleHAiOjE1NzA4MDc3NTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiIyNWUyNzNjMC02ZGY4LTQ1NjYtYmFkYS03ODE3MjM0ZTc3ZDgiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsInBhcnRuZXItdXNlciIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkpvaG4gRG9lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiam9obmRvZSIsImdpdmVuX25hbWUiOiJKb2huIiwiZmFtaWx5X25hbWUiOiJEb2UiLCJlbWFpbCI6ImpvaG4uZG9lQGV4YW1wbGUub3JnIn0.kbPLtlf2j6ZteG2xuoBXOs1kMQqYgK4eynIL3J3A31h5mAGYkVP7-ad_fwiTElRL7T_RLI-9Tu_MhOPTb5kjP5_sPdM1iFHwiXbA6rcVBCp4qTACFHDwAi6FQCzVXNFQvb2y3GJLrpFNbUUE9EPR6082rZpu-8q9iuuhe91k82jaB5UApBKv7AFU8Uf07lIUsXIda3mGTusaltCBs2B5Tu5roToR7PvlOebWVP5ufUb6UwYAr6cTDcURDwLV3wg1E2OUs6bW3LfECzmyVFlTUJ9lEm3StIYsK8kqslHroTvXBmUgZhg21BlI9uJfYLZZ69gMvk36spDwPArkSXNWqg",
  "expires_in": 300,
  "refresh_expires_in": 1800,
  "refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiIyMDY1ZWM5MC00MTkwLTRlNzYtOGIxNi1mNzBlNjE3MGY5NTQiLCJleHAiOjE1NzA4MDkyNTQsIm5iZiI6MCwiaWF0IjoxNTcwODA3NDU0LCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjI1ZTI3M2MwLTZkZjgtNDU2Ni1iYWRhLTc4MTcyMzRlNzdkOCIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjJlNjgwNzUwLWZlYjQtNDcwZC1hNTg0LWVjOGMwYTNhMWIzMyIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0IiwicGFydG5lci11c2VyIiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.gVN2wbq3tnGbBIn_T1Lg_LZpzu_nP0j1BV-ApS1mENs"
}
$ export TOKEN=<access_token>
$ curl -i -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" http://localhost:5004/api/1/stat/counters/HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Fri, 11 Oct 2019 15:25:42 GMT
Connection: keep-alive
Content-Type: application/json
Vary: Accept, Cookie
Allow: OPTIONS, GET, HEAD, OPTIONS
X-Frame-Options: SAMEORIGIN
Content-Length: 2

{}

We can now see the rate limiting headers are no more present as the johndoe user has the throttling-exempted permission.

To play with Keycloak realm configuration, you can log in as the admin user in the administration console reachable from http://localhost:5080/keycloak/auth/admin/SoftwareHeritage/console/index.html

The possibility to authenticate with an existing GithHub or GitLab account has also been added for testing purposes, either:

• through the browser by reaching that login page: http://localhost:5080/keycloak/auth/realms/SoftwareHeritage/account
• through the web api by exchanging a Github/Gitlab access token to a Keycloak one, for instance

curl -X POST http://localhost:5004/api/1/auth/token/exchange/ -d "issuer=gitlab" -d "token=<gitlab_acces_token>"{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURWdfR1o5d1FCXzdDZWk5bUZiYXJYemVJQXlBQ2tTSkVnY0x5Uk1WcVlnIn0.eyJqdGkiOiJmNzhlM2ViYy0xMjc2LTRjYzQtYjZkNi04YjdkN2Q2MDY3ZWMiLCJleHAiOjE1NzA4MDg4MjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoic3doLXdlYi1hcGkiLCJzdWIiOiI1YzBhNjJiZS04OWEyLTQxMmQtYjExYy0zZDNjODMxMWRkZTIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsImFjciI6IjEiLCJyZXNvdXJjZV9hY2Nlc3MiOnsic3doLXdlYi1hcGkiOnsicm9sZXMiOlsiZGVmYXVsdCIsIm5vcm1hbC11c2VyIl19fSwic2NvcGUiOiJzd2gtc2VydmljZXMgZW1haWwgcHJvZmlsZSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkFudG9pbmUgTGFtYmVydCIsInByZWZlcnJlZF91c2VybmFtZSI6ImFubGFtYmVydCIsImdpdmVuX25hbWUiOiJBbnRvaW5lIiwiZmFtaWx5X25hbWUiOiJMYW1iZXJ0IiwiZW1haWwiOiJhbnRvaW5lLmxhbWJlcnQzM0BnbWFpbC5jb20ifQ.QTv8yMgbScN2GsHwtsdbZjuQa7sg-dSsyL7oehQMGVsKRFBa9f9CskDczvfVPcIfgBr7LcGCfNxsYrO6yc83_OkiyEB9xed9vS5Kxnk9YY-iqVi11zlqEkj08o1hz0h-jUVmUHYugg3cK-XaFTlT2MywzCCZoNdf6DEtUl7f7u2BTLqWyWzpylnYvAI9UE86fNZmdZb233k7_cFe9eJ22nPkT5Muc7lys9-SZyNbF76349QKaoZTtfJ1FM4H9T0nDT-q0NIv8FcYVF43dXykiyUdMwZLD5G8-ARkMY8dfX5CAG5KmaaIX_CAZ__n3dJz80oGQDkWpnFSS4n0Sd-MeA","expires_in":300,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIyMTI2NzU3ZC1jYTVhLTQwODMtOGIzOS0xMjlmZDFiODNmNGYifQ.eyJqdGkiOiJjNjlmMDI2OC1iYmM1LTQ0ODgtYTY2OS1kOGMzY2Q4OGQ0OWQiLCJleHAiOjE1NzA4MTAzMjIsIm5iZiI6MCwiaWF0IjoxNTcwODA4NTIyLCJpc3MiOiJodHRwOi8va2V5Y2xvYWs6ODA4MC9hdXRoL3JlYWxtcy9Tb2Z0d2FyZUhlcml0YWdlIiwiYXVkIjoiaHR0cDovL2tleWNsb2FrOjgwODAvYXV0aC9yZWFsbXMvU29mdHdhcmVIZXJpdGFnZSIsInN1YiI6IjVjMGE2MmJlLTg5YTItNDEyZC1iMTFjLTNkM2M4MzExZGRlMiIsInR5cCI6IlJlZnJlc2giLCJhenAiOiJzd2gtd2ViLWFwaSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjNhMDY5NTFkLTg1ZTktNGNhNS1hMzI2LTc5NjM4ZWRjMjhiNCIsInJlc291cmNlX2FjY2VzcyI6eyJzd2gtd2ViLWFwaSI6eyJyb2xlcyI6WyJkZWZhdWx0Iiwibm9ybWFsLXVzZXIiXX19LCJzY29wZSI6InN3aC1zZXJ2aWNlcyBlbWFpbCBwcm9maWxlIn0.4d5EkQoPMGGudLgCzfxg37VkhnlrgTTQ5_8v7kQCHK0"}

Depends on swh-web!994 (closed)

Related swh/infra/sysadm-environment#2020 (closed)

---

Migrated from D2131 (view on Phabricator)