Skip to content

Add gitiles lister

Tests in docker are happy [1].

Gitiles instance returns voluntarily a malformed json output (json prefixed with )]}'\n) [2]. The lister deals with it to properly parse the json response nonetheless. It drops the prefix and then parses the json.

If at some point, they drop this prefix to return json directly, the lister will be able to deal with it too. There are 2 tests one with 'standard' gitile format and another with standard json to account for both case.

Refs. swh/meta#5045 (closed)

[1]

2023-07-12 18:11:26 swh-scheduler@localhost:5433 λ select instance_name, count(*) from listed_origins lo inner join listers l on l.id=lo.lister_id where lister_id in (select id from listers where name='gitiles') group by instance_name;
+------------------------------+-------+
|        instance_name         | count |
+------------------------------+-------+
| apache.googlesource.com      |  2476 |
| webrtc.googlesource.com      |     8 |
| dart.googlesource.com        |   117 |
| kernel.googlesource.com      |  1207 |
| skia.googlesource.com        |    45 |
| r8.googlesource.com          |     3 |
| webkit.googlesource.com      |     1 |
| melange.googlesource.com     |     3 |
| swiftshader.googlesource.com |     2 |
| boringssl.googlesource.com   |     3 |
| android.googlesource.com     |  2541 |
| gn.googlesource.com          |     3 |
| gem5.googlesource.com        |    18 |
| cue.googlesource.com         |     3 |
| gerrit.googlesource.com      |   229 |
| aomedia.googlesource.com     |     7 |
| go.googlesource.com          |    48 |
| coral.googlesource.com       |   108 |
| chromium.googlesource.com    |  1839 |
| fuchsia.googlesource.com     |   406 |
| pdfium.googlesource.com      |     4 |
| gvisor.googlesource.com      |     1 |
+------------------------------+-------+
(22 rows)

Time: 4.976 ms
Edited by Antoine R. Dumont

Merge request reports

Loading