launchpad: Allow bzr origins listing
Related to swh-loader-bzr#3945 (closed)
Test Plan
tox
and docker is happy too:
Lister log (with a pprint on the collection it lists)
swh-lister_1 | [2022-02-16 16:33:50,268: INFO/MainProcess] Task swh.lister.launchpad.tasks.FullLaunchpadLister[f3e3f3aa-8f4a-4e2c-8821-facd4952e53e] received
swh-lister_1 | ('git', <lazr.restfulclient.resource.Collection object at 0x7ff52ebef290>)
swh-lister_1 | ('bzr', <lazr.restfulclient.resource.Collection object at 0x7ff52d1a9450>)
...
- [1] scheduler db in docker, it's listing new bzr origins (no bzr prior to the run):
17:58:57 swh-scheduler@localhost:5433=# select now(), count(*) from listed_origins where visit_type='bzr';
+-------------------------------+-------+
| now | count |
+-------------------------------+-------+
| 2022-02-16 16:59:07.267496+00 | 21000 |
+-------------------------------+-------+
(1 row)
Time: 5.236 ms
17:59:07 swh-scheduler@localhost:5433=# select now(), count(*) from listed_origins where visit_type='bzr';
+-------------------------------+-------+
| now | count |
+-------------------------------+-------+
| 2022-02-16 16:59:46.291344+00 | 22000 |
+-------------------------------+-------+
(1 row)
Time: 5.584 ms
18:00:23 swh-scheduler@localhost:5433=# select now(), * from listed_origins where visit_type='bzr' order by last_update desc limit 1;
+-[ RECORD 1 ]-----------+-------------------------------------------------------------------------------+
| now | 2022-02-16 17:00:34.779024+00 |
| lister_id | 9290a3f8-6896-47ea-81b3-e3adc9df21be |
| url | https://code.launchpad.net/~ubuntu-branches/ubuntu/karmic/libvncserver/karmic |
| visit_type | bzr |
| extra_loader_arguments | {} |
| enabled | t |
| first_seen | 2022-02-16 16:59:53.309055+00 |
| last_seen | 2022-02-16 16:59:53.309055+00 |
| last_update | 2009-06-27 00:56:06.928908+00 |
+------------------------+-------------------------------------------------------------------------------+
Time: 11.362 ms
After an incremental run:
19:59:26 swh-scheduler@localhost:5433=# select now(), count(*) from listed_origins where visit_type='bzr';
+-------------------------------+--------+
| now | count |
+-------------------------------+--------+
| 2022-02-17 08:18:45.201575+00 | 168000 |
+-------------------------------+--------+
(1 row)
Time: 20.536 ms
09:18:45 swh-scheduler@localhost:5433=# select * from listers where name='launchpad';
+-[ RECORD 1 ]--+-----------------------------------------------------------------------------------------------------------------------+
| id | 9290a3f8-6896-47ea-81b3-e3adc9df21be |
| name | launchpad |
| instance_name | launchpad |
| created | 2022-02-16 16:24:45.466527+00 |
| current_state | {"bzr_date_last_modified": "2009-09-10T10:21:25+00:00", "git_date_last_modified": "2022-02-16T19:07:16.970183+00:00"} |
| updated | 2022-02-16 21:25:33.628123+00 |
+---------------+-----------------------------------------------------------------------------------------------------------------------+
Time: 0.414 ms
Migrated from D7193 (view on Phabricator)