Skip to content

debian: Reimplement lister using new Lister API

Antoine Lambert requested to merge generated-differential-D4925-source into generated-differential-D4925-target

Port debian lister to swh.lister.pattern.Lister API.

The new implementation will produce one instance of ListedOrigin model per package, notably containing the set of parameters expected by the debian loader.

The lister is also stateful, meaning only new packages and those with new found versions since the last listing will be returned.

It has been successfully tested for Debian and Ubuntu distributions.

Porting to the new API was a little bit tricky as that lister is different from others based on querying a REST API. Here, each page corresponds to the list of package sources info related to a given debian suite (e.g. buster, bullseye) and component (e.g. main, universe). Thus a package name can be seen multiple times across pages so I had to ensure that the total number of listed origins was accurate in returned lister statistics.

Depends on swh-scheduler!213 (closed)

Closes #2979 (closed)

Migrated from D4925 (view on Phabricator)

Merge request reports