feat: Add Hex.pm lister

Hex is the package manager for Erlang ecosystem.

All the source code files are available in .tar format which can be ingested by a dedicated loader

Related to #4687 (closed)

---

Migrated from D8907 (view on Phabricator)

Build is green

Patch application report for D8907 (id=32108)

Rebasing onto f4aafe02...

First, rewinding head to replay your work on top of it...
Applying: feat: Add Hex.pm lister

Changes applied before test

commit 716fce14c3ba640fec3a3929a3029e3e20899aed
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Thu Dec 1 00:43:09 2022 +0530

    feat: Add Hex.pm lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/867/ for more details.

could you reorder the list so it's in alphabetic order? (and fix the typo in fedora's line)

ditto

You need to pass complete_qs to requests_mock to actually test the query string: https://requests-mock.readthedocs.io/en/latest/matching.html#query-strings (and then you'll need to add the sort param, because it's missing from tests).

Did you test incremental listings? the sort value looks wrong (should be updated_at instead of name), but if you change it, then pagination is unusable because it's offset-based. Looks like this API is too badly designed to support it.

Merge request was returned for changes

Let's see how this goes: https://github.com/hexpm/hexpm/pull/1168

but if you change it, then pagination is unusable because it's offset-based.

Exactly. That's why I went with sort=name (which changes less frequently than the latest package version release date)

Thanks for https://github.com/hexpm/hexpm/pull/1168. But there's one more issue. ?sort=updated_at actually works in DESC order only. So it will make search=updated_after:xyz ineffective on the next pages.

We need to choose one of the following options:

• add a new query param ?order=DESC
• replace updated_after with updated_before

order sounds best. Do you want to do it?

order sounds best. Do you want to do it?

Sure. Thanks!

Created these in Hex.pm official github repo:

• PR to add order param in /packages API
• Issue for missing inserted_at values in release metadata. A corresponding PR was created by the maintainer to fix this issue. This shall be helpful for the hex.pm loader.

mentioned in merge request !457 (merged)

replaced by !457 (merged)

closed