Skip to content

cli/origin: Make utils function compute only the required data

Antoine R. Dumont requested to merge improve-cli-check-ingestion into master

And let the caller function do the actual display. This also makes the utility function compute only the required data to be displayed (the listing data is optionally displayed so compute on-demand).

This also renames the 'list' bool to 'with_listing'. 'list' is a python keyword (it's ok to reuse but editors usually color it differently than plain function names and that can be confusing).

Behavior conserved:

$ time swh scheduler --config-file ~/.config/swh/scheduler.yml origin check-ingested-origins nixguix ngyro.com
Forge ngyro.com (nixguix) has 5801 scheduled ingests in the scheduler.
failed      : 289
None        : 2
not_found   : 462
successful  : 5048
total       : 5801
success rate: 87.02%
swh scheduler --config-file ~/.config/swh/scheduler.yml origin  nixguix   1.78s user 0.15s system 28% cpu 6.814 total
$ time swh scheduler --config-file ~/.config/swh/scheduler.yml origin check-ingested-origins -l nixguix ngyro.com
----------------------------------------------------------------------------------------------------------------------------------------------  -------------------  --------------------------------
ftp://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.xz                                                                                                successful           2024-02-08 04:39:23.941275+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.xz                                                                                                successful           2024-02-07 23:50:26.484864+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.xz                                                                                                successful           2024-02-08 08:56:14.142427+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.xz                                                                                                successful           2024-02-08 00:18:30.762865+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.xz                                                                                                successful           2024-02-07 23:59:05.212409+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.7.tar.xz                                                                                                successful           2024-02-08 03:22:35.817142+00:00
ftp://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.xz                                                                                                successful           2024-02-07 22:49:27.080809+00:00
...
http://www.x.org/releases/individual/xserver/xorg-server-1.20.4.tar.bz2                                                                         successful           2024-02-07 21:29:13.581224+00:00
http://www.x.org/releases/individual/xserver/xorg-server-1.20.5.tar.bz2                                                                         successful           2024-02-07 21:07:13.210124+00:00
http://znc.in/releases/archive/znc-1.7.3.tar.gz                                                                                                 successful           2024-02-07 22:01:21.859966+00:00
http://znc.in/releases/archive/znc-1.7.4.tar.gz                                                                                                 successful           2024-02-08 03:57:37.882209+00:00
http://znc.in/releases/archive/znc-1.7.5.tar.gz                                                                                                 failed               2024-02-06 16:44:18.993278+00:00

Forge ngyro.com (nixguix) has 5801 scheduled ingests in the scheduler.
failed      : 289
None        : 2
not_found   : 462
successful  : 5048
total       : 5801
success rate: 87.02%
swh scheduler --config-file ~/.config/swh/scheduler.yml origin  -l nixguix   2.32s user 0.17s system 32% cpu 7.752 total
Edited by Antoine R. Dumont

Merge request reports